Inconsistent outputs
#2
by
fernandofernandes
- opened
Hi! I was able to load the models (base, large and xl), but it seems all of them are generating very inconsistent outputs (I'm not even testing on other languages than english). Any ideas?
These models are trained on updated mC4 corpus and has the same limitation of producing inconsistent outputs in their raw form as that of mT5 model.
So, I suggest training these models on a downstream fine-tuning corpus of your desired language and it should work fine.