mt5-aym-zero

This model is a fine-tuned version of alvations/mt5-aym-zero on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2035
• Chrf: 23.9937
• Bleu: 2.9371
• Gen Len: 17.1956

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 100000

Training results

Training Loss	Epoch	Step	Bleu	Chrf	Gen Len	Validation Loss
0.0658	5.13	20000	2.9963	23.9471	17.2839	0.1892
0.0633	10.26	40000	2.9953	24.004	17.2648	0.1932
0.059	15.39	60000	2.9368	24.0104	17.2207	0.1975
0.057	20.52	80000	2.9962	23.978	17.1725	0.2006
0.055	25.65	100000	0.2035	23.9937	2.9371	17.1956

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.0+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3