mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0210
• Rouge1: 17.0885
• Rouge2: 8.4139
• Rougel: 16.6189
• Rougelsum: 16.7761

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.706	1.0	1209	3.2245	16.2371	7.969	15.6425	15.7165
3.6602	2.0	2418	3.0803	16.6646	7.7573	16.0524	16.1609
3.4306	3.0	3627	3.0504	17.9626	9.2086	17.4429	17.5335
3.3144	4.0	4836	3.0394	17.7522	8.4791	17.2721	17.36
3.2345	5.0	6045	3.0431	17.3159	8.5519	17.0148	17.0964
3.1684	6.0	7254	3.0299	17.4355	8.6873	17.0855	17.2222
3.1375	7.0	8463	3.0238	16.8874	8.3565	16.4901	16.6159
3.112	8.0	9672	3.0210	17.0885	8.4139	16.6189	16.7761

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.0+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3