mt5-small-finetuned-amazon-en-de

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5620
• Rouge1: 19.3915
• Rouge2: 10.59
• Rougel: 18.7811
• Rougelsum: 18.9784

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.8704	1.0	651	2.5780	17.9954	9.8425	17.421	17.5202
2.8213	2.0	1302	2.5719	18.3944	9.9329	17.8166	17.9457
2.7672	3.0	1953	2.5643	17.4605	9.7057	16.9978	17.0939
2.7311	4.0	2604	2.5633	19.5332	11.0145	19.0127	19.1008
2.6985	5.0	3255	2.5672	19.3155	10.1678	18.6334	18.8022
2.6644	6.0	3906	2.5589	19.3282	10.3801	18.8039	18.9073
2.654	7.0	4557	2.5540	19.2307	10.4068	18.6708	18.896
2.6318	8.0	5208	2.5620	19.3915	10.59	18.7811	18.9784

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1