t-5-base-bertsum-500

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2994
• Rouge1: 0.6466
• Rouge2: 0.3657
• Rougel: 0.5798
• Rougelsum: 0.5798
• Wer: 0.5246
• Bleurt: -0.0784

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Wer	Bleurt
No log	0.13	250	1.4553	0.6223	0.3344	0.552	0.552	0.557	-0.4294
1.9648	0.27	500	1.3993	0.6301	0.3443	0.5613	0.5614	0.5467	-0.4022
1.9648	0.4	750	1.3747	0.6341	0.35	0.5661	0.5661	0.5402	-0.3802
1.4858	0.53	1000	1.3547	0.638	0.3533	0.5693	0.5693	0.5378	-0.0447
1.4858	0.66	1250	1.3431	0.639	0.3559	0.5715	0.5715	0.5342	-0.0292
1.4484	0.8	1500	1.3321	0.6406	0.3578	0.573	0.573	0.5322	-0.0292
1.4484	0.93	1750	1.3238	0.6418	0.3593	0.5747	0.5747	0.5306	-0.0784
1.4226	1.06	2000	1.3185	0.6433	0.3616	0.5762	0.5762	0.5281	-0.1084
1.4226	1.2	2250	1.3131	0.6442	0.3624	0.5775	0.5775	0.5277	-0.1084
1.3917	1.33	2500	1.3102	0.6453	0.3638	0.5783	0.5783	0.5266	-0.0784
1.3917	1.46	2750	1.3060	0.6458	0.3641	0.5788	0.5788	0.5256	-0.0292
1.4048	1.6	3000	1.3040	0.6461	0.3649	0.5792	0.5792	0.5253	-0.0784
1.4048	1.73	3250	1.3015	0.6463	0.3653	0.5796	0.5795	0.525	-0.0292
1.3803	1.86	3500	1.2999	0.6463	0.3654	0.5795	0.5795	0.5247	-0.0784
1.3803	1.99	3750	1.2994	0.6466	0.3657	0.5798	0.5798	0.5246	-0.0784

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2