t5-small-final_combined_dataset

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1478
• Rouge1: 21.6168
• Rouge2: 19.898
• Rougel: 21.5524
• Rougelsum: 21.5762
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	24	0.9771	22.1616	19.8054	20.8289	21.8465	19.0
No log	2.0	48	0.5761	21.3116	18.9119	19.847	20.8814	19.0
No log	3.0	72	0.4177	21.6168	19.898	21.5524	21.5762	19.0
No log	4.0	96	0.3053	21.6168	19.898	21.5524	21.5762	19.0
No log	5.0	120	0.2355	21.6168	19.898	21.5524	21.5762	19.0
No log	6.0	144	0.1961	21.6168	19.898	21.5524	21.5762	19.0
No log	7.0	168	0.1718	21.6168	19.898	21.5524	21.5762	19.0
No log	8.0	192	0.1577	21.6168	19.898	21.5524	21.5762	19.0
No log	9.0	216	0.1497	21.6168	19.898	21.5524	21.5762	19.0
No log	10.0	240	0.1478	21.6168	19.898	21.5524	21.5762	19.0

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.0+cpu
• Datasets 2.17.0
• Tokenizers 0.15.2