T5-model-1-feedback-0611-4e

This model is a fine-tuned version of theojolliffe/T5-model-1-feedback-1109 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1221
• Rouge1: 93.9996
• Rouge2: 89.632
• Rougel: 93.4347
• Rougelsum: 93.4442
• Gen Len: 15.0185

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.2124	1.0	2571	0.1594	93.667	89.1053	93.052	93.0963	15.1218
0.1531	2.0	5142	0.1413	94.2879	89.9026	93.6752	93.7059	15.0332
0.1347	3.0	7713	0.1263	94.2554	90.1846	93.726	93.7635	15.0369
0.142	4.0	10284	0.1221	93.9996	89.632	93.4347	93.4442	15.0185

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.1