flant5-amazon

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0716
• Rouge1: 16.211
• Rouge2: 5.3808
• Rougel: 15.788
• Rougelsum: 15.7869

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 3
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
1.8226	1.0	4999	1.9639	16.082	5.1085	15.6181	15.6148
1.7216	2.0	9998	1.9378	16.1156	5.1237	15.6217	15.6192
1.6741	3.0	14997	1.9296	16.3796	5.2765	15.9041	15.8982
1.6447	4.0	19996	1.9281	16.1864	5.2477	15.7235	15.7111
1.6441	5.0	24995	2.0763	16.1546	5.1955	15.7273	15.7153
1.6145	6.0	29994	2.0747	16.27	5.3388	15.838	15.8316
1.6011	7.0	34993	2.0714	16.1953	5.3381	15.7569	15.7502
1.5953	8.0	39992	2.0716	16.211	5.3808	15.788	15.7869

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1
• Datasets 2.15.0
• Tokenizers 0.15.0