t5-fine-tuned-with-yake-keywords

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7156
• Rouge1: 25.7203
• Rouge2: 11.4526
• Rougel: 20.9169
• Rougelsum: 24.2292
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.408	1.0	604	1.3840	25.8101	11.1365	20.9036	24.1453	18.992
1.2639	2.0	1208	1.3853	25.3277	11.0463	20.7012	23.7892	18.994
1.2049	3.0	1812	1.3945	25.7271	11.437	20.9819	24.2143	18.99
1.1008	4.0	2416	1.4060	25.4068	10.9722	20.3943	23.7708	18.994
1.0033	5.0	3020	1.4295	25.0656	10.7868	20.1689	23.4475	18.992
0.9373	6.0	3624	1.4425	25.5643	11.1203	20.5504	23.8398	19.0
0.9054	7.0	4228	1.4600	25.6599	11.3601	20.7753	24.0562	19.0
0.8699	8.0	4832	1.4881	25.8057	11.6887	20.9235	24.2292	19.0
0.8253	9.0	5436	1.5049	25.5996	11.2568	20.6128	23.9682	19.0
0.7706	10.0	6040	1.5304	25.5366	11.2432	20.5506	23.8558	19.0
0.7406	11.0	6644	1.5443	25.5608	11.3835	20.7609	23.9875	19.0
0.7122	12.0	7248	1.5608	25.4136	11.2439	20.6605	23.9639	19.0
0.6983	13.0	7852	1.5777	26.1446	11.7678	21.1421	24.587	19.0
0.6506	14.0	8456	1.5987	26.0036	11.8495	21.1601	24.5154	19.0
0.627	15.0	9060	1.6229	25.7103	11.5427	20.9998	24.2446	19.0
0.625	16.0	9664	1.6448	25.4558	11.4994	20.82	23.9969	19.0
0.5918	17.0	10268	1.6613	25.8231	11.6828	20.9607	24.2662	19.0
0.577	18.0	10872	1.6786	25.9623	11.6631	21.0312	24.4392	19.0
0.5917	19.0	11476	1.6810	25.9725	11.5125	20.9832	24.4222	19.0
0.5543	20.0	12080	1.6896	25.8324	11.5314	20.9358	24.2487	19.0
0.5551	21.0	12684	1.7021	25.753	11.5405	21.022	24.2551	19.0
0.5334	22.0	13288	1.7114	25.7559	11.4908	20.9003	24.2192	19.0
0.5301	23.0	13892	1.7137	25.7977	11.4324	20.8802	24.241	19.0
0.5333	24.0	14496	1.7150	25.7444	11.4107	20.8701	24.205	19.0
0.5216	25.0	15100	1.7156	25.7203	11.4526	20.9169	24.2292	19.0

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3