t5-small-finetuned-laws_articles

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.4459
• Rouge1: 28.5123
• Rouge2: 10.7847
• Rougel: 23.1779
• Rougelsum: 23.1054
• Gen Len: 18.7143

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	87	3.0316	27.6181	11.4154	23.0601	23.0895	18.6494
No log	2.0	174	2.8530	28.2521	11.5239	23.2405	23.2335	18.5325
No log	3.0	261	2.7724	28.1619	10.4465	22.4885	22.5004	18.4545
No log	4.0	348	2.7206	27.6965	10.4247	22.5084	22.504	18.5714
No log	5.0	435	2.6820	28.0781	10.4734	23.0002	22.9096	18.5455
3.0366	6.0	522	2.6535	28.2097	10.5925	23.4952	23.474	18.4675
3.0366	7.0	609	2.6330	27.9031	10.4349	23.1427	23.1499	18.5325
3.0366	8.0	696	2.6191	27.7764	10.1563	22.7929	22.6971	18.4286
3.0366	9.0	783	2.5998	27.2375	9.7613	22.2731	22.2206	18.2727
3.0366	10.0	870	2.5866	27.2295	9.6843	22.5222	22.3697	18.3377
3.0366	11.0	957	2.5723	26.7174	9.3882	22.2336	22.0906	18.3377
2.7071	12.0	1044	2.5651	27.4541	10.5671	22.7291	22.6063	18.2468
2.7071	13.0	1131	2.5539	27.1483	9.9641	22.252	22.0612	18.2857
2.7071	14.0	1218	2.5451	27.4547	10.3266	22.6335	22.4141	18.3247
2.7071	15.0	1305	2.5366	27.3057	10.0068	22.5939	22.4321	18.3377
2.7071	16.0	1392	2.5291	27.0093	9.8964	22.1854	22.0594	18.4416
2.7071	17.0	1479	2.5210	27.4076	9.8442	22.4911	22.3641	18.3117
2.5937	18.0	1566	2.5130	27.8067	10.1345	22.5757	22.4506	18.3896
2.5937	19.0	1653	2.5086	28.5205	10.8396	22.902	22.7628	18.4416
2.5937	20.0	1740	2.5043	28.1191	10.3793	22.6607	22.5025	18.5584
2.5937	21.0	1827	2.4971	28.0615	10.3535	22.7174	22.5795	18.6104
2.5937	22.0	1914	2.4935	28.0575	10.2809	22.9132	22.7487	18.5974
2.5208	23.0	2001	2.4886	27.8841	10.3683	22.66	22.5543	18.7532
2.5208	24.0	2088	2.4851	27.5211	9.9966	22.5687	22.4658	18.6753
2.5208	25.0	2175	2.4823	27.7727	10.2515	22.7401	22.5633	18.5974
2.5208	26.0	2262	2.4785	28.042	10.4358	22.9115	22.7964	18.7273
2.5208	27.0	2349	2.4731	27.6472	10.2882	22.6181	22.4979	18.6364
2.5208	28.0	2436	2.4714	27.8088	10.4361	22.885	22.7203	18.6623
2.4649	29.0	2523	2.4694	27.73	10.5222	22.7567	22.6331	18.7013
2.4649	30.0	2610	2.4660	27.6279	10.377	22.4668	22.4061	18.7013
2.4649	31.0	2697	2.4649	27.3504	10.1574	22.224	22.1656	18.7013
2.4649	32.0	2784	2.4639	27.3448	9.9756	22.1962	22.1349	18.7013
2.4649	33.0	2871	2.4609	27.3083	10.0656	22.2803	22.2228	18.7143
2.4649	34.0	2958	2.4593	27.51	10.0941	22.3699	22.3382	18.6623
2.4181	35.0	3045	2.4562	27.5434	10.1665	22.5002	22.4415	18.7143
2.4181	36.0	3132	2.4545	27.5664	10.0953	22.5101	22.4741	18.6364
2.4181	37.0	3219	2.4530	27.5874	10.333	22.5099	22.4576	18.7013
2.4181	38.0	3306	2.4522	27.3705	10.1022	22.4695	22.3849	18.6623
2.4181	39.0	3393	2.4512	27.8864	10.2695	22.834	22.759	18.6753
2.4181	40.0	3480	2.4504	27.6797	10.2472	22.5788	22.5353	18.7013
2.4038	41.0	3567	2.4495	27.6797	10.2472	22.5788	22.5353	18.7013
2.4038	42.0	3654	2.4492	27.1595	9.6186	22.1003	22.0487	18.6883
2.4038	43.0	3741	2.4490	27.8341	10.2826	22.8117	22.7244	18.7013
2.4038	44.0	3828	2.4479	28.0005	10.4067	22.8833	22.8177	18.7013
2.4038	45.0	3915	2.4475	28.0811	10.4117	22.8975	22.8276	18.7013
2.3785	46.0	4002	2.4471	28.0811	10.4117	22.8975	22.8276	18.7013
2.3785	47.0	4089	2.4466	28.4435	10.793	23.0757	23.0082	18.7013
2.3785	48.0	4176	2.4463	28.5123	10.7847	23.1779	23.1054	18.7143
2.3785	49.0	4263	2.4460	28.2551	10.653	22.9815	22.8896	18.7143
2.3785	50.0	4350	2.4459	28.5123	10.7847	23.1779	23.1054	18.7143

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.0+cu121
• Datasets 2.14.7
• Tokenizers 0.14.1