themetagsv1

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1490
• Rouge1: 0.4434
• Rouge2: 0.2049
• Rougel: 0.4334
• Gen Len: 12.4621

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 10
• eval_batch_size: 10
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Gen Len
No log	0.1	100	0.2118	0.3207	0.1074	0.3112	12.4598
No log	0.19	200	0.1978	0.3409	0.1145	0.3283	12.4621
No log	0.29	300	0.1849	0.3511	0.128	0.3421	12.4621
No log	0.38	400	0.1795	0.3778	0.1458	0.3697	12.4621
No log	0.48	500	0.1751	0.3797	0.1505	0.3696	12.4609
No log	0.57	600	0.1723	0.3909	0.1569	0.3816	12.4621
No log	0.67	700	0.1695	0.3911	0.1599	0.3851	12.4621
No log	0.76	800	0.1668	0.3922	0.1555	0.384	12.4621
No log	0.86	900	0.1636	0.3956	0.1585	0.3869	12.4621
No log	0.96	1000	0.1632	0.4037	0.1705	0.3961	12.4621
No log	1.05	1100	0.1610	0.4164	0.1807	0.4096	12.4621
No log	1.15	1200	0.1593	0.416	0.1789	0.409	12.4621
No log	1.24	1300	0.1583	0.4173	0.1839	0.4089	12.4621
No log	1.34	1400	0.1573	0.4123	0.1752	0.4049	12.4621
No log	1.43	1500	0.1561	0.4224	0.1861	0.4148	12.4621
No log	1.53	1600	0.1558	0.4179	0.1821	0.4091	12.4621
No log	1.63	1700	0.1542	0.4264	0.1861	0.4169	12.4621
No log	1.72	1800	0.1539	0.4323	0.1926	0.4229	12.4621
No log	1.82	1900	0.1526	0.4301	0.1917	0.4222	12.4621
No log	1.91	2000	0.1521	0.4326	0.1965	0.423	12.4621
No log	2.01	2100	0.1513	0.4309	0.1985	0.4226	12.4621
No log	2.1	2200	0.1512	0.4287	0.1907	0.4184	12.4621
No log	2.2	2300	0.1509	0.439	0.2	0.4302	12.4621
No log	2.29	2400	0.1512	0.4397	0.202	0.4307	12.4621
No log	2.39	2500	0.1506	0.4415	0.2068	0.4316	12.4621
No log	2.49	2600	0.1504	0.4426	0.2072	0.4338	12.4621
No log	2.58	2700	0.1500	0.4418	0.1994	0.4316	12.4621
No log	2.68	2800	0.1500	0.4413	0.202	0.4308	12.4621
No log	2.77	2900	0.1492	0.4392	0.2006	0.4297	12.4621
No log	2.87	3000	0.1492	0.443	0.206	0.4329	12.4621
No log	2.96	3100	0.1490	0.4434	0.2049	0.4334	12.4621

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2+cu121
• Datasets 2.16.1
• Tokenizers 0.15.2