t5_recommendation_jobs_skills_p2

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3964
• Rouge1: 56.9756
• Rouge2: 34.2311
• Rougel: 56.9100
• Rougelsum: 56.8854
• Gen Len: 3.7743

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.01
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	187	0.5018	43.2644	20.0376	43.2200	43.2499	3.5588
No log	2.0	375	0.4504	49.0121	26.3671	48.9428	48.9444	3.5626
0.8598	3.0	562	0.4304	51.2228	29.2167	51.1510	51.1381	3.6700
0.8598	4.0	750	0.4183	51.5133	28.5328	51.3758	51.3621	3.6700
0.8598	5.0	937	0.4106	53.6532	31.0763	53.5764	53.4992	3.6591
0.3439	6.0	1125	0.4010	52.7163	29.5176	52.5233	52.5780	3.7370
0.3439	7.0	1312	0.4027	54.6573	32.0853	54.5163	54.5007	3.6591
0.2889	8.0	1500	0.3963	54.5537	31.8771	54.4623	54.4597	3.6475
0.2889	9.0	1687	0.3952	55.0573	32.2229	54.9448	54.9567	3.6514
0.2889	10.0	1875	0.3907	55.0968	32.9791	55.0473	55.0184	3.7089
0.248	11.0	2062	0.3915	56.5185	34.3867	56.4045	56.4487	3.6918
0.248	12.0	2250	0.3942	57.3052	34.2798	57.2348	57.2058	3.7689
0.248	13.0	2437	0.3972	55.5294	33.1886	55.4932	55.4813	3.7214
0.2203	14.0	2625	0.3939	55.9577	33.3766	55.8957	55.8786	3.7479
0.2203	14.96	2805	0.3964	56.9756	34.2311	56.9100	56.8854	3.7743

Framework versions

• Transformers 4.27.0
• Pytorch 2.1.2
• Datasets 2.8.0
• Tokenizers 0.13.3