pegasus-samsum

This model is a fine-tuned version of google/pegasus-cnn_dailymail on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 6.0105

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
10.6079	0.0109	10	10.7159
10.5191	0.0217	20	10.6397
10.5539	0.0326	30	10.5210
10.5117	0.0434	40	10.3669
10.3111	0.0543	50	10.1941
10.2	0.0652	60	10.0230
10.1121	0.0760	70	9.8584
10.0677	0.0869	80	9.7129
9.7897	0.0977	90	9.5781
9.744	0.1086	100	9.4538
9.533	0.1195	110	9.3431
9.5248	0.1303	120	9.2552
9.3331	0.1412	130	9.1575
9.2551	0.1520	140	9.0751
9.2382	0.1629	150	8.9993
9.1323	0.1738	160	8.9287
9.0574	0.1846	170	8.8628
9.0137	0.1955	180	8.7964
8.9097	0.2064	190	8.7340
8.8268	0.2172	200	8.6765
8.7116	0.2281	210	8.6173
8.7483	0.2389	220	8.5521
8.6252	0.2498	230	8.4884
8.5844	0.2607	240	8.4275
8.4614	0.2715	250	8.3626
8.4375	0.2824	260	8.2901
8.445	0.2932	270	8.2102
8.2966	0.3041	280	8.1135
8.0934	0.3150	290	8.0113
8.1551	0.3258	300	7.8961
8.0542	0.3367	310	7.7751
8.0183	0.3475	320	7.6504
7.9352	0.3584	330	7.5371
7.7615	0.3693	340	7.4084
7.6484	0.3801	350	7.2715
7.5845	0.3910	360	7.1340
7.4799	0.4018	370	7.0563
7.3388	0.4127	380	6.9495
7.1078	0.4236	390	6.8582
7.0819	0.4344	400	6.7707
7.0465	0.4453	410	6.6897
6.9038	0.4561	420	6.6184
6.9359	0.4670	430	6.5533
6.8038	0.4779	440	6.4962
6.8648	0.4887	450	6.4452
6.7589	0.4996	460	6.3966
6.6804	0.5105	470	6.3588
6.5603	0.5213	480	6.3247
6.6002	0.5322	490	6.2937
6.59	0.5430	500	6.2665
6.5103	0.5539	510	6.2434
6.4911	0.5648	520	6.2210
6.4606	0.5756	530	6.2054
6.5193	0.5865	540	6.1870
6.4794	0.5973	550	6.1724
6.4579	0.6082	560	6.1598
6.3855	0.6191	570	6.1482
6.3071	0.6299	580	6.1367
6.4043	0.6408	590	6.1279
6.354	0.6516	600	6.1188
6.4038	0.6625	610	6.1114
6.3475	0.6734	620	6.1008
6.257	0.6842	630	6.0958
6.4359	0.6951	640	6.0872
6.2238	0.7059	650	6.0820
6.3904	0.7168	660	6.0754
6.2488	0.7277	670	6.0706
6.2648	0.7385	680	6.0644
6.303	0.7494	690	6.0601
6.3133	0.7602	700	6.0553
6.3229	0.7711	710	6.0516
6.3165	0.7820	720	6.0469
6.3353	0.7928	730	6.0438
6.2581	0.8037	740	6.0391
6.2688	0.8146	750	6.0361
6.2193	0.8254	760	6.0342
6.2247	0.8363	770	6.0305
6.1711	0.8471	780	6.0284
6.3126	0.8580	790	6.0259
6.3182	0.8689	800	6.0239
6.2298	0.8797	810	6.0214
6.287	0.8906	820	6.0198
6.2472	0.9014	830	6.0181
6.205	0.9123	840	6.0165
6.2359	0.9232	850	6.0147
6.3013	0.9340	860	6.0135
6.2035	0.9449	870	6.0129
6.2529	0.9557	880	6.0122
6.2043	0.9666	890	6.0114
6.2785	0.9775	900	6.0110
6.3018	0.9883	910	6.0106
6.1616	0.9992	920	6.0105

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.0+cu121
• Datasets 3.1.0
• Tokenizers 0.19.1