roberta-large-fomc_long

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8275
• Accuracy: 0.6822

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.0083	1	1.1053	0.2733
1.0762	0.2149	26	1.0661	0.4636
1.0904	0.4215	51	1.0652	0.4636
1.0903	0.6281	76	1.0493	0.4656
1.0416	0.8347	101	1.0238	0.4980
0.9313	1.0	121	0.8957	0.5668
0.9313	1.0413	126	0.9420	0.5567
0.9943	1.2479	151	0.8193	0.6316
0.8029	1.4545	176	0.7896	0.6518
0.7335	1.6612	201	0.8053	0.6660
0.763	1.8678	226	0.7800	0.6640
0.7384	2.0	242	0.8398	0.6377
0.7316	2.0744	251	0.8587	0.6741
0.5971	2.2810	276	0.8520	0.6619
0.7952	2.4876	301	0.7661	0.6862
0.632	2.6942	326	0.7477	0.6640
0.5979	2.9008	351	0.9390	0.6215
0.5995	3.0	363	0.8275	0.6822
0.7325	3.1074	376	0.7512	0.6741
0.5238	3.3140	401	0.8282	0.6923
0.5401	3.5207	426	0.8515	0.6802
0.5937	3.7273	451	0.8372	0.6802
0.521	3.9339	476	1.0131	0.6518
0.6484	4.0	484	0.8845	0.6235
0.4641	4.1405	501	1.1492	0.6700
0.4919	4.3471	526	0.7645	0.7045
0.47	4.5537	551	0.9051	0.6842
0.4698	4.7603	576	0.8752	0.6964
0.6327	4.9669	601	0.8473	0.6721
0.6327	5.0	605	1.1093	0.6680
0.37	5.1736	626	1.0581	0.6903
0.3295	5.3802	651	0.9647	0.6842
0.4251	5.5868	676	0.9839	0.7004
0.4478	5.7934	701	0.9300	0.6964
0.4365	6.0	726	1.0642	0.7206
0.4365	6.0	726	1.0642	0.7206
0.239	6.2066	751	1.3570	0.6680
0.3339	6.4132	776	1.0710	0.6923
0.2864	6.6198	801	1.0177	0.6741
0.5973	6.8264	826	1.3977	0.6741
0.2812	7.0	847	1.0341	0.6964
0.325	7.0331	851	1.1641	0.6741
0.2835	7.2397	876	1.2173	0.6923
0.2406	7.4463	901	1.4326	0.6943
0.1369	7.6529	926	1.6347	0.6802
0.2019	7.8595	951	1.2877	0.6862
0.277	8.0	968	1.3664	0.6964
0.2004	8.0661	976	1.4982	0.7105
0.168	8.2727	1001	1.7011	0.7004
0.133	8.4793	1026	1.8177	0.7045
0.2772	8.6860	1051	1.4516	0.7045
0.0536	8.8926	1076	1.6896	0.7146
0.2335	9.0	1089	1.6829	0.7045
0.0846	9.0992	1101	1.9997	0.7085
0.0468	9.3058	1126	2.2480	0.6842
0.1376	9.5124	1151	1.9996	0.6964
0.1422	9.7190	1176	1.5541	0.7045
0.0717	9.9256	1201	1.8728	0.6822
0.125	10.0	1210	1.8979	0.7045
0.0339	10.1322	1226	1.9404	0.7146
0.0581	10.3388	1251	2.0144	0.6903
0.0804	10.5455	1276	2.1959	0.7004
0.1289	10.7521	1301	2.1261	0.6984
0.1011	10.9587	1326	2.1063	0.7024
0.0841	11.0	1331	2.1062	0.7045
0.0579	11.1653	1351	2.1912	0.7146
0.0383	11.3719	1376	2.3198	0.7004
0.0322	11.5785	1401	2.3495	0.6984
0.0579	11.7851	1426	2.2680	0.7004
0.0575	11.9917	1451	2.3905	0.6842
0.0575	12.0	1452	2.3978	0.6822
0.0003	12.1983	1476	2.4618	0.6903
0.0029	12.4050	1501	2.4325	0.6923
0.0638	12.6116	1526	2.4757	0.6862
0.0196	12.8182	1551	2.5483	0.6802
0.0731	13.0	1573	2.4884	0.6822
0.0731	13.0248	1576	2.4746	0.6862
0.0002	13.2314	1601	2.4790	0.6923
0.0002	13.4380	1626	2.5076	0.6822
0.0677	13.6446	1651	2.4820	0.6862
0.0002	13.8512	1676	2.4739	0.6903
0.0172	14.0	1694	2.4303	0.6923
0.0002	14.0579	1701	2.4298	0.6923
0.0002	14.2645	1726	2.4557	0.7004
0.0002	14.4711	1751	2.4311	0.7004
0.0504	14.6777	1776	2.4225	0.7024
0.0003	14.8843	1801	2.4239	0.7024
0.0342	15.0	1815	2.4238	0.7024

Framework versions

• Transformers 4.40.2
• Pytorch 1.12.0
• Datasets 2.19.1
• Tokenizers 0.19.1