roberta-large-fomc

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7874
• Accuracy: 0.6660

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.0083	1	1.0582	0.4980
1.0574	0.2149	26	1.0428	0.4980
1.0702	0.4215	51	1.0500	0.4980
1.1065	0.6281	76	1.0374	0.4980
1.0241	0.8347	101	1.0391	0.4980
1.0324	1.0	121	1.0191	0.4980
1.0324	1.0413	126	1.0097	0.4980
0.9751	1.2479	151	1.0542	0.4737
1.0134	1.4545	176	0.9746	0.5931
0.9276	1.6612	201	0.8633	0.5648
0.8469	1.8678	226	0.7729	0.6538
0.7992	2.0	242	0.7874	0.6660
0.8853	2.0744	251	0.8597	0.6680
0.6466	2.2810	276	0.7767	0.6498
0.778	2.4876	301	1.0588	0.6498
0.7202	2.6942	326	0.7493	0.6721
0.7108	2.9008	351	0.8892	0.6397
0.6354	3.0	363	0.8265	0.6579
0.7704	3.1074	376	0.7833	0.6781
0.6867	3.3140	401	0.9702	0.6478
0.6973	3.5207	426	1.0300	0.6700
0.6682	3.7273	451	0.8206	0.6781
0.6605	3.9339	476	0.8862	0.6822
0.8521	4.0	484	0.8093	0.6316
0.6442	4.1405	501	0.9483	0.6437
0.577	4.3471	526	0.8860	0.6883
0.5252	4.5537	551	0.8797	0.7045
0.5274	4.7603	576	0.7289	0.7024
0.467	4.9669	601	0.8224	0.6903
0.467	5.0	605	0.8218	0.6903

Framework versions

• Transformers 4.40.2
• Pytorch 1.12.0
• Datasets 2.19.1
• Tokenizers 0.19.1