roberta-large-sst-2-16-13-30

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6901
• Accuracy: 0.625

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 5
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	1	0.6957	0.5
No log	2.0	2	0.6955	0.5
No log	3.0	3	0.6952	0.5
No log	4.0	4	0.6944	0.5
No log	5.0	5	0.6937	0.5
No log	6.0	6	0.6933	0.5
No log	7.0	7	0.6929	0.5
No log	8.0	8	0.6942	0.5
No log	9.0	9	0.6931	0.5
0.6903	10.0	10	0.6917	0.5
0.6903	11.0	11	0.6905	0.5
0.6903	12.0	12	0.6891	0.5312
0.6903	13.0	13	0.6883	0.625
0.6903	14.0	14	0.6874	0.6562
0.6903	15.0	15	0.6849	0.5312
0.6903	16.0	16	0.6822	0.5312
0.6903	17.0	17	0.6790	0.5
0.6903	18.0	18	0.6742	0.5
0.6903	19.0	19	0.6650	0.5312
0.626	20.0	20	0.6524	0.5312
0.626	21.0	21	0.6444	0.5312
0.626	22.0	22	0.6361	0.5625
0.626	23.0	23	0.6327	0.5938
0.626	24.0	24	0.6337	0.625
0.626	25.0	25	0.6437	0.625
0.626	26.0	26	0.6580	0.6562
0.626	27.0	27	0.6725	0.6562
0.626	28.0	28	0.6812	0.625
0.626	29.0	29	0.6873	0.625
0.4393	30.0	30	0.6901	0.625

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3