ES-ENG-xlm-roberta-sentiment

This model is a fine-tuned version of xlm-roberta-base on a Custom dataset.

The best model (stopped after 20 epochs) achieves the following results on the evaluation set:

• Loss: 0.7743
• Accuracy: 0.6702
• F1: 0.6672
• Precision: 0.6664
• Recall: 0.6702

Intended uses & limitations

Note that commercial use with this model is prohibited.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
1.1099	1.0	208	1.0718	0.3968	0.3851	0.4857	0.3968
1.0057	2.0	416	0.8926	0.5492	0.5080	0.5639	0.5492
0.8988	3.0	624	0.8384	0.5883	0.5792	0.5789	0.5883
0.8606	4.0	832	0.8209	0.6168	0.6086	0.6086	0.6168
0.8338	5.0	1040	0.8006	0.6120	0.6068	0.6046	0.6120
0.8081	6.0	1248	0.8074	0.6026	0.5935	0.5966	0.6026
0.7872	7.0	1456	0.7786	0.6194	0.6149	0.6127	0.6194
0.7624	8.0	1664	0.7783	0.6379	0.6277	0.6342	0.6379
0.7446	9.0	1872	0.7643	0.6366	0.6287	0.6314	0.6366
0.7274	10.0	2080	0.7846	0.6395	0.6297	0.6351	0.6395
0.7116	11.0	2288	0.7465	0.6495	0.6425	0.6462	0.6495
0.6998	12.0	2496	0.7599	0.6537	0.6474	0.6494	0.6537
0.6852	13.0	2704	0.7651	0.6515	0.6443	0.6465	0.6515
0.6726	14.0	2912	0.7571	0.6576	0.6536	0.6530	0.6576
0.6665	15.0	3120	0.7597	0.6557	0.6506	0.6514	0.6557
0.6541	16.0	3328	0.7590	0.6615	0.6584	0.6576	0.6615
0.6513	17.0	3536	0.7617	0.6599	0.6544	0.6555	0.6599
0.6392	18.0	3744	0.7740	0.6628	0.6585	0.6582	0.6628
0.6369	19.0	3952	0.7666	0.6631	0.6588	0.6585	0.6631
0.6268	20.0	4160	0.7743	0.6702	0.6672	0.6664	0.6702
0.62	21.0	4368	0.7712	0.6680	0.6638	0.6638	0.6680
0.619	22.0	4576	0.7720	0.6689	0.6656	0.6649	0.6689
0.6074	23.0	4784	0.7729	0.6663	0.6630	0.6621	0.6663

Framework versions

• Transformers 4.29.1
• Pytorch 2.0.1+cu117
• Datasets 2.12.0
• Tokenizers 0.13.3