scenario-kd-pre-ner-half-xlmr_data-univner_full66

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 119.4132
• Precision: 0.4396
• Recall: 0.4046
• F1: 0.4213
• Accuracy: 0.9482

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 32
• seed: 66
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
284.1285	0.29	500	210.7202	0.0	0.0	0.0	0.9241
194.1198	0.58	1000	183.1519	0.1100	0.0170	0.0295	0.9252
175.8237	0.87	1500	169.5916	0.1989	0.0410	0.0680	0.9263
165.0206	1.16	2000	160.4465	0.215	0.0558	0.0886	0.9279
156.1986	1.46	2500	153.7663	0.2666	0.1428	0.1860	0.9325
151.2919	1.75	3000	148.4524	0.2818	0.1733	0.2146	0.9345
146.4505	2.04	3500	144.1935	0.2983	0.2017	0.2407	0.9355
142.0868	2.33	4000	141.0608	0.3472	0.1945	0.2493	0.9366
139.2831	2.62	4500	138.2455	0.3387	0.2598	0.2941	0.9388
136.7819	2.91	5000	135.6073	0.3525	0.2927	0.3199	0.9406
133.7896	3.2	5500	133.6945	0.3469	0.3021	0.3230	0.9410
132.221	3.49	6000	131.9122	0.3756	0.3238	0.3478	0.9420
130.4828	3.78	6500	130.4884	0.3579	0.3295	0.3431	0.9419
129.2979	4.08	7000	129.0718	0.3921	0.3178	0.3511	0.9430
127.4707	4.37	7500	127.8377	0.3954	0.3430	0.3673	0.9433
126.2401	4.66	8000	126.8584	0.3824	0.3525	0.3668	0.9438
125.6309	4.95	8500	125.8471	0.3999	0.3691	0.3839	0.9443
124.62	5.24	9000	125.0322	0.4002	0.3519	0.3745	0.9449
123.3763	5.53	9500	124.1776	0.4175	0.3584	0.3857	0.9451
122.3042	5.82	10000	123.5417	0.4107	0.3774	0.3934	0.9457
121.8611	6.11	10500	122.9038	0.4184	0.3797	0.3981	0.9461
121.1695	6.4	11000	122.4267	0.4213	0.3702	0.3941	0.9461
121.0527	6.69	11500	121.9642	0.4172	0.4087	0.4129	0.9469
120.1228	6.99	12000	121.4310	0.4241	0.3953	0.4092	0.9472
119.9357	7.28	12500	121.1448	0.4368	0.3764	0.4044	0.9469
119.0999	7.57	13000	120.7298	0.4295	0.3966	0.4124	0.9470
119.1017	7.86	13500	120.4837	0.4319	0.3909	0.4104	0.9475
118.622	8.15	14000	120.2208	0.4227	0.3907	0.4061	0.9473
118.3995	8.44	14500	119.9204	0.4237	0.3978	0.4103	0.9477
118.1158	8.73	15000	119.8377	0.4298	0.3956	0.4120	0.9475
117.752	9.02	15500	119.6274	0.4347	0.4151	0.4246	0.9482
117.8189	9.31	16000	119.5818	0.4377	0.3999	0.4180	0.9481
117.8235	9.61	16500	119.4980	0.4385	0.3962	0.4163	0.9481
117.6293	9.9	17000	119.4132	0.4396	0.4046	0.4213	0.9482

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3