RuBioRoBERTa_neg

This model is a fine-tuned version of alexyalunin/RuBioRoBERTa on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5876
• Precision: 0.584
• Recall: 0.6053
• F1: 0.5945
• Accuracy: 0.9040

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	50	0.6516	0.0	0.0	0.0	0.7729
No log	2.0	100	0.6625	0.0	0.0	0.0	0.7706
No log	3.0	150	0.5142	0.0081	0.0058	0.0068	0.7944
No log	4.0	200	0.4359	0.0788	0.1464	0.1024	0.8281
No log	5.0	250	0.3580	0.2362	0.3141	0.2696	0.8642
No log	6.0	300	0.3419	0.2819	0.3237	0.3013	0.8762
No log	7.0	350	0.3492	0.35	0.3642	0.3569	0.8841
No log	8.0	400	0.2633	0.3549	0.4432	0.3942	0.8982
No log	9.0	450	0.2819	0.3871	0.4624	0.4214	0.9001
0.4095	10.0	500	0.2522	0.5035	0.5491	0.5253	0.9119
0.4095	11.0	550	0.2831	0.4704	0.5511	0.5075	0.9077
0.4095	12.0	600	0.3013	0.5245	0.6185	0.5676	0.9105
0.4095	13.0	650	0.3070	0.4711	0.6127	0.5327	0.9048
0.4095	14.0	700	0.3398	0.4771	0.6416	0.5472	0.9039
0.4095	15.0	750	0.3275	0.4661	0.6224	0.5330	0.9114
0.4095	16.0	800	0.3730	0.5118	0.6281	0.5640	0.9141
0.4095	17.0	850	0.3847	0.5593	0.6358	0.5951	0.9160
0.4095	18.0	900	0.4070	0.5824	0.6262	0.6035	0.9182
0.4095	19.0	950	0.3583	0.5433	0.6281	0.5827	0.9161
0.0776	20.0	1000	0.3096	0.5152	0.5877	0.5491	0.9154
0.0776	21.0	1050	0.4015	0.5669	0.6204	0.5925	0.9224
0.0776	22.0	1100	0.5603	0.4251	0.6667	0.5191	0.8753
0.0776	23.0	1150	0.3353	0.6220	0.6089	0.6154	0.9230
0.0776	24.0	1200	0.3800	0.6133	0.6204	0.6169	0.9254
0.0776	25.0	1250	0.4451	0.5792	0.6127	0.5955	0.9153
0.0776	26.0	1300	0.4639	0.6060	0.6224	0.6141	0.9220
0.0776	27.0	1350	0.4141	0.5574	0.6647	0.6063	0.9194
0.0776	28.0	1400	0.4258	0.5675	0.6397	0.6014	0.9143
0.0776	29.0	1450	0.4131	0.5880	0.6435	0.6145	0.9193
0.0374	30.0	1500	0.4104	0.5823	0.6609	0.6191	0.9200
0.0374	31.0	1550	0.4047	0.6190	0.6667	0.6419	0.9213
0.0374	32.0	1600	0.4615	0.6233	0.6185	0.6209	0.9205
0.0374	33.0	1650	0.4597	0.6430	0.5934	0.6172	0.9169
0.0374	34.0	1700	0.3851	0.5043	0.6821	0.5799	0.9040
0.0374	35.0	1750	0.3989	0.6241	0.6590	0.6410	0.9206
0.0374	36.0	1800	0.4866	0.5710	0.6667	0.6151	0.9156
0.0374	37.0	1850	0.4198	0.6208	0.6339	0.6273	0.9241
0.0374	38.0	1900	0.4526	0.5615	0.6243	0.5912	0.9164
0.0374	39.0	1950	0.5038	0.6149	0.6031	0.6089	0.9187
0.0337	40.0	2000	0.3879	0.5684	0.6243	0.5950	0.9196
0.0337	41.0	2050	0.5178	0.5913	0.6301	0.6101	0.9170
0.0337	42.0	2100	0.4898	0.6558	0.5838	0.6177	0.9155

Framework versions

• Transformers 4.40.1
• Pytorch 2.2.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1