results

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3359
• Accuracy: 0.8828
• F1: 0.4499
• Precision: 0.4845
• Recall: 0.4348

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 3
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
0.3514	0.1149	50	0.3432	0.8724	0.3106	0.2908	0.3333
0.3392	0.2299	100	0.3374	0.8724	0.3106	0.2908	0.3333
0.3378	0.3448	150	0.3371	0.8724	0.3106	0.2908	0.3333
0.3375	0.4598	200	0.3370	0.8724	0.3106	0.2908	0.3333
0.3369	0.5747	250	0.3368	0.8724	0.3106	0.2908	0.3333
0.3366	0.6897	300	0.3368	0.8724	0.3106	0.2908	0.3333
0.3366	0.8046	350	0.3367	0.8724	0.3106	0.2908	0.3333
0.336	0.9195	400	0.3367	0.8724	0.3106	0.2908	0.3333
0.3375	1.0345	450	0.3366	0.8724	0.3106	0.2908	0.3333
0.3363	1.1494	500	0.3365	0.8724	0.3106	0.2908	0.3333
0.3362	1.2644	550	0.3365	0.8736	0.3335	0.4917	0.3445
0.3358	1.3793	600	0.3365	0.8724	0.3184	0.4578	0.3369
0.337	1.4943	650	0.3363	0.8747	0.3409	0.5143	0.3485
0.3363	1.6092	700	0.3363	0.8690	0.3771	0.4353	0.3714
0.3367	1.7241	750	0.3362	0.8736	0.3532	0.4744	0.3552
0.3367	1.8391	800	0.3362	0.8736	0.3532	0.4744	0.3552
0.3363	1.9540	850	0.3360	0.8690	0.3957	0.4466	0.3857
0.3363	2.0690	900	0.3359	0.8713	0.3933	0.4558	0.3830
0.3363	2.1839	950	0.3359	0.8713	0.3977	0.4565	0.3865
0.3356	2.2989	1000	0.3358	0.8655	0.4048	0.4388	0.3951
0.3359	2.4138	1050	0.3358	0.8701	0.3628	0.4412	0.3611
0.3362	2.5287	1100	0.3358	0.8678	0.4243	0.4509	0.4138
0.3359	2.6437	1150	0.3358	0.8644	0.4274	0.4443	0.4197
0.336	2.7586	1200	0.3357	0.8655	0.4048	0.4388	0.3951
0.3352	2.8736	1250	0.3357	0.8667	0.4095	0.4435	0.3991
0.3356	2.9885	1300	0.3357	0.8667	0.4095	0.4435	0.3991

Framework versions

• Transformers 4.41.2
• Pytorch 2.0.0+cu118
• Datasets 2.17.0
• Tokenizers 0.19.1