bert-large-uncased_stereoset_finetuned

This model is a fine-tuned version of bert-large-uncased on the stereoset dataset. It achieves the following results on the evaluation set:

• Loss: 1.0729
• Accuracy: 0.7716

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.21	5	0.6925	0.5071
No log	0.42	10	0.6978	0.5008
No log	0.62	15	0.6891	0.5275
No log	0.83	20	0.6850	0.5487
No log	1.04	25	0.7521	0.5126
No log	1.25	30	0.6577	0.6177
No log	1.46	35	0.6759	0.5440
No log	1.67	40	0.6395	0.6405
No log	1.88	45	0.6064	0.6719
No log	2.08	50	0.5822	0.6986
No log	2.29	55	0.5566	0.7096
No log	2.5	60	0.5411	0.7331
No log	2.71	65	0.5448	0.7551
No log	2.92	70	0.5384	0.7339
No log	3.12	75	0.5487	0.7535
No log	3.33	80	0.5572	0.7567
No log	3.54	85	0.5763	0.7614
No log	3.75	90	0.5756	0.7645
No log	3.96	95	0.5524	0.7645
No log	4.17	100	0.6320	0.7614
No log	4.38	105	0.6512	0.7575
No log	4.58	110	0.6582	0.7606
No log	4.79	115	0.6731	0.7669
No log	5.0	120	0.6944	0.7575
No log	5.21	125	0.7142	0.7575
No log	5.42	130	0.7004	0.7645
No log	5.62	135	0.6794	0.7630
No log	5.83	140	0.7108	0.7606
No log	6.04	145	0.7730	0.7590
No log	6.25	150	0.8083	0.7614
No log	6.46	155	0.8361	0.7653
No log	6.67	160	0.8498	0.7692
No log	6.88	165	0.8769	0.7700
No log	7.08	170	0.8324	0.7582
No log	7.29	175	0.7945	0.7645
No log	7.5	180	0.8480	0.7684
No log	7.71	185	0.8905	0.7724
No log	7.92	190	0.9560	0.7700
No log	8.12	195	0.9976	0.7669
No log	8.33	200	1.0315	0.7677
No log	8.54	205	1.0413	0.7692
No log	8.75	210	1.0216	0.7708
No log	8.96	215	1.0251	0.7716
No log	9.17	220	1.0483	0.7716
No log	9.38	225	1.0616	0.7716
No log	9.58	230	1.0703	0.7708
No log	9.79	235	1.0731	0.7732
No log	10.0	240	1.0729	0.7716

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.9.0
• Tokenizers 0.13.2