gbert-large-finetuned-cust

This model is a fine-tuned version of deepset/gbert-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1846

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.8251	1.0	157	0.5204
0.508	2.0	314	0.3953
0.4009	3.0	471	0.3242
0.3587	4.0	628	0.3300
0.3276	5.0	785	0.3137
0.302	6.0	942	0.2826
0.2777	7.0	1099	0.2768
0.2609	8.0	1256	0.2726
0.244	9.0	1413	0.2660
0.2274	10.0	1570	0.2391
0.2132	11.0	1727	0.2353
0.2014	12.0	1884	0.2134
0.1835	13.0	2041	0.2278
0.1896	14.0	2198	0.2110
0.1974	15.0	2355	0.2132
0.1775	16.0	2512	0.1973
0.1715	17.0	2669	0.1941
0.1777	18.0	2826	0.2105
0.1741	19.0	2983	0.2127
0.1607	20.0	3140	0.1762
0.1562	21.0	3297	0.2095
0.1548	22.0	3454	0.1805
0.1534	23.0	3611	0.1852
0.1484	24.0	3768	0.1773
0.1473	25.0	3925	0.1759
0.1354	26.0	4082	0.1734
0.136	27.0	4239	0.1902
0.1306	28.0	4396	0.1769
0.1353	29.0	4553	0.1705
0.1368	30.0	4710	0.1846

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3