testc8-1

This model is a fine-tuned version of shafin/chemical-bert-uncased-finetuned-cust-c2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1490

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.0415	1.0	16	0.1392
0.0443	2.0	32	0.1289
0.0471	3.0	48	0.1363
0.042	4.0	64	0.1598
0.0452	5.0	80	0.1571
0.0446	6.0	96	0.1733
0.0466	7.0	112	0.1301
0.0391	8.0	128	0.1359
0.0425	9.0	144	0.1324
0.0436	10.0	160	0.0939
0.0406	11.0	176	0.1495
0.0387	12.0	192	0.1592
0.0335	13.0	208	0.1118
0.0413	14.0	224	0.1508
0.0363	15.0	240	0.1471
0.0428	16.0	256	0.1721
0.0384	17.0	272	0.1853
0.0381	18.0	288	0.1578
0.0373	19.0	304	0.1707
0.0351	20.0	320	0.1241
0.0346	21.0	336	0.1602
0.0386	22.0	352	0.1207
0.0274	23.0	368	0.1642
0.0338	24.0	384	0.1169
0.0327	25.0	400	0.1461
0.026	26.0	416	0.1323
0.0315	27.0	432	0.1403
0.042	28.0	448	0.1056
0.0346	29.0	464	0.1186
0.0294	30.0	480	0.1490

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1+cu113
• Datasets 2.7.1
• Tokenizers 0.13.2