random_first_noditransitive_seed-21_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4741
• Accuracy: 0.3750

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 64
• seed: 21
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 32000
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
6.1465	0.9998	1495	4.6626	0.2658
4.6143	1.9997	2990	4.2027	0.3044
4.0277	2.9995	4485	3.9460	0.3259
3.8611	4.0	5981	3.7883	0.3405
3.6323	4.9998	7476	3.6931	0.3499
3.5592	5.9997	8971	3.6298	0.3563
3.4438	6.9995	10466	3.5884	0.3606
3.4055	8.0	11962	3.5589	0.3636
3.3357	8.9998	13457	3.5381	0.3663
3.3179	9.9997	14952	3.5222	0.3682
3.2689	10.9995	16447	3.5109	0.3697
3.2564	12.0	17943	3.5009	0.3708
3.2238	12.9998	19438	3.5008	0.3713
3.2153	13.9997	20933	3.4895	0.3722
3.1924	14.9995	22428	3.4841	0.3732
3.1897	16.0	23924	3.4814	0.3736
3.1709	16.9998	25419	3.4810	0.3738
3.1676	17.9997	26914	3.4800	0.3745
3.157	18.9995	28409	3.4774	0.3745
3.1553	19.9967	29900	3.4741	0.3750

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.20.0