random_first_noditransitive_seed-63_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4683
• Accuracy: 0.3757

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 64
• seed: 63
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 32000
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
6.1397	0.9998	1495	4.6645	0.2652
4.6172	1.9997	2990	4.1987	0.3050
4.0314	2.9995	4485	3.9518	0.3255
3.8633	4.0	5981	3.7929	0.3399
3.637	4.9998	7476	3.6933	0.3499
3.561	5.9997	8971	3.6266	0.3569
3.4427	6.9995	10466	3.5855	0.3608
3.4054	8.0	11962	3.5528	0.3644
3.3332	8.9998	13457	3.5351	0.3667
3.3122	9.9997	14952	3.5174	0.3686
3.2653	10.9995	16447	3.5048	0.3702
3.2514	12.0	17943	3.5013	0.3711
3.2181	12.9998	19438	3.4940	0.3720
3.2097	13.9997	20933	3.4829	0.3734
3.1862	14.9995	22428	3.4822	0.3737
3.1818	16.0	23924	3.4781	0.3741
3.1633	16.9998	25419	3.4738	0.3746
3.1605	17.9997	26914	3.4749	0.3752
3.1498	18.9995	28409	3.4698	0.3752
3.1478	19.9967	29900	3.4683	0.3757

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.20.0