short_first_noditransitive_seed-63_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.2456
• Accuracy: 0.3980

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 64
• seed: 63
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 32000
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
6.0204	0.9998	1495	4.4457	0.2904
4.4072	1.9997	2990	3.9658	0.3292
3.7995	2.9995	4485	3.6990	0.3514
3.6235	4.0	5981	3.5407	0.3664
3.3971	4.9998	7476	3.4475	0.3755
3.3235	5.9997	8971	3.3841	0.3819
3.2097	6.9995	10466	3.3465	0.3854
3.1742	8.0	11962	3.3189	0.3884
3.1056	8.9998	13457	3.2979	0.3908
3.0854	9.9997	14952	3.2868	0.3925
3.0406	10.9995	16447	3.2746	0.3936
3.0273	12.0	17943	3.2689	0.3944
2.9947	12.9998	19438	3.2647	0.3955
2.9867	13.9997	20933	3.2592	0.3962
2.9643	14.9995	22428	3.2591	0.3968
2.9598	16.0	23924	3.2537	0.3967
2.9422	16.9998	25419	3.2500	0.3973
2.9399	17.9997	26914	3.2506	0.3977
2.9294	18.9995	28409	3.2454	0.3977
2.9276	19.9967	29900	3.2456	0.3980

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.20.0