Mistral-7B-v0.3_metamath_reverse

This model is a fine-tuned version of unsloth/mistral-7b-v0.3 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 4.4478

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.7436 0.0211 13 7.4054
9.1068 0.0421 26 6.9800
6.6988 0.0632 39 6.4271
6.4684 0.0842 52 6.2893
6.1245 0.1053 65 6.1245
5.9117 0.1264 78 5.8770
5.8448 0.1474 91 5.7834
5.742 0.1685 104 5.8941
5.6054 0.1896 117 6.0972
5.6465 0.2106 130 5.4808
5.5659 0.2317 143 5.5371
5.4175 0.2527 156 5.5688
5.3148 0.2738 169 5.3646
5.2376 0.2949 182 5.2052
5.2313 0.3159 195 5.1473
5.1381 0.3370 208 5.2471
5.0545 0.3580 221 5.0579
5.0218 0.3791 234 5.0434
5.1901 0.4002 247 5.1862
5.0809 0.4212 260 5.0103
5.0357 0.4423 273 5.0488
5.0375 0.4633 286 5.0026
5.0348 0.4844 299 5.0081
4.8927 0.5055 312 4.8912
4.878 0.5265 325 4.8665
4.8092 0.5476 338 4.8402
4.8342 0.5687 351 4.7689
4.7834 0.5897 364 4.7842
4.7428 0.6108 377 4.7396
4.7318 0.6318 390 4.6987
4.6442 0.6529 403 4.6854
4.6454 0.6740 416 4.6917
4.7075 0.6950 429 4.6419
4.6744 0.7161 442 4.5826
4.5861 0.7371 455 4.5793
4.5707 0.7582 468 4.5944
4.5675 0.7793 481 4.5611
4.5286 0.8003 494 4.5216
4.5302 0.8214 507 4.5140
4.5191 0.8424 520 4.5084
4.5023 0.8635 533 4.4878
4.4661 0.8846 546 4.4593
4.4942 0.9056 559 4.4660
4.4691 0.9267 572 4.4433
4.4153 0.9478 585 4.4531
4.4609 0.9688 598 4.4450
4.4444 0.9899 611 4.4478

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
15
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for imdatta0/Mistral-7B-v0.3_metamath_reverse

Adapter
(37)
this model