---
base_model: mistralai/Mistral-7B-v0.3
library_name: peft
license: apache-2.0
tags:
- unsloth
- generated_from_trainer
model-index:
- name: Mistral-7B-v0.3_pct_reverse_r32
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Mistral-7B-v0.3_pct_reverse_r32

This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0458

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.02
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 1.991         | 0.0206 | 8    | 2.0312          |
| 2.0461        | 0.0413 | 16   | 2.0335          |
| 2.0456        | 0.0619 | 24   | 2.0601          |
| 2.0584        | 0.0825 | 32   | 2.0879          |
| 2.1123        | 0.1032 | 40   | 2.0809          |
| 2.0666        | 0.1238 | 48   | 2.0890          |
| 2.0733        | 0.1444 | 56   | 2.0954          |
| 2.1236        | 0.1651 | 64   | 2.0971          |
| 2.1103        | 0.1857 | 72   | 2.1008          |
| 2.0876        | 0.2063 | 80   | 2.1042          |
| 2.1107        | 0.2270 | 88   | 2.1155          |
| 2.0889        | 0.2476 | 96   | 2.1083          |
| 2.097         | 0.2682 | 104  | 2.1186          |
| 2.0962        | 0.2889 | 112  | 2.1202          |
| 2.1415        | 0.3095 | 120  | 2.1305          |
| 2.1294        | 0.3301 | 128  | 2.1169          |
| 2.1476        | 0.3508 | 136  | 2.1300          |
| 2.1725        | 0.3714 | 144  | 2.1245          |
| 2.1159        | 0.3920 | 152  | 2.1172          |
| 2.0921        | 0.4127 | 160  | 2.1221          |
| 2.141         | 0.4333 | 168  | 2.1334          |
| 2.1312        | 0.4539 | 176  | 2.1259          |
| 2.106         | 0.4746 | 184  | 2.1269          |
| 2.1015        | 0.4952 | 192  | 2.1197          |
| 2.1368        | 0.5158 | 200  | 2.1164          |
| 2.0751        | 0.5364 | 208  | 2.1104          |
| 2.135         | 0.5571 | 216  | 2.1105          |
| 2.0718        | 0.5777 | 224  | 2.1003          |
| 2.0393        | 0.5983 | 232  | 2.1025          |
| 2.1034        | 0.6190 | 240  | 2.0946          |
| 2.045         | 0.6396 | 248  | 2.0939          |
| 2.077         | 0.6602 | 256  | 2.0814          |
| 2.0514        | 0.6809 | 264  | 2.0800          |
| 2.0222        | 0.7015 | 272  | 2.0774          |
| 2.075         | 0.7221 | 280  | 2.0749          |
| 2.1013        | 0.7428 | 288  | 2.0705          |
| 2.0929        | 0.7634 | 296  | 2.0643          |
| 2.0996        | 0.7840 | 304  | 2.0692          |
| 2.0507        | 0.8047 | 312  | 2.0588          |
| 2.0353        | 0.8253 | 320  | 2.0574          |
| 2.0128        | 0.8459 | 328  | 2.0570          |
| 2.0508        | 0.8666 | 336  | 2.0503          |
| 2.067         | 0.8872 | 344  | 2.0472          |
| 2.0821        | 0.9078 | 352  | 2.0476          |
| 2.0461        | 0.9285 | 360  | 2.0471          |
| 2.0666        | 0.9491 | 368  | 2.0461          |
| 2.0639        | 0.9697 | 376  | 2.0458          |
| 1.9859        | 0.9904 | 384  | 2.0458          |


### Framework versions

- PEFT 0.12.0
- Transformers 4.44.2
- Pytorch 2.3.0+cu121
- Datasets 2.21.0
- Tokenizers 0.19.1