---
base_model: mistralai/Mistral-7B-v0.3
library_name: peft
license: apache-2.0
tags:
- unsloth
- generated_from_trainer
model-index:
- name: Mistral-7B-v0.3_pct_default_r32
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Mistral-7B-v0.3_pct_default_r32

This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0448

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.02
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 1.9915        | 0.0206 | 8    | 2.0385          |
| 2.054         | 0.0413 | 16   | 2.0376          |
| 2.0356        | 0.0619 | 24   | 2.0604          |
| 2.0385        | 0.0825 | 32   | 2.0639          |
| 2.1223        | 0.1032 | 40   | 2.0833          |
| 2.0677        | 0.1238 | 48   | 2.0910          |
| 2.0729        | 0.1444 | 56   | 2.0872          |
| 2.1197        | 0.1651 | 64   | 2.0973          |
| 2.1053        | 0.1857 | 72   | 2.0919          |
| 2.0848        | 0.2063 | 80   | 2.1035          |
| 2.1015        | 0.2270 | 88   | 2.1114          |
| 2.0872        | 0.2476 | 96   | 2.1133          |
| 2.0948        | 0.2682 | 104  | 2.1221          |
| 2.097         | 0.2889 | 112  | 2.1219          |
| 2.147         | 0.3095 | 120  | 2.1240          |
| 2.1315        | 0.3301 | 128  | 2.1189          |
| 2.1563        | 0.3508 | 136  | 2.1368          |
| 2.1836        | 0.3714 | 144  | 2.1271          |
| 2.1245        | 0.3920 | 152  | 2.1198          |
| 2.0947        | 0.4127 | 160  | 2.1240          |
| 2.1472        | 0.4333 | 168  | 2.1354          |
| 2.1348        | 0.4539 | 176  | 2.1261          |
| 2.1099        | 0.4746 | 184  | 2.1275          |
| 2.1006        | 0.4952 | 192  | 2.1196          |
| 2.1339        | 0.5158 | 200  | 2.1170          |
| 2.0841        | 0.5364 | 208  | 2.1105          |
| 2.1344        | 0.5571 | 216  | 2.1079          |
| 2.0732        | 0.5777 | 224  | 2.1043          |
| 2.0417        | 0.5983 | 232  | 2.1035          |
| 2.1003        | 0.6190 | 240  | 2.0967          |
| 2.0501        | 0.6396 | 248  | 2.1007          |
| 2.078         | 0.6602 | 256  | 2.0862          |
| 2.0507        | 0.6809 | 264  | 2.0840          |
| 2.0235        | 0.7015 | 272  | 2.0762          |
| 2.0743        | 0.7221 | 280  | 2.0723          |
| 2.1028        | 0.7428 | 288  | 2.0721          |
| 2.0987        | 0.7634 | 296  | 2.0662          |
| 2.0985        | 0.7840 | 304  | 2.0663          |
| 2.0548        | 0.8047 | 312  | 2.0602          |
| 2.0365        | 0.8253 | 320  | 2.0563          |
| 2.0102        | 0.8459 | 328  | 2.0564          |
| 2.0497        | 0.8666 | 336  | 2.0522          |
| 2.0721        | 0.8872 | 344  | 2.0471          |
| 2.0812        | 0.9078 | 352  | 2.0468          |
| 2.0475        | 0.9285 | 360  | 2.0462          |
| 2.0687        | 0.9491 | 368  | 2.0452          |
| 2.065         | 0.9697 | 376  | 2.0450          |
| 1.991         | 0.9904 | 384  | 2.0448          |


### Framework versions

- PEFT 0.12.0
- Transformers 4.44.2
- Pytorch 2.3.0+cu121
- Datasets 2.21.0
- Tokenizers 0.19.1