File size: 563 Bytes
237c45c 4bd28a5 c661a85 237c45c d647b79 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
license: mit
tags:
- mistral
- unsloth
pipeline_tag: text-generation
---
# Model Card for mistral7b-v0.3-alpaca-cleaned
<!-- Provide a quick summary of what the model is/does. -->
Mistral 7B v0.3 finetuned on cleaned Stanford Alpaca dataset using LoRA
Model was finetuned on for 1 epoch using paged_adamw_8bit optimizer with these params:
per_device_train_batch_size = 10,
gradient_accumulation_steps = 4,
warmup_steps = 5,
num_train_epochs=1,
learning_rate = 2e-4,
optim = "paged_adamw_8bit",
weight_decay = 0.01,
lr_scheduler_type = "linear",
seed = 3407 |