|
--- |
|
license: mit |
|
tags: |
|
- mistral |
|
- unsloth |
|
pipeline_tag: text-generation |
|
--- |
|
# Model Card for mistral7b-v0.3-alpaca-cleaned |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
Mistral 7B v0.3 finetuned on cleaned Stanford Alpaca dataset using LoRA |
|
|
|
Model was finetuned on for 1 epoch using paged_adamw_8bit optimizer with these params: |
|
per_device_train_batch_size = 10, |
|
gradient_accumulation_steps = 4, |
|
warmup_steps = 5, |
|
num_train_epochs=1, |
|
learning_rate = 2e-4, |
|
optim = "paged_adamw_8bit", |
|
weight_decay = 0.01, |
|
lr_scheduler_type = "linear", |
|
seed = 3407 |
