metadata

license: mit
tags:
  - mistral
  - unsloth
pipeline_tag: text-generation

Model Card for mistral7b-v0.3-alpaca-cleaned

Mistral 7B v0.3 finetuned on cleaned Stanford Alpaca dataset using LoRA

Model was finetuned on for 1 epoch using paged_adamw_8bit optimizer with these params: per_device_train_batch_size = 10, gradient_accumulation_steps = 4, warmup_steps = 5, num_train_epochs=1, learning_rate = 2e-4, optim = "paged_adamw_8bit", weight_decay = 0.01, lr_scheduler_type = "linear", seed = 3407