---
library_name: transformers
base_model: wandb/mistral-7b-zephyr-sft
license: apache-2.0
---

# A Pruned Mistral model

This model is a pruned Mistral model re-aligned using the Zephyr Recipe

## details

- This model has 2 stages training: SFT and DPO
- The initial model consist on selecting some layers of the mistral model to make a smaller model
- the code can be found here: github.com/tcapelle/shear


## W&B workspace

https://wandb.ai/llm_surgery/shearllama/