---
license: mit
library_name: transformers
datasets:
- argilla/dpo-mix-7k
base_model: wandb/mistral-7b-zephyr-sft
---

[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/llm_surgery/gemma-zephyr)

# Mistral 7B Zephyr DPO V2

The [Zephyr](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) DPO recipe applied on top of Mistral 7B (new recipe with chatML format)

## Model description

- **Model type:** A 7.2B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
- **Language(s) (NLP):** Primarily English
- **Finetuned from model:** [wandb/mistral-7b-zephyr-sft](https://huggingface.co/wandb/mistral-7b-zephyr-sft)

## Recipe

We trained using the [alignment handbook recipe](https://github.com/huggingface/alignment-handbook/blob/main/scripts/run_dpo.py) and logging to W&B

Visit the [W&B workspace here](https://wandb.ai/llm_surgery/gemma-zephyr?nw=nwusercapecape)

## Compute provided by Lambda Labs - 8xA100 80GB node