|
--- |
|
license: apache-2.0 |
|
language: |
|
- fr |
|
- it |
|
- de |
|
- es |
|
- en |
|
inference: false |
|
--- |
|
# Model Card for Mixtral-8x7B |
|
The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. The Mixtral-8x7B outperforms Llama 2 70B on most benchmarks we tested. |
|
|
|
For full details of this model please read our [release blog post](https://mistral.ai/news/mixtral-of-experts/). |
|
|
|
## Instruction format |
|
|
|
This format must be strictly respected, otherwise the model will generate sub-optimal outputs. |
|
|
|
The template used to build a prompt for the Instruct model is defined as follows: |
|
``` |
|
<s> [INST] Instruction [/INST] Model answer</s> [INST] Follow-up instruction [/INST] |
|
``` |
|
Note that `<s>` and `</s>` are special tokens for beginning of string (BOS) and end of string (EOS) while [INST] and [/INST] are regular strings. |
|
|
|
## Run the model |
|
|
|
```bash |
|
# Install mlx, mlx-examples, huggingface-cli |
|
pip install mlx |
|
pip install huggingface_hub hf_transfer |
|
git clone https://github.com/ml-explore/mlx-examples.git |
|
|
|
# Download model |
|
export HF_HUB_ENABLE_HF_TRANSFER=1 |
|
huggingface-cli download --local-dir Mixtral-8x7B-Instruct-v0.1 mlx-community/Mixtral-8x7B-Instruct-v0.1 |
|
|
|
# Run example |
|
python mlx-examples/mixtral/mixtral.py --model_path Mixtral-8x7B-Instruct-v0.1 |
|
``` |
