mistralai

/

Mistral-7B-Instruct-v0.3

like 719

There's no system message with Mistral's prompt format. Never has been. Unfortunately!

You can see in the chat template that it explicitly blocks anything other than user and assistant

{ raise_exception('Only user and assistant roles are supported!') }

I just deployed this model to AWS SageMaker and I'm blown away by how easily it integrates with Langchain agent and tools right out of the box! It's simply the best open-source 7B model I've seen for function and tool calling. 🤯

Here is the link
https://www.linkedin.com/pulse/effortless-ai-your-guide-self-hosted-mistral-7b-v03-langchain-lee-wowxc/

There's no system message with Mistral's prompt format. Never has been. Unfortunately!

But why is this in the official example? It clearly uses "system" in the input message.

from transformers import pipeline

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]
chatbot = pipeline("text-generation", model="mistralai/Mistral-7B-Instruct-v0.3")
chatbot(messages)

There's no system message with Mistral's prompt format. Never has been. Unfortunately!

But why is this in the official example? It clearly uses "system" in the input message.

from transformers import pipeline

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]
chatbot = pipeline("text-generation", model="mistralai/Mistral-7B-Instruct-v0.3")
chatbot(messages)

Clearly, the recommended library for this model is mistral inference. However, the HF team added that part here https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3/discussions/7

Now, why would they add an example that clearly won't work?

Why?

@mclassHF2023 @wolfram @bartowski @lyoh001 @c6sneaky @MaziyarPanahi

You can use this repo: https://github.com/chujiezheng/chat_templates

It supports the system message according to Mistral's tutorial (but this is not reflected in Mistral's HF repos).

@mclassHF2023 @wolfram @bartowski @lyoh001 @c6sneaky @MaziyarPanahi

You can use this repo: https://github.com/chujiezheng/chat_templates

It supports the system message according to Mistral's tutorial (but this is not reflected in Mistral's HF repos).

Beautiful job on this repo! Super useful as I often had to make my own!

Thanks for sharing it here, appreciate it!