Correct mistral format
Hi,
What mistral format do we need to use for this model? I am using koboldcpp and there are r mistral formats available.
Mistral V1
User: " [INST] "
assistant: " [/INST]"
Mistral V2 & V3
user: "[INST] "
assistant: "[/INST]"
Mistral V3-Tekken
user: "[INST]"
assistant: "[/INST]"
Which of these should we use?
Thanks.
Hi,
What mistral format do we need to use for this model? I am using koboldcpp and there are r mistral formats available.
Mistral V1
User: " [INST] "
assistant: " [/INST]"Mistral V2 & V3
user: "[INST] "
assistant: "[/INST]"Mistral V3-Tekken
user: "[INST]"
assistant: "[/INST]"Which of these should we use?
Thanks.
Somebody posted this in Reddit and it helps remedy some of the confusion:
https://github.com/mistralai/cookbook/blob/main/concept-deep-dive/tokenization/chat_templates.md
Tokenizer V3
This tokenizer powers models such as Mixtral 8x22B, Codestral 22B, Mathstral 7B, Mamba Codestral 7B, Small 2409 and Large 2 (Large 2407).
Tekken
Tekken is a different version of the V3 tokenizer and powers Mistral Nemo.
So looks to be V2/V3.