anthracite-org

/

magnum-v3-34b

like 28

Follow

Anthracite 257

I've tested locally with this chat_template and it worked fine.
I've also checked other models that use ChatML(e.g: https://huggingface.co/cognitivecomputations/dolphin-2.9.4-llama3.1-8b) and it also uses same prompt template.
I'm using vllm for inference, not SillyTavern, so I'm not sure how it will affect that.

This is the output that this jinja produces:
<|im_start|>system
This is the system prompt<|im_end|>
<|im_start|>user
This is user message<|im_end|>
<|im_start|>assistant
This is assistant message<|im_end|>