fix: update chat_template to ChatML

#6
by andriadze - opened

I've tested locally with this chat_template and it worked fine.
I've also checked other models that use ChatML(e.g: https://huggingface.co/cognitivecomputations/dolphin-2.9.4-llama3.1-8b) and it also uses same prompt template.
I'm using vllm for inference, not SillyTavern, so I'm not sure how it will affect that.

This is the output that this jinja produces:
<|im_start|>system
This is the system prompt<|im_end|>
<|im_start|>user
This is user message<|im_end|>
<|im_start|>assistant
This is assistant message<|im_end|>

Anthracite org

Thanks!

lucyknada changed pull request status to merged

Sign up or log in to comment