fix: update chat_template to ChatML
#6
by
andriadze
- opened
I've tested locally with this chat_template and it worked fine.
I've also checked other models that use ChatML(e.g: https://huggingface.co/cognitivecomputations/dolphin-2.9.4-llama3.1-8b) and it also uses same prompt template.
I'm using vllm for inference, not SillyTavern, so I'm not sure how it will affect that.
This is the output that this jinja produces:
<|im_start|>system
This is the system prompt<|im_end|>
<|im_start|>user
This is user message<|im_end|>
<|im_start|>assistant
This is assistant message<|im_end|>
Thanks!
lucyknada
changed pull request status to
merged