allenai

/

Molmo-7B-D-0924

like 461

Follow

Ai2 1.76k

I am not from the allenai team, but since this model uses Qwen as LM, you can use the Qwen chat template (here: https://huggingface.co/Qwen/Qwen2-7B/blob/main/tokenizer_config.json) and then just use the tokenizers builtin function:

from transformers import AutoProcessor
processor = AutoProcessor.from_pretrained("allenai/Molmo-7B-D-0924", trust_remote_code=True)
processor.tokenizer.chat_template = <insert chat template here>
processor.tokenizer.apply_chat_template(conversation=conversation)

where conversation is your list of messages in a multi-turn conversation.

Perhaps there is better options, but this works :)

@chrishoertnagl Thanks,Can you provide a specific example?

Running this

from transformers import AutoProcessor
processor = AutoProcessor.from_pretrained("allenai/Molmo-7B-D-0924", trust_remote_code=True)
processor.tokenizer.chat_template = "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"

conversation = [{"role": "system", "content": "You are an AI ..."}, 
                {"role": "user", "content": "Please help me with ..."}, 
                {"role": "assistant", "content": "Sure, here is ..."},
                {"role": "user", "content": "Could you also ..."}]
result = processor.tokenizer.apply_chat_template(conversation=conversation, tokenize=False, add_generation_prompt=True)
print(result)

Should give:

<|im_start|>system
You are an AI ...<|im_end|>
<|im_start|>user
Please help me with ...<|im_end|>
<|im_start|>assistant
Sure, here is ...<|im_end|>
<|im_start|>user
Could you also ...<|im_end|>
<|im_start|>assistant

If you want to pass it to the model, you set tokenize=True and just keep appending messages to your conversation with the respective roles. Hope that is what you were looking for :)

@chrishoertnagl Thanks，I try.It would be great if you could provide an example that can be run directly.