dclm-7b-it / tokenizer_config.json
EtashGuha's picture
Added chat templating to tokenizer
da63c0d verified
raw
history blame
450 Bytes
{"chat_template": "Below is an instruction that describes a task.\n\nWrite a response that appropriately completes the request.\n\n### Instruction:\n{{if messages[-1].role == 'user'}}{{messages[-1].content}}{{else}}{{raise_exception('Last message must be from user')}}{{/if}}\n\n### Response:", "unk_token": "<|endoftext|>", "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "add_prefix_space": false, "tokenizer_class": "GPTNeoXTokenizer"}