prem-1B-chat / added_tokens.json
goku's picture
Upload tokenizer
299c2c9 verified
raw
history blame
106 Bytes
{
"<|end_header_id|>": 32001,
"<|eot_id|>": 32002,
"<|start_header_id|>": 32000,
"[PAD]": 32003
}