YAGPT / added_tokens.json
under-tree's picture
Upload tokenizer
13926a2
{
"</code>": 50259,
"<code>": 50258,
"<history>": 50264,
"<instructionE>": 50261,
"<instructionS>": 50260,
"<next>": 50262,
"<speaker1>": 50265,
"<speaker2>": 50266,
"<|endoftext|>": 50257,
"<|pad|>": 50263
}