nlgp-natural / added_tokens.json
Geert's picture
add tokenizer
60c42b0
raw
history blame contribute delete
240 Bytes
{"<|8space|>": 50265, "<|endofcomment|>": 50267, "<|6space|>": 50264, "<|12space|>": 50258, "<|cell|>": 50266, "<|10space|>": 50257, "<|14space|>": 50259, "<|2space|>": 50262, "<|4space|>": 50263, "<|16space|>": 50260, "<|18space|>": 50261}