infinitejoy's picture
add tokenizer
9a405c3
raw
history blame
23 Bytes
{"<s>": 40, "</s>": 41}