test_ted / special_tokens_map.json
Lakoc's picture
Upload tokenizer
cf08247 verified
raw
history blame contribute delete
135 Bytes
{
"bos_token": "([bos])",
"eos_token": "([eos])",
"mask_token": "([mask])",
"pad_token": "([pad])",
"unk_token": "([unk])"
}