unigram-tokenizer-wikitext / tokenizer_config.json

Commit History

add tokenizer
99b7b7b

sayakpaul HF staff commited on