roberta-base / tokenizer.json

Commit History

fix: only use token_type id 0 for fast tokenizer
67dd433

monologg commited on

feat: add fast tokenizer
cb6e01e

monologg commited on