internlm/internlm2* models changed it's tokenizer name to tokenization_internlm2.InternLM2Tokenizer
0bd41c9
verified
eduagarcia
commited on