broken tokenizer
#1
by
mradermacher
- opened
just fyi, this model is broken - it has 32001 tokens, but the tokenizer model has 32002
just fyi, this model is broken - it has 32001 tokens, but the tokenizer model has 32002