broken tokenizer

#1
by mradermacher - opened

just fyi, this model is broken - it has 32001 tokens, but the tokenizer model has 32002

Sign up or log in to comment