Text Generation
Transformers
multilingual
Inference Endpoints

Cannot load tokenizer with `transformers<4.36.1` (TypeError: unhashable type: 'dict')

#4
by gegallego - opened

Hello,

Thanks for your important contribution to the field :D

I want to inform that the tokenizer is affected by an issue described here, which is solved in the Patch release v4.36.1.

Therefore, the model can only be used with transformers>=4.36.1, maybe you should specify it in the README.

Best regards,
Gerard

how do we by-pass the mismatch in vocabulary sizes when merging?

nevermind. fixed it

MaLA-LM org

@gegallego thanks. I'm adding the requirements.

jisx changed discussion status to closed

Sign up or log in to comment