PLZ!😭When I run the template, I get the error“Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained."
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
@rimahazra The warning message here doesn't mean the model isn't being loaded correctly, it might be the new special tokens we added to the tokenizer for LLaVA. See for more https://github.com/huggingface/transformers/issues/7002, the message is now logged on info level in the latest transformers version. You can also turn off logging if you don't want to see the warning and don't want to upgrade transformers
@RaushanTurganbay , Thanks for your quick response. I am actually getting same issue after finetuning the llava. It was also giving warning that some weights could not be initialized.
Could please mention the exact version of transformer you are referring to?
@rimahazra I didn't test if the message is logged or not, but from the linked issue the version v4.43 and on should have the update (i.e. log on info level). Note if you enabled logging on info level, the message won't go anywhere
Btw, are you experiencing any quality degradation after you started seeing the message?