PretrainedTokenizerFast has no attribute 'legacy'
#43
by
tian9
- opened
Hello, I want to change the LLAVA's base model from llama2 to llama3 and I encountered this problem during fine-tuning the pretrained model:
Should I recognize this as a version dismatch ? I have searched the doc in transformers for the 'legacy', but gain nothing. How can I resolve this problem?
I have tried to delete this 'if not tokenizer.legacy', but run .sh comes with a tokenizer mismatch.
tian9
changed discussion status to
closed