I found that tokenizer of microsoft/Phi-3-medium-4k-instruct didn't add the bos_token by default like that from microsoft/Phi-3-mini-4k-instruct
So I assume that bos_token should be added to the beginning of the tokens ?
· Sign up or log in to comment
