Update the tokenizer which inserted added_tokens to vocab 8196659 verified tuong-nguyen-prd commited on 21 days ago
fix(tokenizer): Also updates tokenizer.json to prevent any mismatch. 8a362e7 verified gugarosa commited on May 2