koala-7B-HF / tokenizer_config.json

Commit History

Update tokenizer_config.json to match other models, like Vicuna - the Koala conversion process does not fill out this file, but based on the code I think these special tokens should be defined.
2d12271

TheBloke commited on

Initial commit.
06ee1d1

TheBloke commited on