Update tokenizer_config.json

#7
by ybelkada - opened
BigScience Workshop org
No description provided.
Muennighoff changed pull request status to merged
BigScience Workshop org

It's annoying that we're missing indent here ... Whoever is writing this file should probably have indent option set.

BigScience Workshop org

+100 (and cc @Narsil )

BigScience Workshop org

Just to be sure, I think tokenizer_config.json comes from transformers, and it's supposed to be fixed for quite a while: https://github.com/huggingface/transformers/blame/862888a35834527fed61beaf42373423ffdbd216/src/transformers/tokenization_utils_base.py#L2155 @ybelkada did you perhaps change this manually instead of going through transformers?

Sign up or log in to comment