tokenizer.json will be required for conversions (for example to CT2 format). It also makes it a consistent file type across different whisper models.

Thanks for this

ylacombe changed pull request status to merged

Sign up or log in to comment