Add tokenizer.json file

This should enable easy loading tokenizer from rust.

Files changed (1) hide show

tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff