open_llama_3b / tokenizer.json
danielhanchen's picture
Enable LlamaTokenizerFast and AutoTokenizer to load in seconds rather than 5 minutes.
379edd8
raw history
No virus
1.99 MB
File too large to display, you can check the raw version instead.