This repo contains the tokenizer for the Llama-2 models. The tokenizer is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form.
This repo contains the tokenizer for the Llama-2 models. The tokenizer is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form.