--- title: SuperBPE Playground emoji: 📝 colorFrom: indigo colorTo: purple sdk: static pinned: false short_description: Experiment with and compare different tokenizers models: - Xenova/gpt-4o - Xenova/llama-3-tokenizer - UW/OLMo2-8B-SuperBPE-t180k - UW/OLMo2-11B-SuperBPE-t180k - UW/OLMo2-8B-SuperBPE-t160k - UW/OLMo2-8B-SuperBPE-t80k - UW/OLMo2-8B-BPE --- Link to source code: https://github.com/xenova/transformers.js/tree/main/examples/tokenizer-playground.