A newer version of the Gradio SDK is available:
4.43.0
metadata
title: Tiktoken Calculator
emoji: 🤗
colorFrom: yellow
colorTo: orange
sdk: gradio
python_version: 3.11
app_file: app.py
pinned: false
tags:
- gpt
- tiktoken
- tokenizer
Tiktoken Calculator
Calculate the token count for GPT-4, GPT-3.5, GPT-3, and GPT-2.
It uses openai/tiktoken to calculate the token count.
How to use
HuggingFace Spaces
Go to https://huggingface.co/spaces/JacobLinCool/tiktoken-calculator and try it out!
Docker
There is a pre-built Docker image available on Docker Hub: https://hub.docker.com/r/jacoblincool/tiktoken-calculator
docker run -p 7860:7860 jacoblincool/tiktoken-calculator
If you prefer to use Docker Compose, you can clone this repository and run:
docker compose up -d
Local
I use Poetry to manage dependencies.
Setup the virtual environment after cloning this repository:
poetry install
Then run the app:
poetry run python app.py