Hugging Face's logo

tf-tpu
/
unigram-tokenizer-wikitext

Model card Files Community
1
sayakpaul HF staff carted-ml commited on
Commit
2ea9793
•
1 Parent(s): 99b7b7b

Create README.md (#1)

Browse files

- Create README.md (3a7416663dff6a6dfc56f31aff119de2bd3a925f)


Co-authored-by: Carted <carted-ml@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +1 -0
README.md ADDED
@@ -0,0 +1 @@
 
 
1
+ This is a [Unigram tokenizer](https://huggingface.co/course/chapter6/7?fw=pt) trained on the [Wikitext dataset](https://huggingface.co/datasets/wikitext). Refer to the `train_unigram.py` script within this repository to know how it was trained.