Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
ybelkada
/
japanese-dummy-tokenizer
like
1
Transformers
snow_simplified_japanese_corpus
English
Japanese
japanese
tokenizer
Inference Endpoints
License:
mit
Model card
Files
Files and versions
Community
1
Train
Deploy
Use this model
f342985
japanese-dummy-tokenizer
2 contributors
History:
2 commits
Younes Belkada
Add tokenizer file
f342985
over 2 years ago
japanese-dummy-tokenizer
Add tokenizer file
over 2 years ago
.gitattributes
1.22 kB
initial commit
over 2 years ago
tokenizer.py
1.11 kB
Add tokenizer file
over 2 years ago