Hugging Face's logo

Spaces:
yhavinga
/
dutch-tokenizer-arena
Running

App Files Community
1
dutch-tokenizer-arena / vocab /albert /__init__.py
eson's picture
eson
update
751936e
raw history blame contribute delete
No virus
105 Bytes
"""
SentencePiece(unigram)
https://huggingface.co/docs/transformers/tokenizer_summary#sentencepiece
"""