yhavinga's picture
Hack some Dutch tokenizers into it
55df72d
raw
history blame contribute delete
133 Bytes
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("GroNLP/gpt2-small-dutch", trust_remote_code=True)