mertcobanov
/

turkish-wordpiece-tokenizer

turkish-wordpiece-tokenizer / tokenizer_config.json

Upload 5 files

8b1439d verified about 2 months ago

115 Bytes

{"model_type": "wordpiece", "unk_token": "[UNK]", "bos_token": "[CLS]", "eos_token": "[SEP]", "pad_token": "[PAD]"}