phonemetransformers
/

BABYLM-TOKENIZER-BPE-TXT

File size: 109 Bytes

5bdc2b2

{
  "bos_token": "UTT_BOUNDARY",
  "eos_token": "UTT_BOUNDARY",
  "pad_token": "PAD",
  "unk_token": "UNK"
}