= commited on
Commit
3c4b987
1 Parent(s): c873531

adding tokenizers

Browse files
wolof-translate/wolof_translate/tokenizers/__init__.py ADDED
File without changes
wolof-translate/wolof_translate/tokenizers/__pycache__/__init__.cpython-310.pyc ADDED
Binary file (255 Bytes). View file
 
wolof-translate/wolof_translate/tokenizers/adverse_tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
wolof-translate/wolof_translate/tokenizers/t5_tokenizers/fr_tokenizer_v1.json ADDED
The diff for this file is too large to render. See raw diff
 
wolof-translate/wolof_translate/tokenizers/t5_tokenizers/tokenizer_v1.json ADDED
The diff for this file is too large to render. See raw diff
 
wolof-translate/wolof_translate/tokenizers/t5_tokenizers/tokenizer_v2.json ADDED
The diff for this file is too large to render. See raw diff
 
wolof-translate/wolof_translate/tokenizers/t5_tokenizers/tokenizer_v3.json ADDED
The diff for this file is too large to render. See raw diff
 
wolof-translate/wolof_translate/tokenizers/t5_tokenizers/tokenizer_v4.json ADDED
The diff for this file is too large to render. See raw diff
 
wolof-translate/wolof_translate/tokenizers/t5_tokenizers/wf_tokenizer_v1.json ADDED
The diff for this file is too large to render. See raw diff
 
wolof-translate/wolof_translate/tokenizers/tokenizer_v1.json ADDED
The diff for this file is too large to render. See raw diff