deliberate-openvino / tokenizer

Commit History

Rename tokenizer/tokenizer_tokenizer_config (1).json to tokenizer/tokenizer_config.json
a94e348

ehristoforu commited on

Rename tokenizer/tokenizer_special_tokens_map (1).json to tokenizer/special_tokens_map.json
ed985c6

ehristoforu commited on

Rename tokenizer/tokenizer_merges (1).txt to tokenizer/merges.txt
5fdf955

ehristoforu commited on

Upload 3 files
ee39c62

ehristoforu commited on

Rename tokenizer_vocab (1).json to tokenizer/vocab.json
7b18326

ehristoforu commited on