Hugging Face's logo

Spaces:
yhavinga
/
dutch-tokenizer-arena
Running

App Files Community
1
dutch-tokenizer-arena / utils

Commit History

Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
c78da21

yhavinga commited on

update
f331792

xu-song commited on

update
9558ae0

xu-song commited on

add compression leaderboard
1b7fc74

xu-song commited on

update compress rate
367a536

xu-song commited on

update compress rate
988921c

xu-song commited on

update
7d2062e

xu-song commited on

add grok mixtral
480ae5d

xu-song commited on

add compress rate
814ee6b

xu-song commited on

add more tokenizer
c75633b

xu-song commited on

update
9820e00

xu-song commited on

fix chatglm; new feature about add_special_tokens;
d27a756

xu-song commited on

update
bb1fd2b

xu-song commited on

add more tokenizers
f4973d4

xu-song commited on

update
9495a4f

xu-song commited on

update
8e0e4e9

xu-song commited on

update
79b95c3

xu-song commited on

update
819cf7f

xu-song commited on

update
d10ecd7

xu-song commited on

update
751936e

xu-song commited on