没有对tokenizer做修改嘛

#3
by jeff31415 - opened

这样的话一个汉字怕不是占2-3个token,无论是训练还是推理都有点低效啊

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment