special tokens
#2
by
kosung
- opened
Why does modeling_internlm_xcomposer2.py use [UNUSED_TOKEN_146] and [UNUSED_TOKEN_145], while these two are not in the special tokens? Instead, <|im_start|> and <|im_end|>.
kosung
changed discussion status to
closed