Why does modeling_internlm_xcomposer2.py use [UNUSED_TOKEN_146] and [UNUSED_TOKEN_145], while these two are not in the special tokens? Instead, <|im_start|> and <|im_end|>.
modeling_internlm_xcomposer2.py
[UNUSED_TOKEN_146]
[UNUSED_TOKEN_145]
<|im_start|>
<|im_end|>
· Sign up or log in to comment
