special tokens
#2
by
kosung
- opened
Why does modeling_internlm_xcomposer2.py
use [UNUSED_TOKEN_146]
and [UNUSED_TOKEN_145]
, while these two are not in the special tokens? Instead, <|im_start|>
and <|im_end|>
.
kosung
changed discussion status to
closed