BOS token

by 1AH - opened 13 days ago

1AH

13 days ago

 "eos_token_id": 151645,
  "bos_token_id": 151645,

is the bos token really "<|im_end|>"?

Owner 13 days ago

You're right. That's a mistake, it should be token ID 151643. Thanks for catching.

alpindale changed discussion status to closed 13 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment