why is pad token id in config.json set to null?

#16

by VedaantJain - opened Jul 23, 2024

Jul 23, 2024

The tokenizer_config has a pad_token as |endoftext| so not sure why config.json does not set the pad_token_id.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment