special tokens map is missing <PRE> <MID> <SUF>

by mljxy - opened Sep 6, 2023

Sep 6, 2023

The special_tokens_map.json is missing the extra 16 tokens. I believe at least we need <PRE>, <MID>, <SUF> defined there for infilling.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment