grafail commited on
Commit
14af36c
1 Parent(s): b1bcda6

Updated eos_token to <|end|>

Browse files

Wrong token causes issues with https://github.com/huggingface/text-generation-inference, as it cannot be easily overridden.

Files changed (1) hide show
  1. special_tokens_map.json +1 -1
special_tokens_map.json CHANGED
@@ -6,6 +6,6 @@
6
  "<|end|>"
7
  ],
8
  "bos_token": "<|endoftext|>",
9
- "eos_token": "<|endoftext|>",
10
  "unk_token": "<|endoftext|>"
11
  }
 
6
  "<|end|>"
7
  ],
8
  "bos_token": "<|endoftext|>",
9
+ "eos_token": "<|end|>",
10
  "unk_token": "<|endoftext|>"
11
  }