Doron Adler commited on
Commit
209851b
1 Parent(s): 8af0c8b

added_tokens.json

Browse files
Files changed (1) hide show
  1. added_tokens.json +1 -1
added_tokens.json CHANGED
@@ -1 +1 @@
1
- {"<|endoftext|>": 50257, "<|startoftext|>": 50258}
 
1
+ {"<|unknown|>": 50259, "<|startoftext|>": 50258, "<|endoftext|>": 50257}