Aeala commited on
Commit
9fe5a8d
1 Parent(s): bf538f4

Add oasst tokens.

Browse files
Files changed (1) hide show
  1. added_tokens.json +7 -0
added_tokens.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "<|assistant|>": 32004,
3
+ "<|prefix_begin|>": 32000,
4
+ "<|prefix_end|>": 32003,
5
+ "<|prompter|>": 32002,
6
+ "<|system|>": 32001
7
+ }