Update README.md
Browse files
README.md
CHANGED
@@ -15,7 +15,7 @@ Changes:
|
|
15 |
1. Remove non-base model tokens
|
16 |
2. Keep/Add `<|pad|>` special token to make sure padding can be differentiated from eos/bos.
|
17 |
|
18 |
-
"
|
19 |
"content": "<|pad|>",
|
20 |
"lstrip": false,
|
21 |
"normalized": false,
|
|
|
15 |
1. Remove non-base model tokens
|
16 |
2. Keep/Add `<|pad|>` special token to make sure padding can be differentiated from eos/bos.
|
17 |
|
18 |
+
"100261": {
|
19 |
"content": "<|pad|>",
|
20 |
"lstrip": false,
|
21 |
"normalized": false,
|