ArthurConmy
commited on
Commit
•
b389e23
1
Parent(s):
c76d915
Update README.md
Browse files
README.md
CHANGED
@@ -1,4 +1,4 @@
|
|
1 |
-
This is
|
2 |
|
3 |
# Neel's README
|
4 |
|
|
|
1 |
+
This is https://huggingface.co/NeelNanda/gpt-neox-tokenizer-digits but with a fix that makes this work on `tokenizers == 0.14` after a breaking change involving `added_tokens`: https://github.com/huggingface/tokenizers/issues/1358
|
2 |
|
3 |
# Neel's README
|
4 |
|