tokenizer: fix number of extra ids
Browse filesIt's 300, as specified here:
https://github.com/google-research/t5x/blob/main/t5x/examples/scalable_t5/umt5/vocab.gin#L5
- tokenizer_config.json +1 -1
tokenizer_config.json
CHANGED
@@ -1 +1 @@
|
|
1 |
-
{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids":
|
|
|
1 |
+
{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 300}
|