Where is the tokenizer?
#2
by
EarthWorm001
- opened
I'm trying to run the Mamba-2 hybrid using megatron, However, the example scripts require a tokenizer prepared. Where to download the tokenizer?
Thanks!
The tokenizer is uploaded in each repository under the filename mt_nlg_plus_multilingual_ja_zh_the_stack_frac_015_256k.model
rwaleffe
changed discussion status to
closed