Can't load in LM studio
Just tried it with it LM studio and got this error loading it. Any solution?
"llama.cpp error: 'error loading model vocabulary: unknown pre-tokenizer type: 'mistral-bpe''"
Its not supported by llamacpp yet, so anything based on llamacpp such as KoboldCpp won't be able to run this.
Other than llamacpp and it's derivatives, what else supports gguf quants?
Quoting GTP4o :
"While the Mistral-Nemo-Instruct-2407-GGUF model is not currently supported by llama.cpp and hence cannot be run on LMStudio, you have several other options. Using the Hugging Face transformers library directly, converting the model for use with ONNX Runtime, leveraging cloud-based services like AWS SageMaker, or setting up a local Docker environment are all viable alternatives to run this model."
Disclaimer : I haven't tried any of the above options, though i'm inclined to try it with Docker and transformers.
The gguf models have already updated, which are based on llama.cpp b3438. If any further issue, please let us know. Thanks a lot!
The gguf models have already updated, which are based on llama.cpp b3438. If any further issue, please let us know. Thanks a lot!
Downloaded today's model, now getting this error: "llama.cpp error: 'error loading model hyperparameters: invalid n_rot: 128, expected 160'" on LM Studio.
Did you use llama.cpp b3438? BTW, you can try to set context size to 4096 instead of 128K when you test.
Did you use llama.cpp b3438? BTW, you can try to set context size to
4096instead of128Kwhen you test.
Same error for me on the latest koboldcpp.
The gguf models have already updated, which are based on llama.cpp b3438. If any further issue, please let us know. Thanks a lot!
Downloaded today's model, now getting this error: "llama.cpp error: 'error loading model hyperparameters: invalid n_rot: 128, expected 160'" on LM Studio.
Same error in LM Studio.
Same error for me
I had the same issue with lmstudio. Upgrade to 0.2.28 did the trick.