Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
JustinLin610 
posted an update Feb 26
Post
https://qwen.readthedocs.io/ 🔥 The official doc of Qwen1.5 is coming! This is a bilingual doc (English and Chinese, and it will be multilingual if I have time for them). The doc includes instructions for simple inference, running locally with GGUF, ollama, etc., quantization, finetuning, deployment, etc. We will continue adding more stuff to the doc. Stay tuned!

im having trouble with quantization of qwen 0.5b model. i am getting an error while loading the quantized model
" terminate called after throwing an instance of 'std::out_of_range'
what(): _Map_base::at"

will you plz help me

·

gguf quantization?