qualcomm

/

Llama-v2-7B-Chat

like 22

Follow

Qualcomm 32

Hi all. I have LM Studio installed and can run many SLMs on CPU mode, including Llama-v2-7B-Chat and PHi3 and even a 20B model... BUT the model doesn't use the GPU or NPU.
Closest I've come to finding a working llama.cpp file for SnapDragon on windows is https://github.com/ggerganov/llama.cpp/discussions/8336#discussioncomment-10472433

Then I joined Qualcomm tech program and found this document...
https://docs.qualcomm.com/bundle/publicresource/topics/80-62010-1/genai-llama-cpp.html
but there is currently no LLVM ARM version pre-compiled.

I would rather develop my agents and compare SLMs than build these fundamental tools in C++. But it looks like I might have to. If there is a SnapDragon llama.cpp download for windows native or WSL, I'm keen to try it.