Hugging Face's logo

llava-hf
/
llava-v1.6-mistral-7b-hf

Image-Text-to-Text
Transformers
Safetensors
English
llava_next
vision
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
43

Llava on llama.cpp?

#30
by danielelongo - opened
Discussion
danielelongo

Hello, is there a quantized version of llava that can run locally on a CPU with llama.cpp?

RaushanTurganbay
 Llava Hugging Face org

@danielelongo Hey! we don't have gguf converted weights in the repo currently, but I saw some for llava-1.6 in https://huggingface.co/cjpais. Let me know if that help but in any case, the conversion script is available in llama.cpp repo if you wanna convert youself :)

danielelongo

Thanks!

danielelongo changed discussion status to closed

Sign up or log in to comment