Hugging Face's logo

SeanScripts
/
Llama-3.2-11B-Vision-Instruct-nf4

Image-Text-to-Text
Transformers
Safetensors
mllama
conversational
text-generation-inference
Inference Endpoints
4-bit precision
bitsandbytes
Model card Files Community
1

I am trying to convert the model into BitsAndBytes with NF4 (4-bit)

#1
by sabaridsnfuji - opened
Discussion
sabaridsnfuji

Could you guide me on how to save a fine-tuned LoRA model for Llama-3.2-11B-Vision-Instruct in 4-bit precision for optimized inference, similar to the repository SeanScripts/Llama-3.2-11B-Vision-Instruct-nf4 ?

Sign up or log in to comment