Merge branch 'main' of https://huggingface.co/OPEA/llama-joycaption-alpha-two-hf-llava-int4-sym-inc into main

README.md

@@ -5,7 +5,7 @@ datasets:
 
 ## Model Details
 
-This model is an int4 model with group_size 128 and symmetric quantization of [fancyfeast/llama-joycaption-alpha-two-hf-llava](https://huggingface.co/fancyfeast/llama-joycaption-alpha-two-hf-llava) generated by [intel/auto-round](https://github.com/intel/auto-round). Load the model with revision="" to use AutoGPTQ format.
+This model is an int4 model with group_size 128 and symmetric quantization of [fancyfeast/llama-joycaption-alpha-two-hf-llava](https://huggingface.co/fancyfeast/llama-joycaption-alpha-two-hf-llava) generated by [intel/auto-round](https://github.com/intel/auto-round). Load the model with revision="bc917a8" to use AutoGPTQ format.
 
 ## How To Use
 
@@ -25,7 +25,11 @@ quantized_model_path="OPEA/llama-joycaption-alpha-two-hf-llava-int4-sym-inc"
 
 # Load JoyCaption INT4 Model
 processor = AutoProcessor.from_pretrained(quantized_model_path)
-model = LlavaForConditionalGeneration.from_pretrained(quantized_model_path, device_map=0)
+model = LlavaForConditionalGeneration.from_pretrained(
+    quantized_model_path,
+    device_map="auto",
+    revision="bc917a8" ## ##AutoGPTQ format
+)
 model.eval()
 
 image_url = "http://images.cocodataset.org/train2017/000000116003.jpg"