Commit
•
3917979
1
Parent(s):
dbb7900
Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ datasets:
|
|
| 6 |
|
| 7 |
## Model Details
|
| 8 |
|
| 9 |
-
This model is an int4 model with group_size 128 and symmetric quantization of [THUDM/cogvlm2-llama3-chat-19B](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B).
|
| 10 |
## How To Use
|
| 11 |
### INT4 Inference
|
| 12 |
```python
|
|
@@ -28,7 +28,7 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
| 28 |
torch_dtype="auto",
|
| 29 |
trust_remote_code=True,
|
| 30 |
device_map=DEVICE,
|
| 31 |
-
##revision="
|
| 32 |
).to(DEVICE).eval()
|
| 33 |
|
| 34 |
content = '描述这张图片'
|
|
|
|
| 6 |
|
| 7 |
## Model Details
|
| 8 |
|
| 9 |
+
This model is an int4 model with group_size 128 and symmetric quantization of [THUDM/cogvlm2-llama3-chat-19B](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B). Load the model with revision="dbb7900" to use AutoGPTQ format.
|
| 10 |
## How To Use
|
| 11 |
### INT4 Inference
|
| 12 |
```python
|
|
|
|
| 28 |
torch_dtype="auto",
|
| 29 |
trust_remote_code=True,
|
| 30 |
device_map=DEVICE,
|
| 31 |
+
##revision="dbb7900" ##AutoGPTQ format
|
| 32 |
).to(DEVICE).eval()
|
| 33 |
|
| 34 |
content = '描述这张图片'
|