xmadai
/

Llama-3.1-8B-Instruct-xMADai-INT4

Text Generation

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

Oscar Wu commited on 27 days ago

Commit

1b33db0

•

1 Parent(s): 78b264a

Updated README

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -34,8 +34,8 @@ Loading the model checkpoint of this xMADified model requires less than 6 GiB of
 **Package prerequisites**: Run the following commands to install the required packages.
 ```bash
-pip install -q --upgrade transformers accelerate optimum
-pip install -q --no-build-isolation auto-gptq
 ```
 **Sample Inference Code**

 **Package prerequisites**: Run the following commands to install the required packages.
 ```bash
+pip install torch==2.4.0 transformers accelerate optimum
+pip install -vvv --no-build-isolation "git+https://github.com/PanQiWei/AutoGPTQ.git@v0.7.1"
 ```
 **Sample Inference Code**