neuralmagic
/

TinyLlama-1.1B-Chat-v1.0-marlin

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

robertgshaw2 commited on Mar 6

Commit

29e8c23

•

1 Parent(s): c7713ac

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -63,7 +63,7 @@ For details on how this model was quantized and converted to marlin format, run
 ```bash
 pip install -r quantization/requirements.txt
-CUDA_VISIBLE_DEVICES=0 python3 quantization/apply_gptq_save_marlin.py --model-id TinyLlama/TinyLlama-1.1B-Chat-v1.0 --save-dir ./tinyllama-marlin
 ```
 ## Slack

 ```bash
 pip install -r quantization/requirements.txt
+python3 quantization/apply_gptq_save_marlin.py --model-id TinyLlama/TinyLlama-1.1B-Chat-v1.0 --save-dir ./tinyllama-marlin
 ```
 ## Slack