---
license: apache-2.0
---
This is 2-bit quantization of [Qwen/Qwen1.5-72B-Chat](https://huggingface.co/Qwen/Qwen1.5-72B-Chat) using QuIP#

Random samples from RedPajama and Skypile (for Chinese) are used as calibration data.

### Model loading

Please follow the instruction of [QuIP-for-all](https://github.com/chu-tianxiang/QuIP-for-all) for usage.

As an alternative, you can use [Aphrodite engine](https://github.com/PygmalionAI/aphrodite-engine) or my [vLLM branch](https://github.com/chu-tianxiang/vllm-gptq) for faster inference. If you have problem installing `fast-hadamard-transform` from pip, you can also install it from [source](https://github.com/Dao-AILab/fast-hadamard-transform)