File size: 1,231 Bytes
e701494 d3ffb36 e701494 d3ffb36 f572f8e d3ffb36 e701494 e4f6a3b e701494 e4f6a3b e701494 d3ffb36 e701494 575f449 d3ffb36 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
---
language:
- ar
- en
model_name: AceGPT 7B Chat
model_type: AceGPT
quantized_by: FreedomIntelligence
---
<!-- description start -->
## Description
This repo contains int4 model(GPTQ) for [AceGPT-7B-Chat](https://huggingface.co/FreedomIntelligence/AceGPT-7B-chat).
The performance of the int4 version has experienced some degradation. For a better user experience, please use the fp16 version.
For details, see [AceGPT-7B-Chat](https://huggingface.co/FreedomIntelligence/AceGPT-7B-chat) and [AceGPT-13B-Chat](https://huggingface.co/FreedomIntelligence/AceGPT-13B-chat).
<!-- description end -->
<!-- README_GPTQ.md-use-from-python start -->
## How to use this GPTQ model from Python code
### Install the necessary packages
Requires: Transformers 4.32.0 or later, Optimum 1.12.0 or later, and AutoGPTQ 0.4.2 or later.
```shell
pip3 install transformers>=4.32.0 optimum>=1.12.0 #See requirements.py for verified versions.
pip3 install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/ # Use cu117 if on CUDA 11.7
```
### You can then generate a simple gradioweb with_quant.py
```bash
python web_quant.py --model-name ${model-path}
```
<!-- README_GPTQ.md-use-from-python end --> |