Update README.md
Browse files
README.md
CHANGED
@@ -36,27 +36,30 @@ It is also expected to be **VERY SLOW**. This is unavoidable at the moment, but
|
|
36 |
|
37 |
To use it you will require:
|
38 |
|
39 |
-
1. AutoGPTQ
|
40 |
2. `pip install einops`
|
41 |
|
42 |
You can then use it immediately from Python code - see example code below - or from text-generation-webui.
|
43 |
|
44 |
## AutoGPTQ
|
45 |
|
46 |
-
|
|
|
|
|
|
|
47 |
```
|
48 |
git clone https://github.com/PanQiWei/AutoGPTQ
|
49 |
cd AutoGPTQ
|
50 |
pip install .
|
51 |
```
|
52 |
|
53 |
-
|
54 |
|
55 |
## text-generation-webui
|
56 |
|
57 |
There is also provisional AutoGPTQ support in text-generation-webui.
|
58 |
|
59 |
-
This requires text-generation-webui as of commit 204731952ae59d79ea3805a425c73dd171d943c3.
|
60 |
|
61 |
So please first update text-genration-webui to the latest version.
|
62 |
|
|
|
36 |
|
37 |
To use it you will require:
|
38 |
|
39 |
+
1. AutoGPTQ
|
40 |
2. `pip install einops`
|
41 |
|
42 |
You can then use it immediately from Python code - see example code below - or from text-generation-webui.
|
43 |
|
44 |
## AutoGPTQ
|
45 |
|
46 |
+
Please install AutoGPTQ version 0.2.1 or later: `pip install auto-gptq`
|
47 |
+
|
48 |
+
If you have any problems installing AutoGPTQ with CUDA support, you can try compiling manually from source:
|
49 |
+
|
50 |
```
|
51 |
git clone https://github.com/PanQiWei/AutoGPTQ
|
52 |
cd AutoGPTQ
|
53 |
pip install .
|
54 |
```
|
55 |
|
56 |
+
The manual installation steps will require that you have the [Nvidia CUDA toolkit](https://developer.nvidia.com/cuda-12-0-1-download-archive) installed.
|
57 |
|
58 |
## text-generation-webui
|
59 |
|
60 |
There is also provisional AutoGPTQ support in text-generation-webui.
|
61 |
|
62 |
+
This requires a text-generation-webui as of commit 204731952ae59d79ea3805a425c73dd171d943c3.
|
63 |
|
64 |
So please first update text-genration-webui to the latest version.
|
65 |
|