Update README.md
Browse files
README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
-
Quantized
|
2 |
|
3 |
-
|
4 |
|
5 |
Command used to quantize:
|
6 |
```python llama.py c:\llama-30b-supercot c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors 4bit-128g.safetensors```
|
|
|
1 |
+
Quantized version of this: https://huggingface.co/ausboss/llama-30b-supercot
|
2 |
|
3 |
+
GPTQ quantization using https://github.com/0cc4m/GPTQ-for-LLaMa for compatibility with 0cc4m's fork of KoboldAI
|
4 |
|
5 |
Command used to quantize:
|
6 |
```python llama.py c:\llama-30b-supercot c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors 4bit-128g.safetensors```
|