File size: 301 Bytes
9078586 3010e32 3eac7da 3010e32 6016916 |
1 2 3 4 5 6 7 8 9 10 11 12 |
---
license: apache-2.0
tags:
- 2bit
- llama
- yi
- 34b
---
You can run it on 11G mem GPU,quantize base QuIP# method, a weights-only quantization method that is able to achieve near fp16 performance using only 2 bits per weight.
url:https://github.com/Cornell-RelaxML/quip-sharp/tree/release20231203 |