squeeze-ai-lab

/

sq-llama-30b-w3-s45

like 1

You offer an intriguing solution for compressing models, but sharing the essential quantisation code would be more beneficial. Uploading pre-quantized models alone is not as valuable since users are interested in obtaining the base model and having the flexibility to fine-tune and quantize it according to their specific requirements. This has been expressed on the GitHub repository many times #12 #7