GPT-Q quants
#1
by
CyberTimon
- opened
Hello
Thank you for this model.
Can someone maybe create GPT-Q quants so I can run it in like 4bit?
Thank you very much!
Timon Käch
TheBloke is doing them now. I'm closing this
CyberTimon
changed discussion status to
closed