5pbw quant request
#3
by
OrangeApples
- opened
@brucethemoose thanks for your work! I enjoyed using the Q4KM iMat quant you uploaded but it seemed quite unstable to me. Sometimes I get emojis and weird formatting issues with that one. I expected that anyways since you warned about the iMat ggufs being experimental.
May I request for you to please upload a 5bpw exl2 of this? Can probably fit 10k context in my 24GB VRAM which is okay with me especially for new chats.