Amazing, thank you so much. Small question :)
#2
by
CyberTimon
- opened
Hello!
Thank you very much for these quants. They work amazing. I would LOVE to see Command-R Plus (the 104b variant) getting quantized with this method. On my 64GB M1 Max it would make an extremely huge difference because I can only load it in 3bit.
Thanks again and kind regards,
Timon
Thank you Timon for your feedback !
RE: CommandR Plus;
Currently this is beyond my hardware's ability, as I quant/build everything locally.
CommandR plus is on my list ; and should it be possible to "upgrade" it ; I will post it and ping you via this post.
Maybe you could tell me the secret to quantize it this good, then I can do it on my own. I have the compute to do this. Would be very thankful. Are you ok with this?