Rerun the conversion
#3
by
michaelfeil
- opened
FYI we just upgraded our weights. The alignment should be much better now. Thanks for the 2bit quant
Thanks for the notification. We will do it asap :)
Thank you @sharpenb !
The model is updated ;) Since 2 bits quant is quite aggressive, it might still need some PEFT to recover from it.
sharpenb
changed discussion status to
closed
Yeah, I think for long context, even int8 might be too agressive