it'd be awesome if nf4 was provided

#2
by MayensGuds - opened

i found it to be really good, faster than any other quant, with decent speed, and comparable quality

Owner

I don't have any experience with BNB NF4 outside using it with transformers, so I can't really help with this. Maybe asking in the original repo or one of the NF4 repos would be better?

Sign up or log in to comment