FYI: I quantized this with AutoAWQ
#5
by
tachyphylaxis
- opened
In case you wanted to add it along with info on other quantizations:
tachyphylaxis/QuartetAnemoi-70B-t0.0001-AWQ
Haven't really used it yet, but I did load it and it doesn't output garbage, so ... ;-)
Thanks, added to the card. We made an AWQ, but all I got was single token output in testing so I never made it public.
Lol, it's so funny, that happened to me, but then I just did it again and it worked (?)