AWQ Quant

#4
by andriadze - opened

Hey guys here's an AWQ quant if you care about it:
https://huggingface.co/andriadze/anthracite-magnum-v3-34b-awq

It's a GEMM version with group size of 128.

Anthracite org

Thank you so much for sharing this quant!

Sign up or log in to comment