The impact of quantization on compliance…
#2
by
Tokie
- opened
…is huge.
Specifically, unlike the other quantizations I tested, MiquMaid-v2-2x70B-DPO.IQ3_XXS.gguf
is pretty much completely unrestrained:
This extends to MiquMaid-v2-70B-DPO
:
And to MiquMaid-v2-70B
:
But not to Miqu-70B-DPO
:
What is going on here? 🤔
No idea but this is very interesting to see, thank you for this!