Model comparison

#12
by wh0t-is-love - opened

FYI. In my opinion, the gemma-2-27b-it-Q4_K_L.gguf model is better than the gemma-2-27b-it-Q5_K_L.gguf

Fascinating... Do you have any examples showing that? Would be very intriguing and I'm not sure why!

FYI. In my opinion, the gemma-2-27b-it-Q4_K_L.gguf model is better than the gemma-2-27b-it-Q5_K_L.gguf

I have tested several times and noticed that some models have this particular thing. The q4 or q5 responds better than q6 and q8. It's about half of them for me. I use LM Studio.

This comment has been hidden

Sorry, I can't provide examples, as they have an nda. But I can say that I have tried these models on the task of code generation and machine translation. In code generation, the code itself has not changed much, but its description has. In machine translation, q5 often had incomprehensible words and incorrectly translated technical terms.

Sign up or log in to comment