Hi, is it possible to come out with the IQ4_NL model?
#1
by
Eilian
- opened
I found it to be smaller, but closer to the Q5KM in performance.
Hmm, from the results I have seen, IQ4_NL is never better than Q4_K_S (for normal models), nor would it be expected to. Do you have some actual evidence (i.e. more than a feeling) that it is worth it? If yes, I can add it in general for models.
I'll add it to his repo in any case, it should be up in a few hours.