Perplexity
#1
by
SekkSea
- opened
I'm curious why 2.5bpw has a perplexity score of 10.5543, while 2.25bpw has a score of 8.8764?
I've noticed this before with lower quants of larger models, though not to this degree. I feel like there can be a bit of randomness as to how good a quant ends up and the lower end quants just aren't very consistent.