Perplexity

#1
by SekkSea - opened

I'm curious why 2.5bpw has a perplexity score of 10.5543, while 2.25bpw has a score of 8.8764?

I've noticed this before with lower quants of larger models, though not to this degree. I feel like there can be a bit of randomness as to how good a quant ends up and the lower end quants just aren't very consistent.

Sign up or log in to comment