INST problem?
#2
by
Slayery
- opened
This model also spams INST like all others that have traces of neuraltrix ? (I mean gguf, it doesn't seem to be present on non-quantized models).
I will try to quantize it today and see how it performs
I quantized the model and it definitely has this problem. I think we should focus on finding the root model of the issue.
Here is the quantized version: paulml/OGNO-7B-GGUF
I can also reproduce the INST loop error with Q5_K_M quantization type generated with llama.cpp version b2257.