KnutJaegersberg

/

Deacon-34B-qlora

like 7

Glad you like it. I tried gptq already on a model merged with the adapter, but it said something like llama was expecting an llm.header layer and then instantiated the layer itself. The same for awq.
The model that came out of that only generated rubbish, currently I don't know how to circumvent this issue.

I asked @TheBloke to help out

@CyberTimon you can try https://huggingface.co/KnutJaegersberg/Deacon-34B-200k-AWQ instead :)