TheBloke/CodeLlama-34B-Instruct-GGUF only generates "text": "\n\n\n\n\n\n\n\n
#4
by
hiteshsom
- opened
Hi I am using Langchain agents and in that I am using TheBloke/CodeLlama-34B-Instruct-GGUF llm. But it only gives output
"text": "\n\n\n\n\n\n\n\n
However mistral-7b-instruct-v0.1.Q6_K.gguf worked fine.
Environment:
CPU
RAM: 32 GB
Windows
Had the same problem, mostly it repeated newline characters, but sometimes it started to repead whole sentences of the response.
Also it went into an infinite loop (or i did not have enough patience and killed the process early).
Fixed it by adjusting my ropeconfig:
localAI yaml style:
rope_freq_base: 1000000
rope_freq_scale: 1.0
Original comment that lead me to the fix:
https://huggingface.co/TheBloke/Phind-CodeLlama-34B-v2-GGUF/discussions/2