Empty response
#9
by
Ahmad223
- opened
Hello All,
The model at sometimes generates empty response.
Did anyone faced this issue before
Hello @alexrs
Thanks for your help.
How are you running the model?
- I'm using the model as a simple RAG setup with FIASS as a vector store DB. The code running well with the previous version of the model (V1) but the issue appears once I tried the august version.
What prompts are you using?
- I use the default prompt with more instructions to restrict the model from generate answers outside the files.
What hardware are you using?
- It's 2x A100 GPU with 40G each. and I quantized the model with 8bit to fit the HW.