GGUF (ollama) version are far from API version

#20
by papipsycho - opened

Hello Guys,

I'm testing the GGUF version with ollama, but for now i'm getting result far from API version,

i try to tweak different kind setting such like temperature, top_p, i also try different version of quantize 5 to 8, but i'm not able to get the quality of the API version

do you have any idea what should i change to increase the quality ?

Sign up or log in to comment