Mixed results with the quants
I'm getting mixed results with the quants (the ai feel a bit "crazy" for a lack of a better word), even at q5 quants. It feels a bit like the synthia model released earlier. Is there any suggested parameters to get the best results ?
Just use Alpaca prompting, it's still very experimental, I'm looking how to fix the issue and made low quant to try them too!
Log will help if you have any to see where is exactly the problem, thanks!
Thanks for the answer ! Btw if you can bring back the v2 of this model that would be appreciated (unless of course you are working on yet another 4x7B lol). I was getting pretty decent results with q3 + alpaca and I was curious to see how the q2 performed.
Thanks for the answer ! Btw if you can bring back the v2 of this model that would be appreciated (unless of course you are working on yet another 4x7B lol). I was getting pretty decent results with q3 + alpaca and I was curious to see how the q2 performed.
It can output good shit but sadly Ashhwriter was pushing out of context [Things] like this, so I passed it private, maybe you where lucky!
I will put it public for you to download if needed, will make it private again tomorrow!
Oh interesting, I didn't run into any [Things] when I was messing around with it. If you could put it back up for a bit, that'd be awesome
Done! GGUF is downloadable again
Edit: I made it private again, it's not so good.
Downloaded, thanks a lot !