Coherence
Thanks for uploading these, I've been testing q3_km and q6. Both seem incoherent enough to notice with my testing, and I tried Alpaca (clean + RP presets in ST) and ChatML so far.
The logic of the character seems disjointed in a way that doesn't make sense, such as confusing what some things do or swapping the roles and logic of characters or items around, or mistaking who is doing them based on the card description. Also, the model seems to like to talk with itself and just write questions and answer them in the same response, like normal sentences (no "char:" formatting, just adds a question to the normal quoted text and then answers it.
Alpaca clean is a bit better, but again the stories it comes up are so weird. For example, a character suddenly starts sending videos and pictures to the user, but this is never mentioned, and it really drills hard into writing 20 lines about sending a picture, then another picture, then a video, etc.
Overall, the behavior is strange and not quite coherent, and I expected going to Q6 to fix that, but it didn't. Anyone else experiencing this?
Try lowering the temp and top p. Temp = 0.5 and Top P = 0.8 seems to give me good results. You can also try the Dry versions (i.e., merged with the base) and see if that fixes it.
That crossed my mind but it didn't help as much as you'd expect (i have dynamic temp between 0.5 and 0.75 with similar results and my top p is 0.85 already, 0.8 seemed pretty much the same).
Are you using it with clean alpaca or some specific instruction set?
I will try the dried one next, thanks ^^
Jesus, the dried one (15%) straight-up murdered my {{user}} in the first response and put "The End" at the last line! :D I've never seen that before with my other models, hahah.
I think the incoherence is still quite present. Have you ran the GGUFs or just the safetensors?
I might try merging the HF model into a 9B frankstein with one of the last two I did that seem to be stable, and see if it gets any better, at least with the way I test it. I usually run 2-3 of the exact same character cards for tests and know what to expect, so I am not trying anything crazy different with this one to be fair. I also never really change sampler settings as the way I run them (DT from 0.75 to 1.75) generally has no issues with most models I try, but changing the settings doesn't seem to correct the behavior.
edit: I was able to get a lot more coherent results in LM studio with a very clean Alpaca preset, however it just doesn't really work for chatting so far, and goes quite bonkers with ST's more RP-based instruction set variants. The LM studio experience I got was not following the instructions for the story much, I requested something super specific and it went on to write a story about Donald Trump rally ^^ I guess that can be funny but it's a bit of a rebel with following what's going on
Go merge it with other models. I think the model's moist will stick.
Also, it turns out my v2 is broken.
https://huggingface.co/TheDrummer/Moistral-11B-v2#attention
I'm in the process of adding disclaimers and making a v2.1
Go merge it with other models. I think the model's moist will stick.
Also, it turns out my v2 is broken.
https://huggingface.co/TheDrummer/Moistral-11B-v2#attention
I'm in the process of adding disclaimers and making a v2.1
Ooh, OK, so I guess I was on the right track - in that case I will wait for your update and then likely use it for merges. Even this one is producing quite original results, just disjointed and confused, but I like this creative side a lot. Might just go ahead with the 25% for merges to test it out too now.
Even this one is producing quite original results, just disjointed and confused, but I like this creative side a lot.
Thanks! I'll keep on finetuning until I create the best moist machine. Then we'll go 70B next B-)