2.3 have problems.
For now, I got report that 2.3 is broken on some install/preset.
Try : Mirostat 1 on Oobabooga (if you use it) this is working on my install.
If it don't work, report it here and get back to 2.2, I'm working on it to release something more stable but still uncensored as hell.
Thank you all for your feedback!
Is a predisposition to repeat itself one of these problems (I've mostly used mirostat 2 with koboldcpp)?
Is a predisposition to repeat itself one of these problems (I've mostly used mirostat 2 with koboldcpp)?
From the feedback I got, it sometime give gibberish for any prompt given, repetition and loop probably happen too. Weird because there's no duplicated data, but hey, if it happen it happen!
Mirostat 1 is recommanded (work on my case) but yes, after longer testing I find the model hard to use properly out of the box.
I'm sorry, will do better.
In the meantime just go back to 2.2, go to MLewdBoros or try to bruteforce a preset that would work.
I have to admit the merge was really dirty and all the data float a little around kek, I have multiple project so I dunno when v2.4 will be out.
MLewd 2.3 is great. Actually the best model I tried out so far. It's even better then the later merges like CoT, Chat-1, Chat2, so far. I too had some huge problems in the beginning but they all just boiled down to MLewd being very sensitive with repetition penalties. It works better without mirostat to my experience. The trick is: Put repetition penalty not higher than 1.09. This, by the way, is mandatory for each model based on Mythomax, and set repetition penalty range to 2048. Oh, and don't set temperature too high. values between 0.8 and 1.0 are doing great. With those settings it has been the most stable llama-2 model I've tested so far. Oh, I run it on KoboldCPP by the way, without using any rope settings of course.
Missed your reply. Thanks for letting me know. I never tried mirostat 1, only 2. Yes, exactly what you describe is what happens. It is very unpredictable, too--except with mirostat 2, that always seems to be problematic (looping). Not sure if I'm using it right. I haven't really been able to find any documentation about that. I haven't gotten any outright gibberish, though. I'm gonna try 2.2 now and see if it's more reliable.