Mini-Magnum Feedback

#7
by JaneFB - opened

Had a long feedback post written, but lost it, so here are the highlights. :P

I think Mini-Magnum could possibly be the top custom Nemo model right now, with its greatest asset being its natural ability to write long, intelligent, coherent, insightful responses drawn from the character/scenario right off the bat, even when given almost nothing to work with from the user's end. Other Nemo models at the same optimal settings, in contrast, will 95% of the time only give a couple of words, or a single line at most, until the combined responses in context become long enough to build off.

The standard Instruct is right on Magnum's heels in some respects, while beating it in others, but the other models (e.g. Celeste, Nemomix) all tend to fall apart in various notable ways, being less verbose, creative and coherent, with assorted formatting issues, odd output quirks, and knowledge gaps that neither Instruct nor Magnum suffer from.

Magnum's own primary issue is probably that it is currently just TOO aggressive in terms of unilaterally trying to steer interactions into NSFW territory, while ironically simultaneously being quite limited, predictable, and generally a bit one-note in its NSFW expression, presenting every encounter in a familiar, repetitive way the more organic basic Instruct model doesn't.

On top of that, while LLMs in general will pepper "searing kisses", people being "claimed" or giggling/grinning "impishly", and variations of the infamous "So what do you say, X? Are you ready to Y?" GPTism all over the place, in both SFW and NSFW content, Magnum absolutely LOVES using these particular phrases, to the point where they'll show up constantly regardless of context, character, setting, or story. Even if you edit them out, swipe them away, or reroute the conversation entirely, they'll worm their way right back in within a couple of responses at the latest. It's almost meme-worthy.

Magnum will also quite frequently have characters suddenly refer to the user as "Anon" out of nowhere, which is a bit of a giveaway regarding the source of its training material.

they'll show up constantly regardless of context

Try the DRY trick. It was added to SillyTavern since recently but isn't yet supported by all backends. I use llamacpp_HF and it works great with it (not with the vanilla llama.cpp, though it's in the process of adding this setting). For Mistral I customized the sequence breakers with ["[INST]", "[/INST]", "</s>", "\\n"]. As a result, you should never see these long enough phrases more than once since they're heavily punished by DRY. The model will choose other tokens that might be close but not the same, it won't loop and the prose should become much more varied as it won't settle at some comfortable local minimum.

Sign up or log in to comment