anthracite-org/magnum-v2-123b · Better performance with ChatML for novel writing.

Using exl2 2.7bpw version with ChatML instruct template (to be specific, all settings are from magnum-v3-34b repository with a sprinkle of DRY and XTC).

Trying to do this with the mistral template, both the original and the Nemo one, resulted in annoyingly short chapters and very "dry" stories. My guess is that it's because this model's tuned for roleplay?

All in one message, by the way. Had to press "continue" a bunch of times in silly just to push this through, no idea whether it's an issue with the EOS tokens or Runpod.

https://pastebin.com/xjGdBZr1

EDIT: I'm a dum-dum that apparently can't read. You guys provided context and instruct settings that work WAY better than whatever is in Silly by default. So... yeah, ChatML works as well I guess.