Really good!
I first tested it with roleplay and stories, and quickly noticed that the model followed the guidelines well and took into account the information contained in the character cards very well. I also noticed how nicely and logically it connects the individual threads... it got me thinking, so I decided to test how it deals with puzzles and logic in general (yes, I know this model is not for that)
The effect was incredible! At 0 temp it consistently solved 8 out of 10 test tasks - a very good result! But it turned out to be even more interesting when I increased the temperature to 0.9 (your recommended temperature along with the rest of your settings). Then the model solved 9 out of 10 time after time - making only one mistake in seemingly random puzzles (it even solved those it couldn't before) and this persisted after many attempts - Wow!
In summary, the model is really great considering its size, and it actually does a great job with your recommended settings. The only thing I would complain about is the small context length (it's crying out for llama 3.1).
Thank you very much for this model, it is a worthy successor to Nymeria.
I first tested it with roleplay and stories, and quickly noticed that the model followed the guidelines well and took into account the information contained in the character cards very well. I also noticed how nicely and logically it connects the individual threads... it got me thinking, so I decided to test how it deals with puzzles and logic in general (yes, I know this model is not for that)
The effect was incredible! At 0 temp it consistently solved 8 out of 10 test tasks - a very good result! But it turned out to be even more interesting when I increased the temperature to 0.9 (your recommended temperature along with the rest of your settings). Then the model solved 9 out of 10 time after time - making only one mistake in seemingly random puzzles (it even solved those it couldn't before) and this persisted after many attempts - Wow!
In summary, the model is really great considering its size, and it actually does a great job with your recommended settings. The only thing I would complain about is the small context length (it's crying out for llama 3.1).
Thank you very much for this model, it is a worthy successor to Nymeria.
Yup, the extra horsepower compared to 8B is really noticeable. Sao10K's Stheno and Niitama are also really good at puzzle solving and strong all-around models. In fact, the only 3.0 models that I have noticed are able to solve certain puzzles. SimPO, on the other hand, raises the model's chat/multi-turn ability to a new level. I'm going to release 2x8b's other quants and transformers today.