Best model
Hi, I would be interested to know according to your internal testing, which model performs better?
I understand that there may not be a certain and definitive answer because it may vary by use case, but I would still like to know the best model in general
Thats actually a great question @danielus and I wish the answer was easier.
For code -> fllow diagam honestly thats the original ability and I think the larger the parameter the better the model there, but for story to flow diagram many people are still using MermaidMistral and the Solar Variants, these newer Mixtral models are trained exclusively on Synthetic Data created by the outputs of the original Mermaid Mistral and Mermaid Solar as the new dataset.
The goal is to further my research on improving Models through temperature variation for creating more diverse datasets. Thats what the toolkit I released on my github was for.
It really comes down to how do you use it, as many people are fine tuning on top of my model to get models that can do role play and understand the flow of the story better, and can better understand the separate journeys and doesnt mix up characters and their interactions.
I would say give the largest parameter model you can run a try first, but in my testing I really liked my simple MermaidSolar as its the largest model that was trained on the original dataset that is now used only for evaluation of the models abilities.
Eval loss wise, MermaidMixtral 3x7b and 2x7b are pretty close to the same when evaluating on my original hand curated dataset that the model has never seen.
Hi, thank you very much for such a complete answer :D
Currently I wanted to try to use it to fix my mistakes 😂
Basically I have been working on several projects but never documented them properly, so I would like to try to use this model to schematize my codebases to get an overview. I thank you so much for your contribution and to make this model opensource! Also I would be interested to talk with you privately about more about these models.
If you want you can reach me by Telegram or if you prefer by Email
@danielus feel free to message me on https://www.linkedin.com/in/troyandrewschultz/
I also use Signal if that works better for you.
Lets talk soon, I am always down to nudge my models abilities around for science! :D