dolphincoder tune of starcoder2-15b-instruct?
@ehartford , I've been using dolphincoder, and have been really happy with it (q8_0 gguf via Ollama). Thanks!
I was excited to hear about starcoder2 instruct models, and decided to give them a spin. It wasn't good. It would do what I asked (In rust, my standard eval questions), but as soon as I asked it to refine it (multi-turn), it would default back to python, and make up some function I wasn't even asking about.
Is it worth the effort to fine-tune SC2-I with dolphincoder?
If so, is that something you could add to the pipeline?
I'm happy to give it a spin myself, but it'll take a while, I only have an Nvidia T1000 (4GB VRAM, 7.5 compute) and dual sandylakes with 256GB DDR3. That, and I've never done a fine-tune before...
I wonder if it thought python would be more efficient which would have technically given the correct answer. I would try to re-ask but setup the system prompt to ensure all responses are in the language you want.
I don't tune instruct models. I only tune base models.
But, I could add the dataset for starcoder2-instruct into the mix of dolphin. In fact, I think I will.
I wonder if it thought python would be more efficient which would have technically given the correct answer. I would try to re-ask but setup the system prompt to ensure all responses are in the language you want.
Ha! I get what you're saying, but I'm evaluating models for a specific use case, which will be different languages and multiple users. If it can't follow a thread of conversation like the dolphin tunes can, then that means it's not suitable for my use case. Sadly.
But, I could add the dataset for starcoder2-instruct into the mix of dolphin. In fact, I think I will.
Sweet! That's a damn good idea! Can't wait for the release