cloudyu

/

Pluto_13B_DPO

like 4

Hi! Thanks for the great model!

I am also interested in performing DPO tuning on MoE models like your Mixtral_7Bx2_MoE.

I'm successfully run DPO against non-MoE Mistral models using the Huggingface Trl library. But failing at MoE. If you'd be so kind, could you share your technique? Thanks!