mistralai

/

Mamba-Codestral-7B-v0.1

like 562

Follow

Mistral AI_ 3.73k

I have trying to finetune codestral for my usecase. i tried to use peft and lora but it throws an error stating you need to specify "target_modules". It turns out that normal target_modules like [proj, q_proj, k_proj] are not present in the model. i tried some of the modules inside the model, but it throws an error of not being supported.

can someone help me with this and help me find the proper target_modules.

thanks

shreyasjain231219