togethercomputer

/

RedPajama-INCITE-7B-Base

like 93

Hi, I'm trying to finetune this with QLoRA on the open assistant dataset, following this tutorial

https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing

Is there anyway to know what layers I need to pass to the LoRA config?

Thanks