Is there any code for fine-tuning the LLAVA model?
I am currently encountering several challenges while fine-tuning the LLAVA model. Is there any code for fine-tuning the LLAVA model? Your assistance would be greatly appreciated！
We have a demo fine-tuning notebook here (https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LLaVa/Fine_tune_LLaVa_on_a_custom_dataset_(with_PyTorch_Lightning).ipynb)
We have a demo fine-tuning notebook here (https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LLaVa/Fine_tune_LLaVa_on_a_custom_dataset_(with_PyTorch_Lightning).ipynb)
Thank you so much for your hard work! This notebook is incredibly detailed and thorough. I'm genuinely blown away by the level of work and explanation provided.
We have a demo fine-tuning notebook here (https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LLaVa/Fine_tune_LLaVa_on_a_custom_dataset_(with_PyTorch_Lightning).ipynb)
Hi, thank you for your excellent work! However, I am a beginner and have a question. Is the code designed to process grayscale images only? When I tried applying it to my dataset with color images, I encountered a dimensionality error. I've been trying to fix it for a while, but I haven't made any progress. Could you please advise on which parts of the code need to be modified to work with color images? I would really appreciate any assistance.
@miaoyl are your images 4-channel? AFAIK LLaVA-NeXT can't handle 4-channel images, but you can make rgb from 4-channel yourself before passing it to the processor