LLaVA-Med model for multimodal radiology report generation

This is a model based on LLaVA-Med 1.0, finetuned to generate medical reports, based on a chest X-ray and a prompt, in our case, the instruction was "write the finding section of chest x-ray radiology report".

The dataset used for finetuning was the MIMIC-CXR share for the challenge in Radiology Report Generation for the Association for Computational Linguistics 2024 at BioNLP Workshop

We used the 148,374 findings of MIMIC-CXR for finetuing during 3 epochs.

More details of the challenge can be found on the challenge web page or in workshop site