YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
DPO dataset construction
# Make sure you have mimic files, which should be like MedOmni_DPO/data/xray/dpo/images/physionet.org
cd MedOmni_DPO/data/xray/dpo/utils
python process_mask.py # Mask DPO from https://arxiv.org/abs/2406.06496
cd MedOmni_DPO/utils
python build_dataset.py
You will have two JSON files train.json and val.json in MedOmni_DPO/json_files/medinterp/. If you prefer not to create the dataset yourself, we've already provided these two JSON files.
Finetuning MedVersa checkpoint
sh run.sh
Evaluation
sh eval.sh # Need to specify --epoch-id and --model-id (medomni/output/medomni_v0)