IrohXu/stable-diffusion-mimic-cxr-v0.1

Stable Diffusion MIMIC-CXR Model Card

Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input. We finetune Stable-Diffusion-v1-4 checkpoint on MIMIC-CXR dataset.

This weights here are intended to be used with the 🧨 Diffusers library. Note: this weight can be only used with x-ray image editing, the performance is not good for generate a new x-ray image.

Our code related to this weight: https://github.com/IrohXu/PIE. Please star it if you like this work.

Cite as:

@article{liang2023pie,
  title={PIE: Simulating Disease Progression via Progressive Image Editing},
  author={Liang, Kaizhao and Cao, Xu and Liao, Kuei-Da and Gao, Tianren and Ye, Wenqian and Chen, Zhengyu and Cao, Jianguo and Nama, Tejas and Sun, Jimeng},
  year={2023}
}

PyTorch

pip install --upgrade diffusers transformers scipy     

Running the pipeline with the default PNDM scheduler:

import torch
from diffusers import StableDiffusionPipeline

model_id = "IrohXu/stable-diffusion-mimic-cxr-v0.1"
device = "cuda"


pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16, safety_checker=None)
pipe = pipe.to(device)

prompt = "Severe edema in the left and right lower lobes, severity. Severe right and left pleural effusion is larger."
image = pipe(prompt).images[0]  
    
image.save("result.png")