SegVol: Universal and Interactive Volumetric Medical Image Segmentation

This repo is the official implementation of SegVol: Universal and Interactive Volumetric Medical Image Segmentation.

News🚀

(2023.11.24) You can download weight files of SegVol and ViT(CTs pre-train) here. 🔥

(2023.11.23) The brief introduction and instruction have been uploaded.

(2023.11.23) The inference demo code has been uploaded.

(2023.11.22) The first edition of our paper has been uploaded to arXiv. 📃

Introduction

The SegVol is a universal and interactive model for volumetric medical image segmentation. SegVol accepts point, box and text prompt while output volumetric segmentation. By training on 90k unlabeled Computed Tomography (CT) volumes and 6k labeled CTs, this foundation model supports the segmentation of over 200 anatomical categories.

We will release SegVol's inference code, training code, model params and ViT pre-training params (pre-training is performed over 2,000 epochs on 96k CTs).

Usage

Requirements

The pytorch v1.11.0 (or higher virsion) is needed first. Following install key requirements using commands:

pip install 'monai[all]==0.9.0'
pip install einops==0.6.1
pip install transformers==4.18.0
pip install matplotlib

Config and run demo script

You can download the demo case here, or download the whole demo dataset AbdomenCT-1K and choose any demo case you want.
Please set CT path and Ground Truth path of the case in the config_demo.json.
After that, config the inference_demo.sh file for execution:
- $segvol_ckpt: the path of SegVol's checkpoint (Download from here).
- $work_dir: any path of folder you want to save the log files and visualizaion results.
Finally, you can control the prompt type, zoom-in-zoom-out mechanism and visualizaion switch here.
Now, just run bash script/inference_demo.sh to infer your demo case.

Citation

If you find this repository helpful, please consider citing:

@misc{du2023segvol,
      title={SegVol: Universal and Interactive Volumetric Medical Image Segmentation}, 
      author={Yuxin Du and Fan Bai and Tiejun Huang and Bo Zhao},
      year={2023},
      eprint={2311.13385},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

Thanks for the following amazing works:

HuggingFace.

CLIP.

MONAI.

Image by brgfx on Freepik.

Image by muammark on Freepik.

Image by pch.vector on Freepik.

Image by starline on Freepik.