metadata

tags:
  - image-classification
  - timm
library_name: timm
license: other
license_name: lunit-non-commercial
license_link: https://github.com/lunit-io/benchmark-ssl-pathology/blob/main/LICENSE
datasets:
  - 1aurent/BACH
  - 1aurent/NCT-CRC-HE
  - 1aurent/PatchCamelyon
pipeline_tag: image-classification

Model card for vit_small_patch8_224.lunit_dino

A Vision Transformer (ViT) image classification model.
Trained on 33M histology patches from various pathology datasets.

Model Details

Model Type: Feature backbone
SSL Method: DINO
Model Stats:
- Params (M): 21.7
- Image sizes: 224 × 224 x 3
Papers:
- Benchmarking Self-Supervised Learning on Diverse Pathology Datasets: https://arxiv.org/abs/2212.04690
Datasets:
- BACH
- CRC
- MHIST
- PatchCamelyon
- CoNSeP
Original: https://github.com/lunit-io/benchmark-ssl-pathology
License: lunit-non-commercial

Model Usage

Image Embeddings

from urllib.request import urlopen
from PIL import Image
import timm

# get example histology image
img = Image.open(
  urlopen(
    "https://github.com/owkin/HistoSSLscaling/raw/main/assets/example.tif"
  )
)

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_small_patch8_224.lunit_dino",
  pretrained=True,
).eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

Citation

@inproceedings{kang2022benchmarking,
  author    = {Kang, Mingu and Song, Heon and Park, Seonwook and Yoo, Donggeun and Pereira, Sérgio},
  title     = {Benchmarking Self-Supervised Learning on Diverse Pathology Datasets},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2023},
  pages     = {3344-3354}
}