|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
pipeline_tag: token-classification |
|
library_name: fastai |
|
--- |
|
<!-- Provide a quick summary of what the model is/does. --> |
|
This model is used for high and low classification of survival period and disease-free survival period for pathological multimodal input. |
|
|
|
## How to Get Started with the Model |
|
Use the code ```inference.py``` to get started with the model. |
|
|
|
## Evaluation |
|
In `inference.py`, modify the model path and dataset path to do the model evaluation. |
|
The model outputs the classification accuracy on two categories. |
|
|
|
#### Data |
|
The inputs of the model are the pathological image features and pathological text features extracted based on PLIP [[A visual–language foundation model for pathology image analysis using medical Twitter]](https://huggingface.co/vinid/plip). |
|
You can organize your own dataset for evaluation according to this method. |
|
|
|
### Results |
|
| pathological image | heat map of image|text and importance distribution | |
|
| ---- | ---- | ---- | |
|
|<img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/8-A1.png" width="224" height="224" alt=""> | <img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/8-A1-heatmap.png" width="224" height="224" alt=""> | <img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/8-A1-txt-visual.png" width="224" height="224" alt="">| |
|
|<img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/20225633812.png" width="224" height="224" alt=""> | <img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/20225633812-heatmap.png" width="224" height="224" alt=""> | <img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/20225633812-txt-visual.png" width="224" height="224" alt="">| |
|
|<img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/20225603708.png" width="224" height="224" alt=""> | <img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/20225603708-heatmap.png" width="224" height="224" alt=""> | <img src="https://huggingface.co/ankoninc/pathologyprognosis/resolve/main/images/20225603708-txt-visual.png" width="224" height="224" alt="">| |
|
|
|
This [links](http://aimark.local.ankoninc.com.cn:18443/image2WHU/) shows more visualized results corresponding to pathological images and text inputs. |
|
The visualized results include the heat map distribution of pathological images, the heat map distribution of pathological image patches, and the display of text importance distribution. |
