martinkropf commited on
Commit
1ef0476
1 Parent(s): 1212b6a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -28
README.md CHANGED
@@ -1,28 +1,12 @@
1
- # EchoCLIP: A Multimodal Foundation Model For Echocardiography
2
-
3
- EchoCLIP is a multimodal foundation model for echocardiography. It is finetuned from CLIP weights on a dataset of >1M pairs of echocardiogram images and their associated expert interpretation text. It can be used for semantic search amongst echo videos as well as zero-shot prediction on a wide range of clinically relevant tasks. For more details, see our paper:
4
-
5
- (link will be added once preprint is released)
6
- <!-- [Multimodal Foundation Models For Echocardiogram Interpretation](https://arxiv.org/abs/) -->
7
-
8
- ## Quickstart
9
-
10
- This repo contains example code for loading and using EchoCLIP and its long-context variant, EchoCLIP-R. To get started, clone this repo and navigate into it. Then, create a new `conda` environment and install the required packages:
11
-
12
- ```
13
- git clone https://github.com/echonet/echo_CLIP
14
- cd echo_CLIP
15
- conda env create -n echo-clip
16
- conda activate echo-clip
17
- python -m pip install -r requirements.txt
18
- ```
19
- You should now be able to run `embedding_example.py` and `zero_shot_example.py`.
20
-
21
- ## Repo contents
22
-
23
- * `embedding_example.py` demonstrates how to load EchoCLIP-R's weights and use them to calculate the similarity between an example echocardiogram and example report text.
24
- * `zero_shot_example.py` demonstrates how to load EchoCLIP's weights and use them to perform zero-shot pacemaker identification and zero-shot ejection fraction prediction.
25
- * `utils.py` contains implementations of our methods for performing zero-shot binary classification and zero-shot regression. The functions used in `zero_shot_example.py` are defined in this file. The prompts we use for the zero-shot tasks in our paper are all available here. Additionally, this file contains regexes for cleaning and preparing report text before it is tokenized.
26
- * `template_tokenizer.py` contains the implementation of our custom echocardiography report tokenizer, which is designed to compress Cedars-Sinai echo reports into a small number of tokens.
27
- * `template_vocab.txt` contains a vocabulary of 770 words and phrases constructed from the template file our cardiologists use to create their reports. This vocabulary is used by our template tokenizer to efficiently tokenize long reports.
28
- * `blank_wordpiece.tokenizer` is a default config file for initializing a WordPiece tokenizer using HuggingFace's `tokenizers` library. We use it to initialize our custom tokenizer.
 
1
+ ---
2
+ title: Mkaichristensen Echo Clip
3
+ emoji: 🐨
4
+ colorFrom: yellow
5
+ colorTo: yellow
6
+ sdk: gradio
7
+ sdk_version: 4.4.0
8
+ app_file: app.py
9
+ pinned: false
10
+ ---
11
+
12
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference