kolja-b commited on
Commit
d2dcd50
1 Parent(s): 2a91b40

Update README

Browse files
Files changed (1) hide show
  1. README.md +36 -3
README.md CHANGED
@@ -1,3 +1,36 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+
5
+ # CleanDIFT Model Card
6
+
7
+
8
+ Diffusion models learn powerful world representations that have proven valuable for tasks like semantic correspondence detection,
9
+ depth estimation, semantic segmentation, and classification.
10
+ However, diffusion models require noisy input images, which destroys information and introduces the noise level as a hyperparameter that needs to be tuned for each task.
11
+
12
+
13
+
14
+
15
+ We introduce CleanDIFT, a novel method to extract noise-free, timestep-independent features by enabling diffusion models to work directly with clean input images.
16
+ The approach is efficient, training on a single GPU in just 30 minutes. We publish these models alongside our paper ["CleanDIFT: Diffusion Features without Noise"](https://compvis.github.io/CleanDIFT/).
17
+
18
+ We provide checkpoints for Stable Diffusion 1.5 and Stable Diffusion 2.1.
19
+
20
+
21
+ ## Usage
22
+
23
+ For detailed examples on how to extract features with CleanDIFT and how to use them for downstream tasks, please refer to the notebooks provided [here](https://github.com/CompVis/CleanDIFT/tree/main/notebooks).
24
+
25
+ Our checkpoints are fully compatible with the `diffusers` library.
26
+ If you already have a pipeline using SD 1.5 or SD 2.1 from `diffusers`, you can simply replace the U-Net state dict:
27
+ ```python
28
+ from diffusers import UNet2DConditionModel
29
+ from huggingface_hub import hf_hub_download
30
+
31
+ unet = UNet2DConditionModel.from_pretrained("stabilityai/stable-diffusion-2-1", subfolder="unet")
32
+ ckpt_pth = hf_hub_download(repo_id="CompVis/cleandift", filename="cleandift_sd21_unet.safetensors")
33
+ state_dict = load_file(ckpt_pth)
34
+ unet.load_state_dict(state_dict, strict=True)
35
+ ```
36
+