VIVEK JAYARAM
commited on
Commit
•
a9141e6
1
Parent(s):
4cf6ff4
Yaml for hf spaces
Browse files
README.md
CHANGED
@@ -1,4 +1,5 @@
|
|
1 |
-
|
|
|
2 |
emoji: 🧪
|
3 |
colorFrom: purple
|
4 |
colorTo: blue
|
@@ -7,81 +8,4 @@ sdk_version: 5.1.0
|
|
7 |
app_file: app.py
|
8 |
pinned: true
|
9 |
license: mit
|
10 |
-
|
11 |
-
- google/ddpm-celebahq-256
|
12 |
-
|
13 |
-
# Constrained Diffusion Implicit Models
|
14 |
-
![alt text](Teaser.jpg)
|
15 |
-
|
16 |
-
## Authors
|
17 |
-
[Vivek Jayaram](http://www.vivekjayaram.com/), [John Thickstun](https://johnthickstun.com/), [Ira Kemelmacher-Shlizerman](https://homes.cs.washington.edu/~kemelmi/), and [Steve Seitz](https://homes.cs.washington.edu/~seitz/)
|
18 |
-
|
19 |
-
## [Project Page](www.google.com)
|
20 |
-
(Coming Soon)
|
21 |
-
|
22 |
-
### [Paper](www.google.com)
|
23 |
-
(Coming Soon)
|
24 |
-
|
25 |
-
### Summary
|
26 |
-
We solve noisy linear inverse problems with diffusion models. The method is fast and addresses many problems like inpainting, super-resolution, gaussian deblur, and poisson noise.
|
27 |
-
|
28 |
-
|
29 |
-
## Getting started
|
30 |
-
|
31 |
-
Recommended environment: Python 3.11, Cuda 12, Conda. For lower verions please adjust the dependencies below.
|
32 |
-
|
33 |
-
### 1) Clone the repository
|
34 |
-
|
35 |
-
```
|
36 |
-
git clone https://github.com/vivjay30/cdim
|
37 |
-
|
38 |
-
cd cdim
|
39 |
-
```
|
40 |
-
|
41 |
-
### 2) Install dependencies
|
42 |
-
|
43 |
-
```
|
44 |
-
conda create -n cdim python=3.11
|
45 |
-
|
46 |
-
conda activate cdim
|
47 |
-
|
48 |
-
pip install -r requirements.txt
|
49 |
-
|
50 |
-
pip install torch==2.4.1+cu124 torchvision-0.19.1+cu124 --extra-index-url https://download.pytorch.org/whl/cu124
|
51 |
-
```
|
52 |
-
|
53 |
-
## Inference Examples
|
54 |
-
We recommend using the ddpm models from the diffusers library. These are better and can be run without any manual downloading. Model will be downloaded on first run. The run will produce a noisy_measurement.png and output.png. The ouptut directory can be passed as an argument. You can use kl optimization instead of l2 with the `--loss kl` flag.
|
55 |
-
|
56 |
-
#### CelebHQ Inpainting Example
|
57 |
-
|
58 |
-
`python inference.py sample_images/celebhq_00001.jpg 50 3 operator_configs/box_inpainting_config.yaml noise_configs/gaussian_noise_config.yaml google/ddpm-celebahq-256`
|
59 |
-
|
60 |
-
#### LSUN Churches Gaussian Deblur Example
|
61 |
-
`python inference.py sample_images/lsun_church.png 50 3 operator_configs/gaussian_blur_config.yaml noise_configs/gaussian_noise_config.yaml google/ddpm-church-256`
|
62 |
-
|
63 |
-
#### Poisson Noise Example
|
64 |
-
`python inference.py sample_images/celebhq_29999.jpg 50 3 operator_configs/identity_operator_config.yaml noise_configs/poisson_noise_config.yaml google/ddpm-celebahq-256 --loss kl --eta-type gradnorm`
|
65 |
-
|
66 |
-
#### Discrete KL with Bimodal Noise Example
|
67 |
-
An example to show discrete KL on a non-standard noise distirbution
|
68 |
-
`python inference.py sample_images/celebhq_00001.jpg 200 1 operator_configs/box_inpainting_config.yaml noise_configs/bimodal_noise_config.yaml google/ddpm-celebahq-256 --loss categorical_kl --lambda-val 2 --eta-type gradnorm`
|
69 |
-
|
70 |
-
## FFHQ and Imagenet Models
|
71 |
-
These models are generally not as strong as the huggingface ddpm models, but are used for comparisons with baseline methods.
|
72 |
-
|
73 |
-
From [this link](https://drive.google.com/drive/folders/1jElnRoFv7b31fG0v6pTSQkelbSX3xGZh?usp=sharing), download the checkpoints "ffhq_10m.pt" and "imagenet_256.pt" to models/
|
74 |
-
|
75 |
-
#### Imagenet Super Resolution Example
|
76 |
-
`python inference.py sample_images/imagenet_val_00002.png 50 3 operator_configs/super_resolution_config.yaml noise_configs/gaussian_noise_config.yaml models/imagenet_model_config.yaml`
|
77 |
-
|
78 |
-
#### FFHQ Random Inpainting (Faster)
|
79 |
-
Here we set T=25 and K=1 to show the algorithm running faster
|
80 |
-
`python inference.py sample_images/ffhq_00010.png 25 1 operator_configs/random_inpainting_config.yaml noise_configs/gaussian_noise_config.yaml models/ffhq_model_config.yaml`
|
81 |
-
|
82 |
-
## A note on Eta and Lambda schedules
|
83 |
-
By default the model tries to use expected gradnorm to set the step size schedule (eta). The gradient magnitudes have been precomputed on the train set and are stored in `etas.json`. However, those values are only valid for the specific tasks and number of steps T and K. When using a different task or step configuration, we fall back to `--eta-type gradnorm` which performs individual step gradient normalization as the value of eta. You can always use that flag, which is a less efficient but more general method.
|
84 |
-
|
85 |
-
In addition, the step size schedule eta is scaled by a constant value lambda (the proportionality constant). Getting eta and lambda correct is vital to good convergence, especially with fewer denoising and optimization steps. If you find that the model overfits (loss oscillates wildly) or undefits (loss doesn't go to 0 for KL or sigma^2 for L2), then you should tweak the argument `--lambda-val`. The best guess is printed out for you to use as a starting point.
|
86 |
-
|
87 |
-
|
|
|
1 |
+
---
|
2 |
+
title: CDIM
|
3 |
emoji: 🧪
|
4 |
colorFrom: purple
|
5 |
colorTo: blue
|
|
|
8 |
app_file: app.py
|
9 |
pinned: true
|
10 |
license: mit
|
11 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|