Update README.md
Browse files
README.md
CHANGED
@@ -11,6 +11,10 @@ tags:
|
|
11 |
|
12 |
[TokenCompose_SD21_A](https://mlpc-ucsd.github.io/TokenCompose/) is a [latent text-to-image diffusion model](https://arxiv.org/abs/2112.10752) finetuned from the [**Stable-Diffusion-v2-1**](https://huggingface.co/stabilityai/stable-diffusion-2-1) checkpoint at resolution 768x768 on the [VSR](https://github.com/cambridgeltl/visual-spatial-reasoning) split of [COCO image-caption pairs](https://cocodataset.org/#download) for 32,000 steps with a learning rate of 5e-6. The training objective involves token-level grounding terms in addition to denoising loss for enhanced multi-category instance composition and photorealism. The "_A/B" postfix indicates different finetuning runs of the model using the same above configurations.
|
13 |
|
|
|
|
|
|
|
|
|
14 |
# 🧨Example Usage
|
15 |
|
16 |
We strongly recommend using the [🤗Diffuser](https://github.com/huggingface/diffusers) library to run our model.
|
@@ -39,4 +43,13 @@ image.save("cat_and_wine_glass.png")
|
|
39 |
| TokenCompose (SD21) | 60.10 | 80.48 | 36.69 | 5.71 | 79.51 | 39.59 | 8.13 | 19.15 |
|
40 |
|
41 |
# 📰 Citation
|
42 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
|
12 |
[TokenCompose_SD21_A](https://mlpc-ucsd.github.io/TokenCompose/) is a [latent text-to-image diffusion model](https://arxiv.org/abs/2112.10752) finetuned from the [**Stable-Diffusion-v2-1**](https://huggingface.co/stabilityai/stable-diffusion-2-1) checkpoint at resolution 768x768 on the [VSR](https://github.com/cambridgeltl/visual-spatial-reasoning) split of [COCO image-caption pairs](https://cocodataset.org/#download) for 32,000 steps with a learning rate of 5e-6. The training objective involves token-level grounding terms in addition to denoising loss for enhanced multi-category instance composition and photorealism. The "_A/B" postfix indicates different finetuning runs of the model using the same above configurations.
|
13 |
|
14 |
+
# 📄 Paper
|
15 |
+
|
16 |
+
Please follow [this](https://arxiv.org/abs/2312.03626) link.
|
17 |
+
|
18 |
# 🧨Example Usage
|
19 |
|
20 |
We strongly recommend using the [🤗Diffuser](https://github.com/huggingface/diffusers) library to run our model.
|
|
|
43 |
| TokenCompose (SD21) | 60.10 | 80.48 | 36.69 | 5.71 | 79.51 | 39.59 | 8.13 | 19.15 |
|
44 |
|
45 |
# 📰 Citation
|
46 |
+
```bibtex
|
47 |
+
@misc{wang2023tokencompose,
|
48 |
+
title={TokenCompose: Grounding Diffusion with Token-level Supervision},
|
49 |
+
author={Zirui Wang and Zhizhou Sha and Zheng Ding and Yilin Wang and Zhuowen Tu},
|
50 |
+
year={2023},
|
51 |
+
eprint={2312.03626},
|
52 |
+
archivePrefix={arXiv},
|
53 |
+
primaryClass={cs.CV}
|
54 |
+
}
|
55 |
+
```
|