Lawrence-cj
commited on
Commit
•
b6c4146
1
Parent(s):
d969b7c
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
library_name: sana
|
3 |
+
tags:
|
4 |
+
- text-to-image
|
5 |
+
- Sana
|
6 |
+
- 1024px_based_image_size
|
7 |
+
- BF16
|
8 |
+
language:
|
9 |
+
- en
|
10 |
+
- zh
|
11 |
+
base_model:
|
12 |
+
- Efficient-Large-Model/Sana_1600M_1024px
|
13 |
+
pipeline_tag: text-to-image
|
14 |
+
---
|
15 |
+
<p align="center" style="border-radius: 10px">
|
16 |
+
<img src="https://raw.githubusercontent.com/NVlabs/Sana/refs/heads/main/asset/logo.png" width="35%" alt="logo"/>
|
17 |
+
</p>
|
18 |
+
|
19 |
+
<div style="display:flex;justify-content: center">
|
20 |
+
<a href="https://huggingface.co/collections/Efficient-Large-Model/sana-673efba2a57ed99843f11f9e"><img src="https://img.shields.io/static/v1?label=Demo&message=Huggingface&color=yellow"></a>  
|
21 |
+
<a href="https://github.com/NVlabs/Sana"><img src="https://img.shields.io/static/v1?label=Code&message=Github&color=blue&logo=github"></a>  
|
22 |
+
<a href="https://nvlabs.github.io/Sana/"><img src="https://img.shields.io/static/v1?label=Project&message=Github&color=blue&logo=github-pages"></a>  
|
23 |
+
<a href="https://hanlab.mit.edu/projects/sana/"><img src="https://img.shields.io/static/v1?label=Page&message=MIT&color=darkred&logo=github-pages"></a>  
|
24 |
+
<a href="https://arxiv.org/abs/2410.10629"><img src="https://img.shields.io/static/v1?label=Arxiv&message=Sana&color=red&logo=arxiv"></a>  
|
25 |
+
<a href="https://nv-sana.mit.edu/"><img src="https://img.shields.io/static/v1?label=Demo&message=MIT&color=yellow"></a>  
|
26 |
+
<a href="https://discord.gg/rde6eaE5Ta"><img src="https://img.shields.io/static/v1?label=Discuss&message=Discord&color=purple&logo=discord"></a>  
|
27 |
+
</div>
|
28 |
+
|
29 |
+
# 🐱 Sana Model Card
|
30 |
+
<p align="center" border-raduis="10px">
|
31 |
+
<img src="https://raw.githubusercontent.com/NVlabs/Sana/refs/heads/main/asset/Sana.jpg" width="80%" alt="teaser_page1"/>
|
32 |
+
</p>
|
33 |
+
|
34 |
+
## Model
|
35 |
+
<p align="center" border-raduis="10px">
|
36 |
+
<img src="https://raw.githubusercontent.com/NVlabs/Sana/refs/heads/main/asset/model-incremental.jpg" width="80%" alt="teaser_page1"/>
|
37 |
+
</p>
|
38 |
+
|
39 |
+
We introduce **Sana**, a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution.
|
40 |
+
Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.
|
41 |
+
|
42 |
+
Source code is available at https://github.com/NVlabs/Sana.
|
43 |
+
|
44 |
+
### Model Description
|
45 |
+
|
46 |
+
- **Developed by:** NVIDIA, Sana
|
47 |
+
- **Model type:** Linear-Diffusion-Transformer-based text-to-image generative model
|
48 |
+
- **Model size:** 1648M parameters
|
49 |
+
- **Model precision:** torch.bfloat16 (BF16)
|
50 |
+
- **Model resolution:** This model is developed to generate 1024px based images with multi-scale heigh and width.
|
51 |
+
- **License:** [CC BY-NC-SA 4.0 License](./LICENSE.txt)
|
52 |
+
- **Model Description:** This is a model that can be used to generate and modify images based on text prompts.
|
53 |
+
It is a Linear Diffusion Transformer that uses one fixed, pretrained text encoders ([Gemma2-2B-IT](https://huggingface.co/google/gemma-2-2b-it))
|
54 |
+
and one 32x spatial-compressed latent feature encoder ([DC-AE](https://hanlab.mit.edu/projects/dc-ae)).
|
55 |
+
- **Resources for more information:** Check out our [GitHub Repository](https://github.com/NVlabs/Sana) and the [Sana report on arXiv](https://arxiv.org/abs/2410.10629).
|
56 |
+
|
57 |
+
### Model Sources
|
58 |
+
|
59 |
+
For research purposes, we recommend our `generative-models` Github repository (https://github.com/NVlabs/Sana),
|
60 |
+
which is more suitable for both training and inference and for which most advanced diffusion sampler like Flow-DPM-Solver is integrated.
|
61 |
+
[MIT Han-Lab](https://nv-sana.mit.edu/) provides free Sana inference.
|
62 |
+
- **Repository:** ttps://github.com/NVlabs/Sana
|
63 |
+
- **Demo:** https://nv-sana.mit.edu/
|
64 |
+
|
65 |
+
### 🧨 Diffusers
|
66 |
+
|
67 |
+
PR developing: [Sana](https://github.com/huggingface/diffusers/pull/9982) and [DC-AE](https://github.com/huggingface/diffusers/pull/9708)
|
68 |
+
|
69 |
+
|
70 |
+
## Uses
|
71 |
+
|
72 |
+
### Direct Use
|
73 |
+
|
74 |
+
The model is intended for research purposes only. Possible research areas and tasks include
|
75 |
+
|
76 |
+
- Generation of artworks and use in design and other artistic processes.
|
77 |
+
- Applications in educational or creative tools.
|
78 |
+
- Research on generative models.
|
79 |
+
- Safe deployment of models which have the potential to generate harmful content.
|
80 |
+
|
81 |
+
- Probing and understanding the limitations and biases of generative models.
|
82 |
+
|
83 |
+
Excluded uses are described below.
|
84 |
+
|
85 |
+
### Out-of-Scope Use
|
86 |
+
|
87 |
+
The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
|
88 |
+
|
89 |
+
## Limitations and Bias
|
90 |
+
|
91 |
+
### Limitations
|
92 |
+
|
93 |
+
|
94 |
+
- The model does not achieve perfect photorealism
|
95 |
+
- The model cannot render complex legible text
|
96 |
+
- fingers, .etc in general may not be generated properly.
|
97 |
+
- The autoencoding part of the model is lossy.
|
98 |
+
|
99 |
+
### Bias
|
100 |
+
While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
|