NebulaeWis
commited on
Commit
•
183d7ed
1
Parent(s):
5f3686f
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,279 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: other
|
3 |
+
license_name: fair-ai-public-license-1.0-sd
|
4 |
+
license_link: https://freedevproject.org/faipl-1.0-sd/
|
5 |
+
language:
|
6 |
+
- en
|
7 |
+
base_model:
|
8 |
+
- Laxhar/noobai-XL_v1.0
|
9 |
+
pipeline_tag: text-to-image
|
10 |
+
tags:
|
11 |
+
- safetensors
|
12 |
+
- diffusers
|
13 |
+
- stable-diffusion
|
14 |
+
- stable-diffusion-xl
|
15 |
+
- art
|
16 |
+
library_name: diffusers
|
17 |
+
---
|
18 |
+
|
19 |
+
<h1 align="center"><strong style="font-size: 48px;">NoobAI XL V-Pred 0.5</strong></h1>
|
20 |
+
|
21 |
+
# Model Introduction
|
22 |
+
|
23 |
+
This image generation model, based on Laxhar/noobai-XL_v1.0, leverages full Danbooru and e621 datasets with native tags and natural language captioning.
|
24 |
+
|
25 |
+
Implemented as a v-prediction model (distinct from eps-prediction), it requires specific parameter configurations - detailed in following sections.
|
26 |
+
|
27 |
+
Special thanks to my teammate euge for the coding work, and we're grateful for the technical support from many helpful community members.
|
28 |
+
|
29 |
+
# ⚠️ IMPORTANT NOTICE ⚠️
|
30 |
+
|
31 |
+
## **THIS MODEL WORKS DIFFERENT FROM EPS MODELS!**
|
32 |
+
|
33 |
+
## **PLEASE READ THE GUIDE CAREFULLY!**
|
34 |
+
|
35 |
+
## Model Details
|
36 |
+
|
37 |
+
- **Developed by**: [Laxhar Lab](https://huggingface.co/Laxhar)
|
38 |
+
- **Model Type**: Diffusion-based text-to-image generative model
|
39 |
+
- **Fine-tuned from**: Laxhar/noobai-XL_v1.0
|
40 |
+
- **Sponsored by from**: [Lanyun Cloud](https://cloud.lanyun.net)
|
41 |
+
|
42 |
+
---
|
43 |
+
|
44 |
+
# How to Use the Model.
|
45 |
+
|
46 |
+
## Method I: [reForge](https://github.com/Panchovix/stable-diffusion-webui-reForge/tree/dev_upstream)
|
47 |
+
|
48 |
+
1. (If you haven't installed reForge) Install reForge by following the instructions in the repository;
|
49 |
+
2. Switch to `dev_upstream` branch:
|
50 |
+
|
51 |
+
```bash
|
52 |
+
git checkout dev_upstream
|
53 |
+
```
|
54 |
+
|
55 |
+
3. Update reforge:
|
56 |
+
|
57 |
+
```bash
|
58 |
+
git pull
|
59 |
+
```
|
60 |
+
|
61 |
+
4. Launch WebUI and use the model as usual!
|
62 |
+
|
63 |
+
## Method II: [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
|
64 |
+
|
65 |
+
SAMLPLE with NODES
|
66 |
+
|
67 |
+
[comfy_ui_workflow_sample](/Laxhar/noobai-XL-Vpred-0.5/blob/main/comfy_ui_workflow_sample.png)
|
68 |
+
|
69 |
+
|
70 |
+
## Method III: [WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
|
71 |
+
|
72 |
+
Note that dev branch is not stable and **may contain bugs**.
|
73 |
+
|
74 |
+
1. (If you haven't installed WebUI) Install WebUI by following the instructions in the repository. For simp
|
75 |
+
2. Switch to `dev` branch:
|
76 |
+
|
77 |
+
```bash
|
78 |
+
git switch dev
|
79 |
+
```
|
80 |
+
|
81 |
+
3. Pull latest updates:
|
82 |
+
|
83 |
+
```bash
|
84 |
+
git pull
|
85 |
+
```
|
86 |
+
|
87 |
+
4. Launch WebUI and use the model as usual!
|
88 |
+
|
89 |
+
## Method IV: [Diffusers](https://huggingface.co/docs/diffusers/en/index)
|
90 |
+
|
91 |
+
```python
|
92 |
+
import torch
|
93 |
+
from diffusers import StableDiffusionXLPipeline
|
94 |
+
from diffusers import EulerDiscreteScheduler
|
95 |
+
|
96 |
+
ckpt_path = "/path/to/model.safetensors"
|
97 |
+
pipe = StableDiffusionXLPipeline.from_single_file(
|
98 |
+
ckpt_path,
|
99 |
+
use_safetensors=True,
|
100 |
+
torch_dtype=torch.float16,
|
101 |
+
)
|
102 |
+
scheduler_args = {"prediction_type": "v_prediction", "rescale_betas_zero_snr": True}
|
103 |
+
pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, **scheduler_args)
|
104 |
+
pipe.enable_xformers_memory_efficient_attention()
|
105 |
+
pipe = pipe.to("cuda")
|
106 |
+
|
107 |
+
prompt = """masterpiece, best quality,artist:john_kafka,artist:nixeu,artist:quasarcake, chromatic aberration, film grain, horror \(theme\), limited palette, x-shaped pupils, high contrast, color contrast, cold colors, arlecchino \(genshin impact\), black theme, gritty, graphite \(medium\)"""
|
108 |
+
negative_prompt = "nsfw, worst quality, old, early, low quality, lowres, signature, username, logo, bad hands, mutated hands, mammal, anthro, furry, ambiguous form, feral, semi-anthro"
|
109 |
+
|
110 |
+
image = pipe(
|
111 |
+
prompt=prompt,
|
112 |
+
negative_prompt=negative_prompt,
|
113 |
+
width=832,
|
114 |
+
height=1216,
|
115 |
+
num_inference_steps=28,
|
116 |
+
guidance_scale=5,
|
117 |
+
generator=torch.Generator().manual_seed(42),
|
118 |
+
).images[0]
|
119 |
+
|
120 |
+
image.save("output.png")
|
121 |
+
```
|
122 |
+
|
123 |
+
|
124 |
+
**Note**: Please make sure Git is installed and environment is properly configured on your machine.
|
125 |
+
|
126 |
+
---
|
127 |
+
|
128 |
+
# Recommended Settings
|
129 |
+
|
130 |
+
## Parameters
|
131 |
+
|
132 |
+
- CFG: 4 ~ 5
|
133 |
+
- Steps: 28 ~ 35
|
134 |
+
- Sampling Method: **Euler** (⚠️ Other samplers will not work properly)
|
135 |
+
- Resolution: Total area around 1024x1024. Best to choose from: 768x1344, **832x1216**, 896x1152, 1024x1024, 1152x896, 1216x832, 1344x768
|
136 |
+
|
137 |
+
## Prompts
|
138 |
+
|
139 |
+
- Prompt Prefix:
|
140 |
+
|
141 |
+
```
|
142 |
+
masterpiece, best quality, newest, absurdres, highres, safe,
|
143 |
+
```
|
144 |
+
|
145 |
+
- Negative Prompt:
|
146 |
+
|
147 |
+
```
|
148 |
+
nsfw, worst quality, old, early, low quality, lowres, signature, username, logo, bad hands, mutated hands, mammal, anthro, furry, ambiguous form, feral, semi-anthro
|
149 |
+
```
|
150 |
+
|
151 |
+
# Usage Guidelines
|
152 |
+
|
153 |
+
## Caption
|
154 |
+
|
155 |
+
```
|
156 |
+
<1girl/1boy/1other/...>, <character>, <series>, <artists>, <special tags>, <general tags>, <other tags>
|
157 |
+
```
|
158 |
+
|
159 |
+
## Quality Tags
|
160 |
+
|
161 |
+
For quality tags, we evaluated image popularity through the following process:
|
162 |
+
|
163 |
+
- Data normalization based on various sources and ratings.
|
164 |
+
- Application of time-based decay coefficients according to date recency.
|
165 |
+
- Ranking of images within the entire dataset based on this processing.
|
166 |
+
|
167 |
+
Our ultimate goal is to ensure that quality tags effectively track user preferences in recent years.
|
168 |
+
|
169 |
+
| Percentile Range | Quality Tags |
|
170 |
+
| :--------------- | :------------- |
|
171 |
+
| > 95th | masterpiece |
|
172 |
+
| > 85th, <= 95th | best quality |
|
173 |
+
| > 60th, <= 85th | good quality |
|
174 |
+
| > 30th, <= 60th | normal quality |
|
175 |
+
| <= 30th | worst quality |
|
176 |
+
|
177 |
+
## Aesthetic Tags
|
178 |
+
|
179 |
+
| Tag | Description |
|
180 |
+
| :-------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
181 |
+
| very awa | Top 5% of images in terms of aesthetic score by [waifu-scorer](https://huggingface.co/Eugeoter/waifu-scorer-v4-beta) |
|
182 |
+
| worst aesthetic | All the bottom 5% of images in terms of aesthetic score by [waifu-scorer](https://huggingface.co/Eugeoter/waifu-scorer-v4-beta) and [aesthetic-shadow-v2](https://huggingface.co/shadowlilac/aesthetic-shadow-v2) |
|
183 |
+
| ... | ... |
|
184 |
+
|
185 |
+
## Date Tags
|
186 |
+
|
187 |
+
There are two types of date tags: **year tags** and **period tags**. For year tags, use `year xxxx` format, i.e., `year 2021`. For period tags, please refer to the following table:
|
188 |
+
|
189 |
+
| Year Range | Period tag |
|
190 |
+
| :--------- | :--------- |
|
191 |
+
| 2005-2010 | old |
|
192 |
+
| 2011-2014 | early |
|
193 |
+
| 2014-2017 | mid |
|
194 |
+
| 2018-2020 | recent |
|
195 |
+
| 2021-2024 | newest |
|
196 |
+
|
197 |
+
## Dataset
|
198 |
+
|
199 |
+
- The latest Danbooru images up to the training date (approximately before 2024-10-23)
|
200 |
+
- E621 images [e621-2024-webp-4Mpixel](https://huggingface.co/datasets/NebulaeWis/e621-2024-webp-4Mpixel) dataset on Hugging Face
|
201 |
+
|
202 |
+
**Communication**
|
203 |
+
|
204 |
+
- **QQ Groups:**
|
205 |
+
|
206 |
+
- 875042008
|
207 |
+
- 914818692
|
208 |
+
- 635772191
|
209 |
+
|
210 |
+
- **Discord:** [Laxhar Dream Lab SDXL NOOB](https://discord.com/invite/DKnFjKEEvH)
|
211 |
+
|
212 |
+
**How to train a LoRA on v-pred SDXL model**
|
213 |
+
|
214 |
+
A tutorial is intended for LoRA trainers based on sd-scripts.
|
215 |
+
|
216 |
+
article link: https://civitai.com/articles/8723
|
217 |
+
|
218 |
+
**Utility Tool**
|
219 |
+
|
220 |
+
Laxhar Lab is training a dedicated ControlNet model for NoobXL, and the models are being released progressively. So far, the normal, depth, and canny have been released.
|
221 |
+
|
222 |
+
Model link: https://civitai.com/models/929685
|
223 |
+
|
224 |
+
# Model License
|
225 |
+
|
226 |
+
This model's license inherits from https://huggingface.co/OnomaAIResearch/Illustrious-xl-early-release-v0 fair-ai-public-license-1.0-sd and adds the following terms. Any use of this model and its variants is bound by this license.
|
227 |
+
|
228 |
+
## I. Usage Restrictions
|
229 |
+
|
230 |
+
- Prohibited use for harmful, malicious, or illegal activities, including but not limited to harassment, threats, and spreading misinformation.
|
231 |
+
- Prohibited generation of unethical or offensive content.
|
232 |
+
- Prohibited violation of laws and regulations in the user's jurisdiction.
|
233 |
+
|
234 |
+
## II. Commercial Prohibition
|
235 |
+
|
236 |
+
We prohibit any form of commercialization, including but not limited to monetization or commercial use of the model, derivative models, or model-generated products.
|
237 |
+
|
238 |
+
## III. Open Source Community
|
239 |
+
|
240 |
+
To foster a thriving open-source community,users MUST comply with the following requirements:
|
241 |
+
|
242 |
+
- Open source derivative models, merged models, LoRAs, and products based on the above models.
|
243 |
+
- Share work details such as synthesis formulas, prompts, and workflows.
|
244 |
+
- Follow the fair-ai-public-license to ensure derivative works remain open source.
|
245 |
+
|
246 |
+
## IV. Disclaimer
|
247 |
+
|
248 |
+
Generated models may produce unexpected or harmful outputs. Users must assume all risks and potential consequences of usage.
|
249 |
+
|
250 |
+
# Participants and Contributors
|
251 |
+
|
252 |
+
## Participants
|
253 |
+
|
254 |
+
- **L_A_X:** [Civitai](https://civitai.com/user/L_A_X) | [Liblib.art](https://www.liblib.art/userpage/9e1b16538b9657f2a737e9c2c6ebfa69) | [Huggingface](https://huggingface.co/LAXMAYDAY)
|
255 |
+
- **li_li:** [Civitai](https://civitai.com/user/li_li) | [Huggingface](https://huggingface.co/heziiiii)
|
256 |
+
- **nebulae:** [Civitai](https://civitai.com/user/kitarz) | [Huggingface](https://huggingface.co/NebulaeWis)
|
257 |
+
- **Chenkin:** [Civitai](https://civitai.com/user/Chenkin) | [Huggingface](https://huggingface.co/windsingai)
|
258 |
+
- **Euge:** [Civitai](https://civitai.com/user/Euge_) | [Huggingface](https://huggingface.co/Eugeoter) | [Github](https://github.com/Eugeoter)
|
259 |
+
|
260 |
+
## Contributors
|
261 |
+
|
262 |
+
- **Narugo1992**: Thanks to [narugo1992](https://github.com/narugo1992) and the [deepghs](https://huggingface.co/deepghs) team for open-sourcing various training sets, image processing tools, and models.
|
263 |
+
|
264 |
+
- **Mikubill**: Thanks to [Mikubill](https://github.com/Mikubill) for the [Naifu](https://github.com/Mikubill/naifu) trainer.
|
265 |
+
|
266 |
+
- **Onommai**: Thanks to [OnommAI](https://onomaai.com/) for open-sourcing a powerful base model.
|
267 |
+
|
268 |
+
- **V-Prediction**: Thanks to the following individuals for their detailed instructions and experiments.
|
269 |
+
|
270 |
+
- adsfssdf
|
271 |
+
- [bluvoll](https://civitai.com/user/bluvoll)
|
272 |
+
- [bvhari](https://github.com/bvhari)
|
273 |
+
- [catboxanon](https://github.com/catboxanon)
|
274 |
+
- [parsee-mizuhashi](https://huggingface.co/parsee-mizuhashi)
|
275 |
+
- [very-aesthetic](https://github.com/very-aesthetic)
|
276 |
+
- [momoura](https://civitai.com/user/momoura)
|
277 |
+
- madmanfourohfour
|
278 |
+
|
279 |
+
- **Community**: [aria1th261](https://civitai.com/user/aria1th261), [neggles](https://github.com/neggles/neurosis), [sdtana](https://huggingface.co/sdtana), [chewing](https://huggingface.co/chewing), [irldoggo](https://github.com/irldoggo), [reoe](https://huggingface.co/reoe), [kblueleaf](https://civitai.com/user/kblueleaf), [Yidhar](https://github.com/Yidhar), ageless, 白玲可, Creeper, KaerMorh, 吟游诗人, SeASnAkE, [zwh20081](https://civitai.com/user/zwh20081), Wenaka~喵, 稀里哗啦, 幸运二副, 昨日の約, 445, [EBIX](https://civitai.com/user/EBIX), [Sopp](https://huggingface.co/goyishsoyish), [Y_X](https://civitai.com/user/Y_X), [Minthybasis](https://civitai.com/user/Minthybasis), [Rakosz](https://civitai.com/user/Rakosz)
|