renyuxi Yanzuo commited on
Commit
6e961c9
β€’
1 Parent(s): 00dcb3f

Update README.md (#50)

Browse files

- Update README.md (ba519f28c90c737690fbd82d86acb39a7af2cef2)


Co-authored-by: Yanzuo Lu <Yanzuo@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +25 -3
README.md CHANGED
@@ -18,8 +18,9 @@ Project Page: https://hyper-sd.github.io/
18
 
19
  ## NewsπŸ”₯πŸ”₯πŸ”₯
20
 
21
- * May.13, 2024. πŸ’₯πŸ’₯πŸ’₯ The **12-Steps CFG-Preserved** [Hyper-SDXL-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-12steps-CFG-lora.safetensors) and [Hyper-SD15-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-12steps-CFG-lora.safetensors) is also available now(support 5~8 guidance scales), this could be more practical with better trade-off between performance and speed. Enjoy! πŸ’₯πŸ’₯πŸ’₯
22
- * Apr.30, 2024. Our **8-Steps CFG-Preserved** [Hyper-SDXL-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-8steps-CFG-lora.safetensors) and [Hyper-SD15-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-8steps-CFG-lora.safetensors) is available now(support 5~8 guidance scales), we strongly recommend making the 8-step CFGLora a standard configuration for all SDXL and SD15 models!!!
 
23
  * Apr.28, 2024. ComfyUI workflows on 1-Step Unified LoRA πŸ₯° with TCDScheduler to inference on different steps are [released](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui)! Remember to install ⭕️ [ComfyUI-TCD](https://github.com/JettHu/ComfyUI-TCD) in your `ComfyUI/custom_nodes` folder!!! You're encouraged to adjust the eta parameter to get better results 🌟!
24
  * Apr.26, 2024. Thanks to @[Pete](https://huggingface.co/pngwn) for contributing to our [scribble demo](https://huggingface.co/spaces/ByteDance/Hyper-SD15-Scribble) with larger canvas right now πŸ‘.
25
  * Apr.24, 2024. The ComfyUI [workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-1step-Unet-workflow.json) and [checkpoint](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-1step-Unet-Comfyui.fp16.safetensors) on 1-Step SDXL UNet ✨ is also available! Don't forget ⭕️ to install the custom [scheduler](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui/ComfyUI-HyperSDXL1StepUnetScheduler) in your `ComfyUI/custom_nodes` folder!!!
@@ -36,15 +37,36 @@ Hyper-SDXL One-step Text-to-Image demo host on [πŸ€— T2I](https://huggingface.co
36
  ## Introduction
37
 
38
  Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques.
39
- In this repository, we release the models distilled from [SDXL Base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and [Stable-Diffusion v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)。
40
 
41
  ## Checkpoints
42
 
 
43
  * `Hyper-SDXL-Nstep-lora.safetensors`: Lora checkpoint, for SDXL-related models.
44
  * `Hyper-SD15-Nstep-lora.safetensors`: Lora checkpoint, for SD1.5-related models.
45
  * `Hyper-SDXL-1step-unet.safetensors`: Unet checkpoint distilled from SDXL-Base.
46
 
47
  ## Text-to-Image Usage
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
  ### SDXL-related models
49
  #### 2-Steps, 4-Steps, 8-steps LoRA
50
  Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.
 
18
 
19
  ## NewsπŸ”₯πŸ”₯πŸ”₯
20
 
21
+ * Aug.19, 2024. πŸ’₯πŸ’₯πŸ’₯ **SD3-related CFG LoRAs** are available now! We recommend setting guidance scale to 3.0/5.0/7.0 at 4/8/16-steps. Don't forget to fuse lora with a relatively small scale (e.g. 0.125 that is adaptive with training) before inference with diffusers. Note that 8-steps and 16-steps LoRA can also inference on a little bit smaller steps like 6-steps and 12-steps, respectively. Hope to hear your feedback, FLUX-related models will be coming next week. πŸ’₯πŸ’₯πŸ’₯
22
+ * May.13, 2024. The 12-Steps CFG-Preserved [Hyper-SDXL-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-12steps-CFG-lora.safetensors) and [Hyper-SD15-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-12steps-CFG-lora.safetensors) is also available now(support 5~8 guidance scales), this could be more practical with better trade-off between performance and speed. Enjoy!
23
+ * Apr.30, 2024. Our 8-Steps CFG-Preserved [Hyper-SDXL-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-8steps-CFG-lora.safetensors) and [Hyper-SD15-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-8steps-CFG-lora.safetensors) is available now(support 5~8 guidance scales), we strongly recommend making the 8-step CFGLora a standard configuration for all SDXL and SD15 models!!!
24
  * Apr.28, 2024. ComfyUI workflows on 1-Step Unified LoRA πŸ₯° with TCDScheduler to inference on different steps are [released](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui)! Remember to install ⭕️ [ComfyUI-TCD](https://github.com/JettHu/ComfyUI-TCD) in your `ComfyUI/custom_nodes` folder!!! You're encouraged to adjust the eta parameter to get better results 🌟!
25
  * Apr.26, 2024. Thanks to @[Pete](https://huggingface.co/pngwn) for contributing to our [scribble demo](https://huggingface.co/spaces/ByteDance/Hyper-SD15-Scribble) with larger canvas right now πŸ‘.
26
  * Apr.24, 2024. The ComfyUI [workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-1step-Unet-workflow.json) and [checkpoint](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-1step-Unet-Comfyui.fp16.safetensors) on 1-Step SDXL UNet ✨ is also available! Don't forget ⭕️ to install the custom [scheduler](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui/ComfyUI-HyperSDXL1StepUnetScheduler) in your `ComfyUI/custom_nodes` folder!!!
 
37
  ## Introduction
38
 
39
  Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques.
40
+ In this repository, we release the models distilled from [SD3-Medium](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers), [SDXL Base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and [Stable-Diffusion v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)。
41
 
42
  ## Checkpoints
43
 
44
+ * `Hyper-SD3-Nsteps-CFG-lora.safetensors`: Lora checkpoint, for SD3-related models.
45
  * `Hyper-SDXL-Nstep-lora.safetensors`: Lora checkpoint, for SDXL-related models.
46
  * `Hyper-SD15-Nstep-lora.safetensors`: Lora checkpoint, for SD1.5-related models.
47
  * `Hyper-SDXL-1step-unet.safetensors`: Unet checkpoint distilled from SDXL-Base.
48
 
49
  ## Text-to-Image Usage
50
+
51
+ ### SD3-related models
52
+ ```python
53
+ import torch
54
+ from diffusers import StableDiffusion3Pipeline
55
+ from huggingface_hub import hf_hub_download
56
+ base_model_id = "stabilityai/stable-diffusion-3-medium-diffusers"
57
+ repo_name = "ByteDance/Hyper-SD"
58
+ # Take 8-steps lora as an example
59
+ ckpt_name = "Hyper-SD3-8steps-CFG-lora.safetensors"
60
+ # Load model, please fill in your access tokens since SD3 repo is a gated model.
61
+ pipe = StableDiffusion3Pipeline.from_pretrained(base_model_id, token="xxx")
62
+ pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))
63
+ pipe.fuse_lora(lora_scale=0.125)
64
+ pipe.to("cuda", dtype=torch.float16)
65
+ image=pipe(prompt="a photo of a cat", num_inference_steps=8, guidance_scale=5.0).images[0]
66
+ image.save("output.png")
67
+
68
+ ```
69
+
70
  ### SDXL-related models
71
  #### 2-Steps, 4-Steps, 8-steps LoRA
72
  Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.