Update models

DOCS.md

@@ -10,9 +10,9 @@ Use `+` or `-` to increase the weight of a token. The weight grows exponentially
 
 Groups of tokens can be weighted together by wrapping in parantheses and multiplying by a float between 0 and 2. For example, `(masterpiece, best quality)1.2` will increase the weight of both `masterpiece` and `best quality` by 1.2x.
 
-This is the same syntax used in [InvokeAI](https://invoke-ai.github.io/InvokeAI/features/PROMPTS/) and it differs from AUTOMATIC1111:
+This is the same syntax used in [InvokeAI](https://invoke-ai.github.io/InvokeAI/features/PROMPTS/) and it differs from [A1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui):
 
-| Compel      | AUTOMATIC1111 |
+| Compel      | A1111         |
 | ----------- | ------------- |
 | `blue++`    | `((blue))`    |
 | `blue--`    | `[[blue]]`    |
@@ -21,32 +21,22 @@ This is the same syntax used in [InvokeAI](https://invoke-ai.github.io/InvokeAI/
 
 ### Models
 
-Each model checkpoint has a different aesthetic:
+Some require specific parameters to get the best results, so check the model's link for more information:
 
-* [Comfy-Org/stable-diffusion-v1-5](https://huggingface.co/Comfy-Org/stable-diffusion-v1-5-archive): base
-* [cyberdelia/CyberRealistic_V5](https://huggingface.co/cyberdelia/CyberRealistic): realistic
-* [Lykon/dreamshaper-8](https://huggingface.co/Lykon/dreamshaper-8): general purpose (default)
-* [fluently/Fluently-v4](https://huggingface.co/fluently/Fluently-v4): general purpose stylized
-* [Linaqruf/anything-v3-1](https://huggingface.co/Linaqruf/anything-v3-1): anime
-* [prompthero/openjourney-v4](https://huggingface.co/prompthero/openjourney-v4): Midjourney art style
-* [SG161222/Realistic_Vision_V5](https://huggingface.co/SG161222/Realistic_Vision_V5.1_noVAE): realistic
-* [XpucT/Deliberate_v6](https://huggingface.co/XpucT/Deliberate): general purpose stylized
+* [Lykon/dreamshaper-8](https://huggingface.co/Lykon/dreamshaper-8)(default)
+* [cyberdelia/CyberRealistic_V5](https://huggingface.co/cyberdelia/CyberRealistic)
+* [dreamlike-art/dreamlike-photoreal-2.0](https://huggingface.co/dreamlike-art/dreamlike-photoreal-2.0)
+* [fluently/Fluently-v4](https://huggingface.co/fluently/Fluently-v4)
+* [s6yx/ReV_Animated](https://huggingface.co/s6yx/ReV_Animated)
+* [SG161222/Realistic_Vision_V5](https://huggingface.co/SG161222/Realistic_Vision_V5.1_noVAE)
+* [stable-diffusion-v1-5/stable-diffusion-v1-5](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5)
+* [XpucT/Deliberate_v6](https://huggingface.co/XpucT/Deliberate)
 
 ### Styles
 
-[Styles](https://huggingface.co/spaces/adamelliotfields/diffusion/blob/main/data/styles.json) are prompt templates that wrap your positive and negative prompts. They were originally derived from the [twri/sdxl_prompt_styler](https://github.com/twri/sdxl_prompt_styler) Comfy node, but have since been entirely rewritten.
+[Styles](https://huggingface.co/spaces/adamelliotfields/diffusion/blob/main/data/styles.json) are prompt templates that wrap your positive and negative prompts. Inspired by [twri/sdxl_prompt_styler](https://github.com/twri/sdxl_prompt_styler).
 
-Start by framing a simple subject like `portrait of a cat` or `landscape of a mountain range` and experiment.
-
-#### Anime
-
-The `Anime: *` styles work the best with Dreamshaper. When using the anime-specific Anything model, you should use the `Anime: Anything` style with the following settings:
-
-* Scheduler: `DEIS 2M` or `DPM++ 2M`
-* Guidance: `10`
-* Steps: `50`
-
-You subject should be a few simple tokens like `girl, brunette, blue eyes, armor, nebula, celestial`. Experiment with `Clip Skip` and `Karras`. 
+> 💡 When using syles, start with a simple prompt like `portrait of a cat` or `landscape of a mountain range`.
 
 ### Scale
 
@@ -54,7 +44,7 @@ Rescale up to 4x using [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) wit
 
 ### Image-to-Image
 
-The `Image-to-Image` settings allows you to provide input images for the initial latents, ControlNet, and IP-Adapter.
+The `Image-to-Image` settings allows you to provide input images for the initial latent, ControlNet, and IP-Adapter.
 
 #### Strength
 
@@ -70,7 +60,7 @@ Currently, the only annotator available is [Canny](https://huggingface.co/lllyas
 
 #### IP-Adapter
 
-In an image-to-image pipeline, the input image is used as the initial latent. With [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter), the input image is processed by a separate image encoder and the encoded features are used as conditioning along with the text prompt.
+In an image-to-image pipeline, the input image is used as the initial latent representation. With [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter), the image is processed by a separate image encoder and the encoded features are used as conditioning along with the text prompt.
 
 For capturing faces, enable `IP-Adapter Face` to use the full-face model. You should use an input image that is mostly a face and it should be high quality. You can generate fake portraits with Realistic Vision to experiment.
 
@@ -82,7 +72,7 @@ Enable `Use negative TI` to append [`fast_negative`](https://civitai.com/models/
 
 #### DeepCache
 
-[DeepCache](https://github.com/horseee/DeepCache) caches lower UNet layers and reuses them every `Interval` steps. Trade quality for speed:
+[DeepCache](https://github.com/horseee/DeepCache) caches lower UNet layers and reuses them every _n_ steps. Trade quality for speed:
 * `1`: no caching (default)
 * `2`: more quality
 * `3`: balanced

README.md

@@ -15,31 +15,28 @@ header: mini
 license: apache-2.0
 models:
 - ai-forever/Real-ESRGAN
-- Comfy-Org/stable-diffusion-v1-5-archive
 - cyberdelia/CyberRealistic
+- dreamlike-art/dreamlike-photoreal-2.0
 - fluently/Fluently-v4
 - h94/IP-Adapter
-- Linaqruf/anything-v3-1
 - Lykon/dreamshaper-8
-- prompthero/openjourney-v4
+- s6yx/ReV_Animated
 - SG161222/Realistic_Vision_V5.1_noVAE
+- stable-diffusion-v1-5/stable-diffusion-v1-5
 - XpucT/Deliberate
 preload_from_hub:  # up to 10
-- >-
-  Comfy-Org/stable-diffusion-v1-5-archive
-  v1-5-pruned-emaonly-fp16.safetensors
 - >-
   cyberdelia/CyberRealistic
   CyberRealistic_V5_FP16.safetensors
+- >-
+  dreamlike-art/dreamlike-photoreal-2.0
+  dreamlike-photoreal-2.0.safetensors
 - >-
   fluently/Fluently-v4
   Fluently-v4.safetensors
 - >-
   h94/IP-Adapter
   models/ip-adapter-full-face_sd15.safetensors,models/ip-adapter-plus_sd15.safetensors,models/image_encoder/model.safetensors
-- >-
-  Linaqruf/anything-v3-1
-  anything-v3-2.safetensors
 - >-
   lllyasviel/control_v11p_sd15_canny
   diffusion_pytorch_model.fp16.safetensors
@@ -47,11 +44,14 @@ preload_from_hub:  # up to 10
   Lykon/dreamshaper-8
   feature_extractor/preprocessor_config.json,safety_checker/config.json,scheduler/scheduler_config.json,text_encoder/config.json,text_encoder/model.fp16.safetensors,tokenizer/merges.txt,tokenizer/special_tokens_map.json,tokenizer/tokenizer_config.json,tokenizer/vocab.json,unet/config.json,unet/diffusion_pytorch_model.fp16.safetensors,vae/config.json,vae/diffusion_pytorch_model.fp16.safetensors,model_index.json
 - >-
-  prompthero/openjourney-v4
-  openjourney-v4.ckpt
+  s6yx/ReV_Animated
+  rev_1.2.2/rev_1.2.2-fp16.safetensors
 - >-
   SG161222/Realistic_Vision_V5.1_noVAE
   Realistic_Vision_V5.1_fp16-no-ema.safetensors
+- >-
+  stable-diffusion-v1-5/stable-diffusion-v1-5
+  feature_extractor/preprocessor_config.json,safety_checker/config.json,scheduler/scheduler_config.json,text_encoder/config.json,text_encoder/model.fp16.safetensors,tokenizer/merges.txt,tokenizer/special_tokens_map.json,tokenizer/tokenizer_config.json,tokenizer/vocab.json,unet/config.json,unet/diffusion_pytorch_model.fp16.safetensors,vae/config.json,vae/diffusion_pytorch_model.fp16.safetensors,model_index.json
 - >-
   XpucT/Deliberate
   Deliberate_v6.safetensors
@@ -83,7 +83,7 @@ git remote set-url origin https://adamelliotfields:$HF_TOKEN@huggingface.co/spac
 # install
 python -m venv .venv
 source .venv/bin/activate
-pip install -r requirements.txt torch==2.4.0 torchvision==0.19.0
+pip install -r requirements.txt
 
 # gradio
 python app.py --port 7860

app.py

@@ -185,12 +185,14 @@ with gr.Blocks(
                 negative_prompt = gr.Textbox(
                     label="Negative Prompt",
                     value="nsfw+",
+                    min_width=320,
                     lines=1,
                 )
                 styles = json.loads(read_file("data/styles.json"))
                 style_ids = list(styles.keys())
                 style_ids = [sid for sid in style_ids if not sid.startswith("_")]
                 style = gr.Dropdown(
+                    min_width=320,
                     value=Config.STYLE,
                     label="Style Template",
                     choices=[("None", "none")] + [(styles[sid]["name"], sid) for sid in style_ids],
@@ -345,17 +347,17 @@ with gr.Blocks(
                 )
             with gr.Row():
                 disable_image = gr.Checkbox(
-                    label="Disable Initial Image",
+                    label="Disable initial image",
                     elem_classes=["checkbox"],
                     value=False,
                 )
                 disable_control_image = gr.Checkbox(
-                    label="Disable ControlNet Image",
+                    label="Disable ControlNet",
                     elem_classes=["checkbox"],
                     value=False,
                 )
                 disable_ip_image = gr.Checkbox(
-                    label="Disable IP-Adapter Image",
+                    label="Disable IP-Adapter",
                     elem_classes=["checkbox"],
                     value=False,
                 )
@@ -413,6 +415,7 @@ with gr.Blocks(
         fn=lambda image, control_image, ip_image: (image, control_image, ip_image),
         inputs=[disable_image, disable_control_image, disable_ip_image],
         outputs=[DISABLE_IMAGE_PROMPT, DISABLE_CONTROL_IMAGE_PROMPT, DISABLE_IP_IMAGE_PROMPT],
+        show_api=False,
     )
 
     # Generate images

lib/config.py

@@ -62,14 +62,14 @@ Config = SimpleNamespace(
     HF_MODELS={
         # downloaded on startup
         "ai-forever/Real-ESRGAN": ["RealESRGAN_x2.pth", "RealESRGAN_x4.pth"],
-        "Comfy-Org/stable-diffusion-v1-5-archive": ["v1-5-pruned-emaonly-fp16.safetensors"],
         "cyberdelia/CyberRealistic": ["CyberRealistic_V5_FP16.safetensors"],
+        "dreamlike-art/dreamlike-photoreal-2.0": ["dreamlike-photoreal-2.0.safetensors"],
         "fluently/Fluently-v4": ["Fluently-v4.safetensors"],
-        "Linaqruf/anything-v3-1": ["anything-v3-2.safetensors"],
         "lllyasviel/control_v11p_sd15_canny": ["diffusion_pytorch_model.fp16.safetensors"],
         "Lykon/dreamshaper-8": [*sd_files],
-        "prompthero/openjourney-v4": ["openjourney-v4.ckpt"],
+        "s6yx/ReV_Animated": ["rev_1.2.2/rev_1.2.2-fp16.safetensors"],
         "SG161222/Realistic_Vision_V5.1_noVAE": ["Realistic_Vision_V5.1_fp16-no-ema.safetensors"],
+        "stable-diffusion-v1-5/stable-diffusion-v1-5": [*sd_files],
         "XpucT/Deliberate": ["Deliberate_v6.safetensors"],
     },
     MONO_FONTS=["monospace"],
@@ -88,23 +88,22 @@ Config = SimpleNamespace(
     },
     MODEL="Lykon/dreamshaper-8",
     MODELS=[
-        "Comfy-Org/stable-diffusion-v1-5-archive",
         "cyberdelia/CyberRealistic",
+        "dreamlike-art/dreamlike-photoreal-2.0",
         "fluently/Fluently-v4",
-        "Linaqruf/anything-v3-1",
         "Lykon/dreamshaper-8",
-        "prompthero/openjourney-v4",
+        "s6yx/ReV_Animated",
         "SG161222/Realistic_Vision_V5.1_noVAE",
+        "stable-diffusion-v1-5/stable-diffusion-v1-5",
         "XpucT/Deliberate",
     ],
     # Single-file model weights
     MODEL_CHECKPOINTS={
         # keep keys lowercase for case-insensitive matching in the loader
-        "comfy-org/stable-diffusion-v1-5-archive": "v1-5-pruned-emaonly-fp16.safetensors",
         "cyberdelia/cyberrealistic": "CyberRealistic_V5_FP16.safetensors",
+        "dreamlike-art/dreamlike-photoreal-2.0": "dreamlike-photoreal-2.0.safetensors",
         "fluently/fluently-v4": "Fluently-v4.safetensors",
-        "linaqruf/anything-v3-1": "anything-v3-2.safetensors",
-        "prompthero/openjourney-v4": "openjourney-v4.ckpt",
+        "s6yx/rev_animated": "rev_1.2.2/rev_1.2.2-fp16.safetensors",
         "sg161222/realistic_vision_v5.1_novae": "Realistic_Vision_V5.1_fp16-no-ema.safetensors",
         "xpuct/deliberate": "Deliberate_v6.safetensors",
     },

lib/loader.py

@@ -4,7 +4,6 @@ from threading import Lock
 import torch
 from DeepCache import DeepCacheSDHelper
 from diffusers import ControlNetModel
-from diffusers.models import AutoencoderKL
 from diffusers.models.attention_processor import AttnProcessor2_0, IPAdapterAttnProcessor2_0
 
 from .config import Config
@@ -238,23 +237,6 @@ class Loader:
         if self.pipe is not None:
             self.pipe.set_progress_bar_config(disable=progress is not None)
 
-    # Handle single-file and diffusers-style models
-    def _load_vae(self, model=""):
-        msg = "Loading VAE"
-        with timer(msg, logger=self.log.info):
-            if model.lower() in Config.MODEL_CHECKPOINTS.keys():
-                self.pipe.vae = AutoencoderKL.from_single_file(
-                    f"https://huggingface.co/{model}/{Config.MODEL_CHECKPOINTS[model.lower()]}",
-                    torch_dtype=self.pipe.dtype,
-                ).to(self.pipe.device)
-            else:
-                self.pipe.vae = AutoencoderKL.from_pretrained(
-                    pretrained_model_name_or_path=model,
-                    torch_dtype=self.pipe.dtype,
-                    subfolder="vae",
-                    variant="fp16",
-                ).to(self.pipe.device)
-
     def load(
         self,
         kind,
@@ -267,8 +249,6 @@ class Loader:
         karras,
         progress,
     ):
-        device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-
         scheduler_kwargs = {
             "beta_schedule": "scaled_linear",
             "timestep_spacing": "leading",
@@ -297,16 +277,8 @@ class Loader:
         else:
             pipe_kwargs["variant"] = None
 
-        # convert fp32 to bf16 if possible
-        if model.lower() in ["linaqruf/anything-v3-1"]:
-            pipe_kwargs["torch_dtype"] = (
-                torch.bfloat16
-                if torch.cuda.get_device_properties(device).major >= 8
-                else torch.float16
-            )
-        else:
-            # defaults to float32
-            pipe_kwargs["torch_dtype"] = torch.float16
+        # converts to fp32 by default
+        pipe_kwargs["torch_dtype"] = torch.float16
 
         # config maps the repo to the ID: canny -> lllyasviel/control_sd15_canny
         if kind.startswith("controlnet_"):
@@ -339,9 +311,6 @@ class Loader:
             if not same_scheduler or not same_karras:
                 self.pipe.scheduler = Config.SCHEDULERS[scheduler](**scheduler_kwargs)
 
-        # Load VAE
-        self._load_vae(model)
-
         CURRENT_STEP = 1
         TOTAL_STEPS = sum(
             [