|
--- |
|
library_name: diffusers |
|
pipeline_tag: text-to-image |
|
inference: true |
|
base_model: stabilityai/stable-diffusion-2-1 |
|
--- |
|
# DPO LoRA Stable Diffusion v2-1 |
|
Model trained with LoRA implementation of Diffusion DPO Read more [here](https://github.com/huggingface/diffusers/tree/main/examples/research_projects/diffusion_dpo) |
|
|
|
|
|
Base Model: https://huggingface.co/stabilityai/stable-diffusion-2-1 |
|
|
|
## Running with [🧨 diffusers library](https://github.com/huggingface/diffusers) |
|
|
|
|
|
```python |
|
|
|
from diffusers import DiffusionPipeline |
|
from diffusers.utils import make_image_grid |
|
import torch |
|
|
|
pipe = DiffusionPipeline.from_pretrained( |
|
"stabilityai/sd-turbo", # SD Turbo is a destilled version of Stable Diffusion 2.1 |
|
# "stabilityai/stable-diffusion-2-1", # for the original stable diffusion 2.1 model |
|
torch_dtype=torch.float16, variant="fp16" |
|
) |
|
pipe.to("cuda") |
|
pipe.load_lora_weights("radames/sd-21-DPO-LoRA", adapter_name="dpo-lora-sd21") |
|
pipe.set_adapters(["dpo-lora-sd21"], adapter_weights=[1.0]) # you can play with adapter_weights to increase the effect of the LoRA model |
|
seed = 123123 |
|
prompt = "portrait headshot professional of elon musk" |
|
negative_prompt = "3d render, cartoon, drawing, art, low light" |
|
generator = torch.Generator().manual_seed(seed) |
|
images = pipe( |
|
prompt=prompt, |
|
negative_prompt=negative_prompt, |
|
width=512, |
|
height=512, |
|
num_inference_steps=2, |
|
generator=generator, |
|
guidance_scale=1.0, |
|
num_images_per_prompt=4 |
|
).images |
|
make_image_grid(images, 1, 4) |
|
``` |
|
|
|
## Guidance Scale vs LoRA weights |
|
|
|
 |
|
|
|
|
|
|
|
## Examples |
|
|
|
Left Withoud DPO right with DPO LoRA |
|
|
|
<img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/R8E0hRpWIE6OhhtvgJeEU.png style="max-width: 60rem;"> |
|
<img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/Eg4LbyxCfhmsk2INzqODw.png style="max-width: 60rem;"> |
|
<img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/GD7KumSCNweBWMJ1TArI-.png style="max-width: 60rem;"> |
|
<img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/SO7QoA9lZJY9hI0U4fBLy.png style="max-width: 60rem;"> |
|
<img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/ZWbQwIQ5OklEgF9RW581R.png style="max-width: 60rem;"> |
|
|
|
|
