File size: 3,084 Bytes
87d101a 7fa487f 87d101a 7b22b9e 87d101a e0afa42 87d101a 7b22b9e 87d101a 7b22b9e 87d101a 7b22b9e 87d101a 7b22b9e 87d101a 7b22b9e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
---
base_model: stabilityai/stable-diffusion-xl-base-1.0
library_name: diffusers
license: openrail++
tags:
- text-to-image
- text-to-image
- diffusers-training
- diffusers
- lora
- template:sd-lora
- stable-diffusion-xl
- stable-diffusion-xl-diffusers
datasets:
- data-is-better-together/open-image-preferences-v1-binarized
language:
- en
pipeline_tag: text-to-image
---
# Low Rank Adapted Supervised Fine Tuned Stable Diffusion XL
## Comparison
| Prompt | SDXL | Fine Tuned |
| :--: | :--: | :--: |
| a boat in the canals of Venice, painted in gouache with soft, flowing brushstrokes and vibrant, translucent colors, capturing the serene reflection on the water under a misty ambiance, with rich textures and a dynamic perspective |  |  |
| Grainy shot of a robot cooking in the kitchen, with soft shadows and nostalgic film texture. |  |  |
## Model description
These are ariG23498/open-image-preferences-v1-sdxl-lora LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.
The weights were trained using [DreamBooth](https://github.com/ariG23498/diffusers/blob/aritra/sdxl-lora/examples/dreambooth/train_dreambooth_lora_sdxl.py) using the
[open-image-preferences-v1-binarized](https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1-binarized) dataset.
## Use with `diffusers`
```py
from diffusers import AutoPipelineForText2Image
import torch
pipeline = AutoPipelineForText2Image.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.bfloat16
).to('cuda')
pipeline.load_lora_weights('ariG23498/open-image-preferences-v1-sdxl-lora', weight_name='pytorch_lora_weights.safetensors')
prompt = "ENTER PROMPT"
image = pipeline(prompt).images[0]
```
## Command to train the model
```shell
!accelerate launch examples/dreambooth/train_dreambooth_lora_sdxl.py \
--pretrained_model_name_or_path "stabilityai/stable-diffusion-xl-base-1.0" \
--dataset_name "data-is-better-together/open-image-preferences-v1-binarized" \
--hub_model_id "ariG23498/open-image-preferences-v1-sdxl-lora" \
--push_to_hub \
--output_dir "open-image-preferences-v1-sdxl-lora" \
--image_column "chosen" \
--caption_column "prompt" \
--mixed_precision="bf16" \
--resolution=1024 \
--train_batch_size=1 \
--repeats=1 \
--report_to="wandb"\
--gradient_accumulation_steps=1 \
--gradient_checkpointing \
--learning_rate=1.0 \
--text_encoder_lr=1.0 \
--optimizer="prodigy"\
--lr_scheduler="constant" \
--lr_warmup_steps=0 \
--rank=8 \
--checkpointing_steps=2000 \
--seed="0"
``` |