Edit model card

sd3-lora-training-rank128-v3

This is a standard PEFT LoRA derived from stabilityai/stable-diffusion-3-medium-diffusers.

The main validation prompt used during training was:

A design of a cute pokemon, featuring a single bird, water type pokemon, on a white background, style of Sugimori Kenji Vector Art

Validation settings

  • CFG: 5.0
  • CFG Rescale: 0.0
  • Steps: 20
  • Sampler: None
  • Seed: 42
  • Resolution: 1024x1024

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)
Negative Prompt
blurry, cropped, ugly, missing body, missing limbs, missing ears, white noise
Prompt
A design of a cute pokemon, featuring a single bird, water type pokemon, on a white background, style of Sugimori Kenji Vector Art
Negative Prompt
blurry, cropped, ugly, missing body, missing limbs, missing ears, white noise

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

  • Training epochs: 59
  • Training steps: 4400
  • Learning rate: 0.0001
  • Effective batch size: 2
    • Micro-batch size: 2
    • Gradient accumulation steps: 1
    • Number of GPUs: 1
  • Prediction type: flow-matching
  • Rescaled betas zero SNR: False
  • Optimizer: adamw_bf16
  • Precision: bf16
  • Quantised: No
  • Xformers: Not used
  • LoRA Rank: 128
  • LoRA Alpha: None
  • LoRA Dropout: 0.1
  • LoRA initialisation style: default

Datasets

pal_type

  • Repeats: 0
  • Total number of images: 73
  • Total number of aspect buckets: 1
  • Resolution: 1.048576 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square

Inference

import torch
from diffusers import DiffusionPipeline

model_id = 'stabilityai/stable-diffusion-3-medium-diffusers'
adapter_id = 'zwloong/sd3-lora-training-rank128-v3'
pipeline = DiffusionPipeline.from_pretrained(model_id)
pipeline.load_lora_weights(adapter_id)

prompt = "A design of a cute pokemon, featuring a single bird, water type pokemon, on a white background, style of Sugimori Kenji Vector Art"
negative_prompt = 'blurry, cropped, ugly, missing body, missing limbs, missing ears, white noise'
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu')
image = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(1641421826),
    width=1024,
    height=1024,
    guidance_scale=5.0,
).images[0]
image.save("output.png", format="PNG")
Downloads last month
3
Inference API
Examples

Model tree for zwloong/sd3-lora-training-rank128-v3

Adapter
(149)
this model