README.md

metadata

license: creativeml-openrail-m
base_model: black-forest-labs/FLUX.1-dev
tags:
  - stable-diffusion
  - stable-diffusion-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - lora
  - template:sd-lora
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      The image is a black and white illustration of a man with a long beard and
      a long white beard, sitting on the ground with his arms raised above his
      head. He appears to be in a meditative pose, with his eyes closed and a
      peaceful expression on his face. Behind him, there are two other men, one
      lying on their backs and the other lying down. The background shows a
      mountain landscape with a cloudy sky. The text above the illustration
      reads 'Let the day perish where I was born. And they sat down with him
      upon the ground seven days & seven nights and none spoke a word unto him
      for they saw that his grief was very great.' The illustration is framed by
      a decorative border.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      elk_style,  Whimsical kitchen scene: man with feathered hat, striped
      scarf, and exaggerated features stands with arms outstretched, as if
      proclaiming; concerned woman in old-fashioned dress and apron sits at
      table with frying pan, surrounded by utensils and dishes, against a
      backdrop of brick wall, curtained window, and wooden floor with rug.
    output:
      url: images/example_2wixkomof.png

lora-Kirchner-flux

This is a LoRA derived from black-forest-labs/FLUX.1-dev.

The main validation prompt used during training was:

The image is a black and white illustration of a man with a long beard and a long white beard, sitting on the ground with his arms raised above his head. He appears to be in a meditative pose, with his eyes closed and a peaceful expression on his face. Behind him, there are two other men, one lying on their backs and the other lying down. The background shows a mountain landscape with a cloudy sky. The text above the illustration reads 'Let the day perish where I was born. And they sat down with him upon the ground seven days & seven nights and none spoke a word unto him for they saw that his grief was very great.' The illustration is framed by a decorative border.

Validation settings

• CFG: 7.5
• CFG Rescale: 0.0
• Steps: 20
• Sampler: None
• Seed: 42
• Resolution: 1024

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt

unconditional (blank prompt)

Negative Prompt

blurry, cropped, ugly

Prompt

elk_style, Whimsical kitchen scene: man with feathered hat, striped scarf, and exaggerated features stands with arms outstretched, as if proclaiming; concerned woman in old-fashioned dress and apron sits at table with frying pan, surrounded by utensils and dishes, against a backdrop of brick wall, curtained window, and wooden floor with rug.

Prompt

elk_style, The image is a painting of a busy street scene in Paris, France. The painting is done in a realistic style with loose brushstrokes and vibrant colors. It shows a woman walking on the street, wearing a blue skirt and a black top hat with a red pom-pom on top. She is carrying a white box in her hand and appears to be looking down at it. On the left side of the painting, there is a street lamp post with a sign that reads "Cafe". On the right side, there are several buildings with ornate architectural details, including a horse-drawn carriage and a group of people walking on a sidewalk. The sky is overcast and the ground is wet, suggesting that it has recently rained.

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

• Training epochs: 235
• Training steps: 4000
• Learning rate: 0.0001
• Effective batch size: 6
  • Micro-batch size: 6
  • Gradient accumulation steps: 1
  • Number of GPUs: 1
• Prediction type: flow-matching
• Rescaled betas zero SNR: False
• Optimizer: AdamW, stochastic bf16
• Precision: Pure BF16
• Xformers: Not used
• LoRA Rank: 64
• LoRA Alpha: None
• LoRA Dropout: 0.1
• LoRA initialisation style: default

Datasets

ErnstLudwigKirchner

• Repeats: 0
• Total number of images: 102
• Total number of aspect buckets: 1
• Resolution: 1024 px
• Cropped: True
• Crop style: center
• Crop aspect: square

Inference

import torch
from diffusers import DiffusionPipeline

model_id = 'black-forest-labs/FLUX.1-dev'
adapter_id = 'davidrd123/lora-Kirchner-flux'
pipeline = DiffusionPipeline.from_pretrained(model_id)
pipeline.load_lora_weights(adapter_id)

prompt = "The image is a black and white illustration of a man with a long beard and a long white beard, sitting on the ground with his arms raised above his head. He appears to be in a meditative pose, with his eyes closed and a peaceful expression on his face. Behind him, there are two other men, one lying on their backs and the other lying down. The background shows a mountain landscape with a cloudy sky. The text above the illustration reads 'Let the day perish where I was born. And they sat down with him upon the ground seven days & seven nights and none spoke a word unto him for they saw that his grief was very great.' The illustration is framed by a decorative border."


pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu')
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(1641421826),
    width=1024,
    height=1024,
    guidance_scale=7.5,
).images[0]
image.save("output.png", format="PNG")