metadata

license: other
base_model: diffusers/stable-diffusion-xl-base-0.9
tags:
  - stable-diffusion-xl
  - stable-diffusion-xl-diffusers
  - stable-diffusion
  - text-to-image
  - diffusers
  - di.ffusion.ai
inference: true
extra_gated_prompt: >-
  Copyright (c) Stability AI Ltd. and Source Code Bulgaria Ltd.

  This License Agreement (as may be amended in accordance with this License
  Agreement, “License”), between you, or your employer or other entity (if you
  are entering into this agreement on behalf of your employer or other entity)
  (“Licensee” or “you”) and Stability AI Ltd. (“Stability AI” or “we”) and
  Source Code Bulgaria Ltd. ("Source Code Bulgaria" or "we") applies to your use
  of any computer program, algorithm, source code, object code, software,
  models, or model weights that is made available by Stability AI and Source
  Code Bulgaria under this License (“Software”) and any specifications, manuals,
  documentation, and other written information provided by Stability AI and
  Source Code Bulgaria related to the Software (“Documentation”).

  By using the Software, you agree to the terms of this License. If you do not
  agree to this License, then you do not have any rights to use the Software or
  Documentation (collectively, the “Software Products”), and you must
  immediately cease using the Software Products. If you are agreeing to be bound
  by the terms of this License on behalf of your employer or other entity, you
  represent and warrant to Stability AI and Source Code Bulgaria that you have
  full legal authority to bind your employer or such entity to this License. If
  you do not have the requisite authority, you may not accept the License or
  access the Software Products on behalf of your employer or other entity.

  This Model is a derivative work based on the SDXL 0.9 software provided by
  Stability AI Ltd under the SDXL Research License,  Copyright (c) Stability AI
  Ltd. All Rights Reserved. Use of the Model must adhere to the terms outlined
  in the original SDXL Research License, which can be found at
  https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9/blob/main/LICENSE.md

  If you have any questions or seek permissions beyond those granted in this
  License, please reach out to di@ffusion.ai


  1. LICENSE GRANT

  a. Subject to your compliance with the Documentation and Sections 2, 3, and 5,
  Stability AI grants you a non-exclusive, worldwide, non-transferable,
  non-sublicensable, revocable, royalty free and limited license under Stability
  AI’s copyright interests to use, reproduce, and create derivative works of the
  Software solely for your non-commercial research purposes. The foregoing
  license is personal to you, and you may not assign, sublicense, distribute,
  publish, host, or otherwise make available this Software, derivative works of
  the Software, models or model weights associated with the Software, this
  License, or any other rights or obligations under this License without
  Stability AI’s prior written consent; any such assignment or sublicense
  without Stability AI’s prior written consent will be void and will
  automatically and immediately terminate this License.  For sake of clarity,
  this License does not grant to you the right or ability to extend any license
  to the Software, derivative works of the Software, or associated models or
  model weights to a non-Licensee, nor does this License permit you to create a
  new Licensee, such as by making available a copy of this License.  If you
  would like rights not granted by this License, you may seek permission by
  sending an email to legal@stability.ai.

  b. You may make a reasonable number of copies of the Documentation solely for
  your use in connection with the license to the Software granted above.

  c. The grant of rights expressly set forth in this Section 1 (License Grant)
  are the complete grant of rights to you in the Software Products, and no other
  licenses are granted, whether by waiver, estoppel, implication, equity or
  otherwise. Stability AI and its licensors reserve all rights not expressly
  granted by this License.


  2. RESTRICTIONS

  You will not, and will not permit, assist or cause any third party to:

  a. use, modify, copy, reproduce, create derivative works of, or distribute the
  Software Products (or any derivative works thereof, works incorporating the
  Software Products, or any data produced by the Software), in whole or in part,
  for (i) any commercial or production purposes, (ii) military purposes or in
  the service of nuclear technology, (iii) purposes of surveillance, including
  any research or development relating to surveillance, (iv) biometric
  processing, (v) in any manner that infringes, misappropriates, or otherwise
  violates any third-party rights, or (vi) in any manner that violates any
  applicable law and violating any privacy or security laws, rules, regulations,
  directives, or governmental requirements (including the General Data Privacy
  Regulation (Regulation (EU) 2016/679), the California Consumer Privacy Act,
  and any and all laws governing the processing of biometric information), as
  well as all amendments and successor laws to any of the foregoing;

  b. alter or remove copyright and other proprietary notices which appear on or
  in the Software Products;

  c. utilize any equipment, device, software, or other means to circumvent or
  remove any security or protection used by Stability AI in connection with the
  Software, or to circumvent or remove any usage restrictions, or to enable
  functionality disabled by Stability AI; or

  d. offer or impose any terms on the Software Products that alter, restrict, or
  are inconsistent with the terms of this License.

  e. 1) violate any applicable U.S. and non-U.S. export control and trade
  sanctions laws (“Export Laws”); 2) directly or indirectly export, re-export,
  provide, or otherwise transfer Software Products: (a) to any individual,
  entity, or country prohibited by Export Laws; (b) to anyone on U.S. or
  non-U.S. government restricted parties lists; or (c) for any purpose
  prohibited by Export Laws, including nuclear, chemical or biological weapons,
  or missile technology applications; 3) use or download Software Products if
  you or they are: (a) located in a comprehensively sanctioned jurisdiction, (b)
  currently listed on any U.S. or non-U.S. restricted parties list, or (c) for
  any purpose prohibited by Export Laws; and (4) will not disguise your location
  through IP proxying or other methods.


  3. ATTRIBUTION

  Together with any copies of the Software Products (as well as derivative works
  thereof or works incorporating the Software Products) that you distribute, you
  must provide (i) a copy of this License, and (ii) the following attribution

  notice: “SDXL 0.9 is licensed under the SDXL Research License, Copyright (c)
  Stability AI Ltd. All Rights Reserved. FFusionXL-09-SDXL is developed and
  maintained by FFusion.AI, a division of Source Code Bulgaria Ltd. All Rights
  Reserved.”


  4. DISCLAIMERS

  THE SOFTWARE PRODUCTS ARE PROVIDED “AS IS” AND “WITH ALL FAULTS” WITH NO
  WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. STABILITY AI EXPRESSLY DISCLAIMS ALL
  REPRESENTATIONS AND WARRANTIES, EXPRESS OR IMPLIED, WHETHER BY STATUTE,
  CUSTOM, USAGE OR OTHERWISE AS TO ANY MATTERS RELATED TO THE SOFTWARE PRODUCTS,
  INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE, TITLE, SATISFACTORY QUALITY, OR
  NON-INFRINGEMENT. STABILITY AI MAKES NO WARRANTIES OR REPRESENTATIONS THAT THE
  SOFTWARE PRODUCTS WILL BE ERROR FREE OR FREE OF VIRUSES OR OTHER HARMFUL
  COMPONENTS, OR PRODUCE ANY PARTICULAR RESULTS.


  5. LIMITATION OF LIABILITY

  TO THE FULLEST EXTENT PERMITTED BY LAW, IN NO EVENT WILL STABILITY AI BE
  LIABLE TO YOU (A) UNDER ANY THEORY OF LIABILITY, WHETHER BASED IN CONTRACT,
  TORT, NEGLIGENCE, STRICT LIABILITY, WARRANTY, OR OTHERWISE UNDER THIS LICENSE,
  OR (B) FOR ANY INDIRECT, CONSEQUENTIAL, EXEMPLARY, INCIDENTAL, PUNITIVE OR
  SPECIAL DAMAGES OR LOST PROFITS, EVEN IF STABILITY AI HAS BEEN ADVISED OF THE
  POSSIBILITY OF SUCH DAMAGES. THE SOFTWARE PRODUCTS, THEIR CONSTITUENT
  COMPONENTS, AND ANY OUTPUT (COLLECTIVELY, “SOFTWARE MATERIALS”) ARE NOT
  DESIGNED OR INTENDED FOR USE IN ANY APPLICATION OR SITUATION WHERE FAILURE OR
  FAULT OF THE SOFTWARE MATERIALS COULD REASONABLY BE ANTICIPATED TO LEAD TO
  SERIOUS INJURY OF ANY PERSON, INCLUDING POTENTIAL DISCRIMINATION OR VIOLATION
  OF AN INDIVIDUAL’S PRIVACY RIGHTS, OR TO SEVERE PHYSICAL, PROPERTY, OR
  ENVIRONMENTAL DAMAGE (EACH, A “HIGH-RISK USE”). IF YOU ELECT TO USE ANY OF THE
  SOFTWARE MATERIALS FOR A HIGH-RISK USE, YOU DO SO AT YOUR OWN RISK. YOU AGREE
  TO DESIGN AND IMPLEMENT APPROPRIATE DECISION-MAKING AND RISK-MITIGATION
  PROCEDURES AND POLICIES IN CONNECTION WITH A HIGH-RISK USE SUCH THAT EVEN IF
  THERE IS A FAILURE OR FAULT IN ANY OF THE SOFTWARE MATERIALS, THE SAFETY OF
  PERSONS OR PROPERTY AFFECTED BY THE ACTIVITY STAYS AT A LEVEL THAT IS
  REASONABLE, APPROPRIATE, AND LAWFUL FOR THE FIELD OF THE HIGH-RISK USE.


  6. INDEMNIFICATION

  You will indemnify, defend and hold harmless Stability AI and our subsidiaries
  and affiliates, and each of our respective shareholders, directors, officers,
  employees, agents, successors, and assigns (collectively, the “Stability AI
  Parties”) from and against any losses, liabilities, damages, fines, penalties,
  and expenses (including reasonable attorneys’ fees) incurred by any Stability
  AI Party in connection with any claim, demand, allegation, lawsuit,
  proceeding, or investigation (collectively, “Claims”) arising out of or
  related to: (a) your access to or use of the Software Products (as well as any
  results or data generated from such access or use), including any High-Risk
  Use (defined below); (b) your violation of this License; or (c) your
  violation, misappropriation or infringement of any rights of another
  (including intellectual property or other proprietary rights and privacy
  rights). You will promptly notify the Stability AI Parties of any such Claims,
  and cooperate with Stability AI Parties in defending such Claims. You will
  also grant the Stability AI Parties sole control of the defense or settlement,
  at Stability AI’s sole option, of any Claims. This indemnity is in addition
  to, and not in lieu of, any other indemnities or remedies set forth in a
  written agreement between you and Stability AI or the other Stability AI
  Parties.


  7. TERMINATION; SURVIVAL

  a. This License will automatically terminate upon any breach by you of the
  terms of this License.

  b. We may terminate this License, in whole or in part, at any time upon notice
  (including electronic) to you.

  c. The following sections survive termination of this License: 2
  (Restrictions), 3 (Attribution), 4 (Disclaimers), 5 (Limitation on Liability),
  6 (Indemnification) 7 (Termination; Survival), 8 (Third Party Materials), 9
  (Trademarks), 10 (Applicable Law; Dispute Resolution), and 11 (Miscellaneous).


  8. THIRD PARTY MATERIALS

  The Software Products may contain third-party software or other components
  (including free and open source software) (all of the foregoing, “Third Party
  Materials”), which are subject to the license terms of the respective
  third-party licensors. Your dealings or correspondence with third parties and
  your use of or interaction with any Third Party Materials are solely between
  you and the third party. Stability AI does not control or endorse, and makes
  no representations or warranties regarding, any Third Party Materials, and
  your access to and use of such Third Party Materials are at your own risk.


  9. TRADEMARKS

  Licensee has not been granted any trademark license as part of this License
  and may not use any name or mark associated with Stability AI without the
  prior written permission of Stability AI, except to the extent necessary to
  make the reference required by the “ATTRIBUTION” section of this Agreement.


  10. APPLICABLE LAW; DISPUTE RESOLUTION

  This License will be governed and construed under the laws of the State of
  California without regard to conflicts of law provisions. Any suit or
  proceeding arising out of or relating to this License will be brought in the
  federal or state courts, as applicable, in San Mateo County, California, and
  each party irrevocably submits to the jurisdiction and venue of such courts.


  11. MISCELLANEOUS

  If any provision or part of a provision of this License is unlawful, void or
  unenforceable, that provision or part of the provision is deemed severed from
  this License, and will not affect the validity and enforceability of any
  remaining provisions. The failure of Stability AI to exercise or enforce any
  right or provision of this License will not operate as a waiver of such right
  or provision. This License does not confer any third-party beneficiary rights
  upon any other person or entity. This License, together with the
  Documentation, contains the entire understanding between you and Stability AI
  regarding the subject matter of this License, and supersedes all other written
  or oral agreements and understandings between you and Stability AI regarding
  such subject matter. No change or addition to any provision of this License
  will be binding unless it is in writing and signed by an authorized
  representative of both you and Stability AI.

  12.ADDITIONAL TERMS FOR FFUSION.AI

  In addition to the terms set forth above, the following terms apply
  specifically to the FFusionXL-09-SDXL model developed by FFusion.AI, a
  division of Source Code Bulgaria Ltd:

  a. Any use of the FFusionXL-09-SDXL model must include proper attribution to
  FFusion.AI and Source Code Bulgaria Ltd. in any publication or public work
  that includes results achieved by or data generated by the model.

  b. The FFusionXL-09-SDXL model is provided for non-commercial research
  purposes only. Any commercial use requires a separate license agreement with
  Source Code Bulgaria Ltd.

  c. You may not reverse engineer, decompile, or disassemble the
  FFusionXL-09-SDXL model.

  d. You agree to indemnify and hold harmless Source Code Bulgaria Ltd. and its
  affiliates from any claims, damages, liabilities, costs, losses, and expenses
  (including reasonable attorney's fees) arising out of your use of the
  FFusionXL-09-SDXL model.
extra_gated_heading: FFXL Researcher Preliminary Access License Agreement
extra_gated_description: FFXL-SDXL 0.9 RESEARCH LICENSE AGREEMENT
extra_gated_button_content: Submit application
extra_gated_fields:
  Organization or Personal Alias: text
  Nature of research(optional): text
  Personal researcher link (Civitai, website, github, HF): text
  Further Information (Feel free to provide additional details): text
  I acknowledge the license agreement stated above and pledge to utilize the Software strictly for non-commercial research: checkbox

FFXL Model Card

Model

FFXL based on SDXL consists of a two-step pipeline for latent diffusion: First, we use a base model to generate latents of the desired output size. In the second step, we use a specialized high-resolution model and apply a technique called SDEdit (https://arxiv.org/abs/2108.01073, also known as "img2img") to the latents generated in the first step, using the same prompt.

Model Description

Trained by: FFusion AI
Model type: Diffusion-based text-to-image generative model
License: FFXL Research License
Model Description: This is a trained model based on SDXL that can be used to generate and modify images based on text prompts. It is a Latent Diffusion Model that uses two fixed, pretrained text encoders (OpenCLIP-ViT/G and CLIP-ViT/L).
Resources for more information: SDXL paper on arXiv.

Model Sources

Demo:

🧨 Diffusers

Make sure to upgrade diffusers to >= 0.18.0:

pip install diffusers --upgrade

In addition make sure to install transformers, safetensors, accelerate as well as the invisible watermark:

pip install invisible_watermark transformers accelerate safetensors

You can use the model then as follows

from diffusers import DiffusionPipeline
import torch

pipe = DiffusionPipeline.from_pretrained("FFusion/FFusionXL-09-SDXL", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
pipe.to("cuda")

# if using torch < 2.0
# pipe.enable_xformers_memory_efficient_attention()

prompt = "An astronaut riding a green horse"

images = pipe(prompt=prompt).images[0]

When using torch >= 2.0, you can improve the inference speed by 20-30% with torch.compile. Simple wrap the unet with torch compile before running the pipeline:

pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)

If you are limited by GPU VRAM, you can enable cpu offloading by calling pipe.enable_model_cpu_offload instead of .to("cuda"):

- pipe.to("cuda")
+ pipe.enable_model_cpu_offload()

Uses

Direct Use

The model is intended for research purposes only. Possible research areas and tasks include

Generation of artworks and use in design and other artistic processes.
Applications in educational or creative tools.
Research on generative models.
Safe deployment of models which have the potential to generate harmful content.
Probing and understanding the limitations and biases of generative models.

Excluded uses are described below.

Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

Limitations and Bias

Limitations

The model does not achieve perfect photorealism
The model cannot render legible text
The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
Faces and people in general may not be generated properly.
The autoencoding part of the model is lossy.