FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Models
Inbarhub
Abstract
Editing real images using a pre-trained text-to-image (T2I) diffusion/flow model often involves inverting the image into its corresponding noise map. However, inversion by itself is typically insufficient for obtaining satisfactory results, and therefore many methods additionally intervene in the sampling process. Such methods achieve improved results but are not seamlessly transferable between model architectures. Here, we introduce FlowEdit, a text-based editing method for pre-trained T2I flow models, which is inversion-free, optimization-free and model agnostic. Our method constructs an ODE that directly maps between the source and target distributions (corresponding to the source and target text prompts) and achieves a lower transport cost than the inversion approach. This leads to state-of-the-art results, as we illustrate with Stable Diffusion 3 and FLUX. Code and examples are available on the project's webpage.
Community
Editing real images using a pre-trained text-to-image (T2I) diffusion/flow model often involves inverting the image into its corresponding noise map. However, inversion by itself is typically insufficient for obtaining satisfactory results, and therefore many methods additionally intervene in the sampling process. Such methods achieve improved results but are not seamlessly transferable between model architectures. Here, we introduce FlowEdit, a text-based editing method for pre-trained T2I flow models, which is inversion-free, optimization-free and model agnostic. Our method constructs an ODE that directly maps between the source and target distributions (corresponding to the source and target text prompts) and achieves a lower transport cost than the inversion approach. This leads to state-of-the-art results, as we illustrate with Stable Diffusion 3 and FLUX.
Hi! Thanks for the great work!
It is so cool to see folks realizing the potential of vector field of flow models.
I wanted to share our concurrent work on "Steering Rectified Flow Models in the Vector Field for Controlled Image Generation." We also present an inversion-free solution for image editing on Flux. And tackle the linear inverse problems simultaneously in a gradient-free manner.
Abs: https://arxiv.org/abs/2412.00100
Project Page: https://flowchef.github.io/
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Stable Flow: Vital Layers for Training-Free Image Editing (2024)
- SwiftEdit: Lightning Fast Text-Guided Image Editing via One-Step Diffusion (2024)
- Pathways on the Image Manifold: Image Editing via Video Generation (2024)
- Free-Mask: A Novel Paradigm of Integration Between the Segmentation Diffusion Model and Image Editing to Improve Segmentation Ability (2024)
- DiT4Edit: Diffusion Transformer for Image Editing (2024)
- TACO: Training-free Sound Prompted Segmentation via Deep Audio-visual CO-factorization (2024)
- Taming Rectified Flow for Inversion and Editing (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper