FLUX.1 [dev] -- Flumina Server App (FP8 Version)

This repository contains an implementation of the FLUX.1 [dev] FP8 version, which uses float8 numerics instead of bfloat16. This optimization leads to 2x faster performance in inference when compared to previous versions, making it ideal for high-speed, resource-efficient applications on Fireworks AI’s Flumina Server App toolkit.

Getting Started -- Serverless deployment on Fireworks

This FP8 Server App is deployed to Fireworks as-is in a "serverless" deployment, enabling you to leverage its performance boost without needing to manage servers manually.

Grab an API Key from Fireworks and set it in your environment variables:

export API_KEY=YOUR_API_KEY_HERE

Text-to-Image Example Call

curl -X POST 'https://api.fireworks.ai/inference/v1/workflows/accounts/fireworks/models/flux-1-dev-fp8/text_to_image' \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -H "Accept: image/jpeg" \
    -d '{
        "prompt": "Woman laying in the grass",
        "aspect_ratio": "16:9",
        "guidance_scale": 3.5,
        "num_inference_steps": 30,
        "seed": 0
    }' \
    --output output.jpg

Deploying FLUX.1 [dev] to Fireworks On-Demand

FLUX.1 [dev] (bfloat16) is available on Fireworks via on-demand deployments. It can be deployed in a few simple steps:

Prerequisite: Install the Flumina CLI

The Flumina CLI is included with the fireworks-ai Python package. It can be installed with pip like so:

pip install 'fireworks-ai[flumina]>=0.15.7'

Also get an API key from the Fireworks site and set it in the Flumina CLI:

flumina set-api-key YOURAPIKEYHERE

Creating an On-Demand Deployment

flumina deploy can be used to create an on-demand deployment. When invoked with a model name that exists already, it will create a new deployment in your account which has that model:

flumina deploy accounts/fireworks/models/flux-1-dev-fp8

Note that fp8 FLUX models require --accelerator-type H100 to successfully deploy

When successful, the CLI will print out example commands to call your new deployment, for example:

curl -X POST 'https://api.fireworks.ai/inference/v1/workflows/accounts/fireworks/models/flux-1-dev-fp8/text_to_image?deployment=accounts/u-6jamesr6-63834f/deployments/a0dab4ba' \
    -H 'Authorization: Bearer API_KEY' \
    -H "Content-Type: application/json" \
    -d '{
        "prompt": "<value>",
        "aspect_ratio": "16:9",
        "guidance_scale": 3.5,
        "num_inference_steps": 30,
        "seed": 0
    }'

Your deployment can also be administered using the Flumina CLI. Useful commands include:

flumina list deployments to show all of your deployments
flumina get deployment to get details about a specific deployment
flumina delete deployment to delete a deployment

What is Flumina?

Flumina is Fireworks.ai’s new system for hosting Server Apps that allows users to deploy deep learning inference to production in minutes, not weeks.

What does Flumina offer for FLUX models?

Flumina offers the following benefits:

Clear, precise definition of the server-side workload by looking at the server app implementation (you are here)
Extensibility interface, which allows for dynamic loading/dispatching of add-ons server-side. For FLUX:
- ControlNet (Union) adapters
- LoRA adapters
Off-the-shelf support for standing up on-demand capacity for the Server App on Fireworks
- Further, customization of the logic of the deployment by modifying the Server App and deploying the modified version.
Now with support for FP8 numerics, delivering enhanced speed and efficiency for intensive workloads.

Deploying Custom FLUX.1 [dev] FP8 Apps to Fireworks On-demand

Coming soon!