city96

/

FLUX.1-dev-gguf

like 664

Hi. The tools folder in the main repo here has some notes on how to do it. You'll have to apply a patch and build llama.cpp from source which is somewhat easier to do on linux than windows, but it works on both. LMK if you run into any issues and I can try to help out.

Thank you so much! would that work with prebuilt windows binaries provided at https://github.com/ggerganov/llama.cpp/releases/?

Sadly no because we have to patch in some image model specific things to make it recognize the Flux/etc architecture and to not add things like the tokenizer/prompt template (that we don't use) in these cases.

Alright thanks a lot, this was helpful

so i followed the steps there & im trying to use convert.py but it doesn't seem to recognize the input file. I'm on arch linux.

python /home/ComfyUI-GGUF/tools/convert.py --src /home/flux1-dev.safetensors --dst /home/flux1-dev.gguf

Console output:

usage: convert.py [-h] --src SRC [--dst DST]
convert.py: error: No input provided!

That means the path is invalid as it's failing this check here:

    if not os.path.isfile(args.src):
        parser.error("No input provided!")

I've had it work on linux before so not sure what you could be running into outside, well, an invalid path.