Thank you for this remarkable work! It helps me a lot.

I have been googling for a v-prediction, Zero-SNR SD1.5 for quite a while, because there's no IPAdapter trained for SD2.1 (someone please let me know if there be one). However I got nothing. Coincidently, a recent paper named Collaborative Control for Geometry-Conditioned PBR Image Generation leads me here. And I believe your work deserves more attentions.

Convert to Diffusers format

The only drawback is that you seem to be providing the original SD version of Runway. To convert it to the more commonly used Diffusers version, you can follow these steps:

1. Download the zd_base_v0-9_ema.safetensors
   Put the file in <YOUR_INPUT_DIR>.
2. Use diffusers's script to convert

git clone https://github.com/huggingface/diffusers.git
cd diffusers
python scripts/convert_original_stable_diffusion_to_diffusers.py \
    --checkpoint_path <YOUR_INPUT_DIR>/zd_base_v0-9_ema.safetensors \
    --from_safetensors --to_safetensors --dump_path <YOUR_OUTPUT_DIR> \
    --prediction_type v_prediction