apple/coreml-stable-diffusion-mixed-bit-palettization

Core ML Stable Diffusion Mixed-Bit Palettization Resources

This repository contains resources related to the use of mixed-bit palettization techniques for Stable Diffusion. As explained in the original repo, this is an advanced compression technique that applies palettization using a varying number of bits per layer to keep quality above the desired threshold (as measured by the PSNR with respect to the original model). This can achieve strong space savings without resorting to learned quantization tables during fine-tuning.

The repository contains the following resources:

• recipes/. JSON files with results from the analysis phase (Step 1 of the instructions) on three popular Stable Diffusion models:
  • Stable Diffusion 1.5
  • Stable Diffusion 2.1 base
  • Stable Diffusion XL 1.0 base

You can apply any of the available recipes on the appropriate model to experiment and compare results.

• unet-mbp-sdxl-1-base/. Three pre-applied recipes on the UNet of Stable Diffusion XL 1.0 base:

Recipe	Effective bits	UNet size (GB)	Size reduction
recipe_3_41_bit_mixedpalette	3.41	1.1	77%
recipe_4_50_bit_mixedpalette	4.50	1.4	71%
recipe_6_55_bit_mixedpalette	6.55	2.0	58%

• coreml-stable-diffusion-xl-base_mbp_4_50_palettized. A full pipeline of a Core ML version of Stable Diffusion XL 1.0 base, with the recipe_4_50_bit_mixedpalette recipe pre-applied to the UNet. You can download it to use in your own Core ML apps.

• coreml-stable-diffusion-mixed-bit-palettization_original_compiled.zip. An archived version of the same pipeline, for use with Hugging Face demo app and other third party tools.

This repository was prepared by Apple and Hugging Face in July 2023, from experiments conducted using public beta versions of iOS 17.0, iPadOS 17.0 and macOS 14.0.