Adaptive Blind All-in-One Image Restoration

Our Adaptive Blind All-in-One Image Restoration (ABAIR) model combines a powerful baseline trained on images with synthetic degradations, low-rank decompositions for task-specific adaptation, and a lightweight estimator to handle complex distortions.

We propose a pre-training strategy based on synthetic degradations, which are parameterized to control both the type and severity of degradations, including rain, blur, noise, haze, and low-light conditions. To establish a strong weight initialization for subsequent fine-tuning, we propose a degradation CutMix method. This approach seamlessly blends multiple degradation types and severity levels within a single image, enhancing the model’s generalization ability. Additionally, we incorporate an auxiliary segmentation head and optimize the model using a Cross-Entropy Loss to perform pixel-wise degradation type estimation. Then, we adapt our baseline model to varying image restoration tasks using independent low-rank adapters (LoRA). Third, we learn to adaptively combine adapters to versatile images via a flexible and lightweight degradation estimator.

Our approach is highly effective at handling known degradations while remaining adaptable to new image restoration tasks, enabled by its ability to learn a disentangled representation for each degradation type. In contrast, current methods require retraining the entire architecture with all degradation types to accommodate a new task, making them computationally expensive and inefficient. By using a robust baseline model and a disentangled representation, our method requires only training a small set of parameters for new tasks, preserving the knowledge acquired from previous degradations.