First commit from github repo

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+models/joost_color_naming.mat filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,14 +1,124 @@
----
-title: NamedCurves
-emoji: 🌖
-colorFrom: gray
-colorTo: red
-sdk: gradio
-sdk_version: 5.11.0
-app_file: app.py
-pinned: false
-license: mit
-short_description: '[ECCV 24] NamedCurves: Learned Image Enhancement via Color N'
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+![thumbnail](/assets/thumbnail.png)
+
+This repository is the official implementation of "NamedCurves: Learned Image Enhancement via Color Naming" @ ECCV24.
+
+[![arXiv](https://img.shields.io/badge/ArXiv-Paper-B31B1B)](https://arxiv.org/abs/2407.09892)
+[![web](https://img.shields.io/badge/Project-Page-orange)](https://namedcurves.github.io/)
+
+[David Serrano-Lozano](https://davidserra9.github.io/), [Luis Herranz](http://www.lherranz.org/), [Michael S. Brown](http://www.cse.yorku.ca/~mbrown/) and [Javier Vazquez-Corral](https://www.jvazquez-corral.net/)
+
+## News 🚀
+- [Nov24]  Code update. We release the inference images for MIT5K-UEGAN.
+- [July24] We realease the code and pretrained models of our paper.
+- [July24] Our paper NamedCurves is accepted to ECCV24!
+
+## TODO:
+- torch Dataset object for PPR10K
+- Create notebook
+- Create gradio demo
+
+## Method
+
+We propose NamedCurves, a learning-based image enhancement technique that decomposes the image into a small set of named colors. Our method learns to globally adjust the image for each specific named color via tone curves and then combines the images using and attention-based fusion mechanism to mimic spatial editing. In contrast to other SOTA methods, NamedCurves allows interpretability thanks to computing a set of tone curves for each universal color name. 
+
+![architecture](/assets/architecture-overview.png)
+
+## Data
+
+In this paper we use two datasets: [MIT-Adobe FiveK](https://data.csail.mit.edu/graphics/fivek/) and [PPR10K](https://github.com/csjliang/PPR10K).
+
+### MIT-Adobe FiveK
+
+MIT FiveK dataset consists of 5,000 photographs taken by SLR cameras by a set of different photographers that cover a broad range of scenes, subjects, and lighting conditions. They are all in RAW format. Then, 5 different photography students adjust the tone of the photos. Each of them retouched all the 5,000 photos using Adobe Lightroom.
+
+Following previous works we decided to use just the expert-C redition. To obtain the retouched images, we have to render the RAW files using Adobe Lightroom. Because of this, researchers have created different rendered versions of the dataset. In this paper, we use 3 different versions: DPE, UPE and UEGAN, dubbed after the method that introduced them. Some methods were evaluated in only some of the versions and their code and models are not available, so we considered it was fair to compare our results in the same conditions as they did. Now, we will provide information on the properties of each version and how to obtain them:
+
+The dataset can be downloaded [here](ttps://data.csail.mit.edu/graphics/fivek/). After downloading the images you will need to use Adobe Lightroom to pre-process them according to each version.
+
+- The **DPE** version uses the first 2,250 images of the dataset for training, the following 2,250 for validation and the last 500 for testing. The images are rendered to have the short edge to 512 pixels. Please see the [issue](https://github.com/sjmoran/CURL/issues/20) for detailed instructions.
+
+- The **UPE** version uses the first 4,500 images of the dataset for training and the last 500 for testing. The images are rendered to have the short edge to 512 pixels. Please see the [issue](https://github.com/dvlab-research/DeepUPE/issues/26) for detailed instructions.
+
+- The **UEGAN** version uses the first 4,500 images of the dataset for training and the last 500 for testing. The images are rendered to have the short edge to 512 pixels. For downloading the rendered images from [Google Drive](https://drive.google.com/drive/folders/1x-DcqFVoxprzM4KYGl8SUif8sV-57FP3). Please see the [official repository](https://github.com/dvlab-research/DeepUPE) for more information.
+
+### PPR10K
+PPR10K contains 1,681 high-quality RAW portraits photos manually retouched by 3 experts. The dataset can be downloaded from the [official repository](https://github.com/csjliang/PPR10K). We used the 480p images.
+
+## Getting started
+
+### Environment setup
+
+We provide a Conda environment file ```requirements.txt``` with all necessary dependencies, except for PyTorch and Torchvision. Follow the instructions below to set up the environment.
+
+First, create and activate the Conda environment:
+
+```
+conda create -n namedcurves python=3.8
+conda activate namedcurves
+```
+
+Alternatively, you can set up a virtual environment:
+```
+python3 -m venv venv
+source venv/bin/activate
+```
+
+Next, install PyTorch and Torchvision with the appropriate versions based on your CUDA and driver dependencies. Visit the [Pytorch Official Page](https://pytorch.org/get-started/previous-versions/) for specific installation commands. For example:
+
+```
+pip install torch==1.12.0+cu113 torchvision==0.13.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
+```
+
+Once PyTorch is installed, you can install the remaining dependencies from the ```requirements.txt``` file:
+
+```
+pip install -r requirements.txt
+```
+
+Alternatively, you can manually install the required packages:
+```
+pip install omegaconf matplotlib scipy scikit-image lpips torchmetrics
+```
+
+### Results
+
+We provide our results for the MIT5K dataset in the following format: aXXXX_Y_Z.png, where XXXX is the 4-digit file ID, Y is the PSNR value, and Z is the $\Delta E2000$ color difference of the image. All numeric values are rounded to two decimal places.
+
+|          | PSNR    | SSIM    | $\Delta E2000$ | Images  |
+| :-------- | :------: | :-------: | :--------------: | :-------: |
+| MIT5K    | 25.59   | 0.936   | 6.07           | [Link](https://cvcuab-my.sharepoint.com/:f:/g/personal/dserrano_cvc_uab_cat/EijObxqdogJHpNufwKKZE4ABI78-4iQnO78V2mHkzfs07A?e=tVTWAq)
+
+### Pre-trained models
+
+Create and store the pre-trained models in a folder inside the repository.
+
+```
+cd namedcurves
+mkdir pretrained
+```
+
+The weights can be found [here](https://github.com/davidserra9/namedcurves/releases/tag/v1.0). Alternatively, you can run:
+
+```
+cd namedcurves
+bash scripts/download_checkpoints.sh
+```
+
+
+## Inference
+
+The following command takes an image file or a folder with images and saves the results in the specified directory.
+
+```
+python test.py --input_path assets/a4957-input.png --output_path output/ --config_path configs/mit5k_upe_config.yaml --model_path pretrained/mit5k_uegan_psnr_25.59.pth 
+```
+
+## Training
+
+Modify the configurations of the ```configs``` folders and run the following command:
+
+```
+python train.py --config configs/mit5k_upe_config.yaml
+```
+
+

app.py

@@ -0,0 +1,470 @@
+import argparse
+from omegaconf import OmegaConf
+import gradio as gr
+from PIL import Image
+import os
+import torch
+import numpy as np
+import io
+import yaml
+from huggingface_hub import hf_hub_download
+import matplotlib.pyplot as plt
+#from gradio_imageslider import ImageSlider
+
+## local code
+from models.interactive_model import NamedCurves
+
+def dict2namespace(config):
+    namespace = argparse.Namespace()
+    for key, value in config.items():
+        if isinstance(value, dict):
+            new_value = dict2namespace(value)
+        else:
+            new_value = value
+        setattr(namespace, key, new_value)
+    return namespace
+
+def get_named_curves(control_points):
+    linspace = torch.linspace(0, 1, steps=101).unsqueeze(0).unsqueeze(2).repeat(1, 3, 1, 1).to(device)
+    outspace = model.bcpe.apply_cubic_bezier(linspace, control_points)
+
+    fig = plt.figure()
+    plt.plot(linspace[0, 0, :, 0].cpu().numpy(),outspace[0, 0, :, 0].cpu().numpy(), 'r')
+    plt.plot(linspace[0, 1, :, 0].cpu().numpy(), outspace[0, 1, :, 0].cpu().numpy(), 'g')
+    plt.plot(linspace[0, 2, :, 0].cpu().numpy(), outspace[0, 2, :, 0].cpu().numpy(), 'b')
+
+    plt.scatter(control_points[0, 0, :, 1].cpu().numpy(), control_points[0, 0, :, 0].cpu().numpy(), c='r', marker='x')
+    plt.scatter(control_points[0, 1, :, 1].cpu().numpy(), control_points[0, 1, :, 0].cpu().numpy(), c='g', marker='x')
+    plt.scatter(control_points[0, 2, :, 1].cpu().numpy(), control_points[0, 2, :, 0].cpu().numpy(), c='b', marker='x')
+
+    plt.xlim(0, 1)
+    plt.ylim(0, 1)
+    plt.grid()
+
+    img_buf = io.BytesIO()
+    plt.savefig(img_buf, format='png', bbox_inches='tight', dpi=300)
+    plt.close(fig)
+    return Image.open(img_buf)
+
+hf_hub_download(repo_id="davidserra9/NamedCurves", filename="mit5k_uegan_psnr_25.59.pth", local_dir="./")
+
+CONFIG = "configs/mit5k_upe_config.yaml"
+model_pt = "mit5k_uegan_psnr_25.59.pth"
+
+# parse config file
+config = OmegaConf.load(CONFIG)
+
+config = dict2namespace(config)
+
+device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
+model = NamedCurves(config.model).to(device)
+model.load_state_dict(torch.load(model_pt)["model_state_dict"])
+
+def load_img(filename, norm=True,):
+    img = np.array(Image.open(filename).convert("RGB"))
+    if norm:
+        img = img / 255.
+        img = img.astype(np.float32)
+    return img
+
+def process_img(image):
+    img = np.array(image)
+    img = img / 255.
+    img = img.astype(np.float32)
+    y = torch.tensor(img).permute(2,0,1).unsqueeze(0).to(device)
+
+    with torch.no_grad():
+        enhanced_img, control_points = model(y, return_curves=True)
+        
+        img_curves = [get_named_curves(control_points_i) for control_points_i in control_points]
+        
+    enhanced_img = enhanced_img.squeeze().permute(1,2,0).clamp_(0, 1).cpu().detach().numpy()
+    enhanced_img = np.clip(enhanced_img, 0. , 1.)
+
+    enhanced_img = (enhanced_img * 255.0).round().astype(np.uint8)  # float32 to uint8
+    oby_points = control_points[0][0, :, :, 0].detach().cpu().numpy()
+    achr_points = control_points[1][0, :, :, 0].detach().cpu().numpy()
+    pp_points = control_points[2][0, :, :, 0].detach().cpu().numpy()
+    red_points = control_points[3][0, :, :, 0].detach().cpu().numpy()
+    green_points = control_points[4][0, :, :, 0].detach().cpu().numpy()
+    blue_points = control_points[5][0, :, :, 0].detach().cpu().numpy()
+
+    return img_curves[0], oby_points[0, 0], oby_points[1, 0], oby_points[2, 0], oby_points[0, 1], oby_points[1, 1], oby_points[2, 1], oby_points[0, 2], oby_points[1, 2], oby_points[2, 2], oby_points[0, 3], oby_points[1, 3], oby_points[2, 3], oby_points[0, 4], oby_points[1, 4], oby_points[2, 4], oby_points[0, 5], oby_points[1, 5], oby_points[2, 5], oby_points[0, 6], oby_points[1, 6], oby_points[2, 6], oby_points[0, 7], oby_points[1, 7], oby_points[2, 7], oby_points[0, 8], oby_points[1, 8], oby_points[2, 8], oby_points[0, 9], oby_points[1, 9], oby_points[2, 9], oby_points[0, 10], oby_points[1, 10], oby_points[2, 10], img_curves[1], achr_points[0, 0], achr_points[1, 0], achr_points[2, 0], achr_points[0, 1], achr_points[1, 1], achr_points[2, 1], achr_points[0, 2], achr_points[1, 2], achr_points[2, 2], achr_points[0, 3], achr_points[1, 3], achr_points[2, 3], achr_points[0, 4], achr_points[1, 4], achr_points[2, 4], achr_points[0, 5], achr_points[1, 5], achr_points[2, 5], achr_points[0, 6], achr_points[1, 6], achr_points[2, 6], achr_points[0, 7], achr_points[1, 7], achr_points[2, 7], achr_points[0, 8], achr_points[1, 8], achr_points[2, 8], achr_points[0, 9], achr_points[1, 9], achr_points[2, 9], achr_points[0, 10], achr_points[1, 10], achr_points[2, 10], img_curves[2], pp_points[0, 0], pp_points[1, 0], pp_points[2, 0], pp_points[0, 1], pp_points[1, 1], pp_points[2, 1], pp_points[0, 2], pp_points[1, 2], pp_points[2, 2], pp_points[0, 3], pp_points[1, 3], pp_points[2, 3], pp_points[0, 4], pp_points[1, 4], pp_points[2, 4], pp_points[0, 5], pp_points[1, 5], pp_points[2, 5], pp_points[0, 6], pp_points[1, 6], pp_points[2, 6], pp_points[0, 7], pp_points[1, 7], pp_points[2, 7], pp_points[0, 8], pp_points[1, 8], pp_points[2, 8], pp_points[0, 9], pp_points[1, 9], pp_points[2, 9], pp_points[0, 10], pp_points[1, 10], pp_points[2, 10], img_curves[3], red_points[0, 0], red_points[1, 0], red_points[2, 0], red_points[0, 1], red_points[1, 1], red_points[2, 1], red_points[0, 2], red_points[1, 2], red_points[2, 2], red_points[0, 3], red_points[1, 3], red_points[2, 3], red_points[0, 4], red_points[1, 4], red_points[2, 4], red_points[0, 5], red_points[1, 5], red_points[2, 5], red_points[0, 6], red_points[1, 6], red_points[2, 6], red_points[0, 7], red_points[1, 7], red_points[2, 7], red_points[0, 8], red_points[1, 8], red_points[2, 8], red_points[0, 9], red_points[1, 9], red_points[2, 9], red_points[0, 10], red_points[1, 10], red_points[2, 10], img_curves[4], green_points[0, 0], green_points[1, 0], green_points[2, 0], green_points[0, 1], green_points[1, 1], green_points[2, 1], green_points[0, 2], green_points[1, 2], green_points[2, 2], green_points[0, 3], green_points[1, 3], green_points[2, 3], green_points[0, 4], green_points[1, 4], green_points[2, 4], green_points[0, 5], green_points[1, 5], green_points[2, 5], green_points[0, 6], green_points[1, 6], green_points[2, 6], green_points[0, 7], green_points[1, 7], green_points[2, 7], green_points[0, 8], green_points[1, 8], green_points[2, 8], green_points[0, 9], green_points[1, 9], green_points[2, 9], green_points[0, 10], green_points[1, 10], green_points[2, 10], img_curves[5], blue_points[0, 0], blue_points[1, 0], blue_points[2, 0], blue_points[0, 1], blue_points[1, 1], blue_points[2, 1], blue_points[0, 2], blue_points[1, 2], blue_points[2, 2], blue_points[0, 3], blue_points[1, 3], blue_points[2, 3], blue_points[0, 4], blue_points[1, 4], blue_points[2, 4], blue_points[0, 5], blue_points[1, 5], blue_points[2, 5], blue_points[0, 6], blue_points[1, 6], blue_points[2, 6], blue_points[0, 7], blue_points[1, 7], blue_points[2, 7], blue_points[0, 8], blue_points[1, 8], blue_points[2, 8], blue_points[0, 9], blue_points[1, 9], blue_points[2, 9], blue_points[0, 10], blue_points[1, 10], blue_points[2, 10], Image.fromarray(enhanced_img)
+
+def process_img_with_sliders(image, oby_red_p0, oby_green_p0, oby_blue_p0, oby_red_p1, oby_green_p1, oby_blue_p1, oby_red_p2, oby_green_p2, oby_blue_p2, oby_red_p3, oby_green_p3, oby_blue_p3, oby_red_p4, oby_green_p4, oby_blue_p4, oby_red_p5, oby_green_p5, oby_blue_p5, oby_red_p6, oby_green_p6, oby_blue_p6, oby_red_p7, oby_green_p7, oby_blue_p7, oby_red_p8, oby_green_p8, oby_blue_p8, oby_red_p9, oby_green_p9, oby_blue_p9, oby_red_p10, oby_green_p10, oby_blue_p10,
+                 achro_red_p0, achro_green_p0, achro_blue_p0, achro_red_p1, achro_green_p1, achro_blue_p1, achro_red_p2, achro_green_p2, achro_blue_p2, achro_red_p3, achro_green_p3, achro_blue_p3, achro_red_p4, achro_green_p4, achro_blue_p4, achro_red_p5, achro_green_p5, achro_blue_p5, achro_red_p6, achro_green_p6, achro_blue_p6, achro_red_p7, achro_green_p7, achro_blue_p7, achro_red_p8, achro_green_p8, achro_blue_p8, achro_red_p9, achro_green_p9, achro_blue_p9, achro_red_p10, achro_green_p10, achro_blue_p10,
+                 pp_red_p0, pp_green_p0, pp_blue_p0, pp_red_p1, pp_green_p1, pp_blue_p1, pp_red_p2, pp_green_p2, pp_blue_p2, pp_red_p3, pp_green_p3, pp_blue_p3, pp_red_p4, pp_green_p4, pp_blue_p4, pp_red_p5, pp_green_p5, pp_blue_p5, pp_red_p6, pp_green_p6, pp_blue_p6, pp_red_p7, pp_green_p7, pp_blue_p7, pp_red_p8, pp_green_p8, pp_blue_p8, pp_red_p9, pp_green_p9, pp_blue_p9, pp_red_p10, pp_green_p10, pp_blue_p10,
+                 red_red_p0, red_green_p0, red_blue_p0, red_red_p1, red_green_p1, red_blue_p1, red_red_p2, red_green_p2, red_blue_p2, red_red_p3, red_green_p3, red_blue_p3, red_red_p4, red_green_p4, red_blue_p4, red_red_p5, red_green_p5, red_blue_p5, red_red_p6, red_green_p6, red_blue_p6, red_red_p7, red_green_p7, red_blue_p7, red_red_p8, red_green_p8, red_blue_p8, red_red_p9, red_green_p9, red_blue_p9, red_red_p10, red_green_p10, red_blue_p10,
+                 green_red_p0, green_green_p0, green_blue_p0, green_red_p1, green_green_p1, green_blue_p1, green_red_p2, green_green_p2, green_blue_p2, green_red_p3, green_green_p3, green_blue_p3, green_red_p4, green_green_p4, green_blue_p4, green_red_p5, green_green_p5, green_blue_p5, green_red_p6, green_green_p6, green_blue_p6, green_red_p7, green_green_p7, green_blue_p7, green_red_p8, green_green_p8, green_blue_p8, green_red_p9, green_green_p9, green_blue_p9, green_red_p10, green_green_p10, green_blue_p10,
+                 blue_red_p0, blue_green_p0, blue_blue_p0, blue_red_p1, blue_green_p1, blue_blue_p1, blue_red_p2, blue_green_p2, blue_blue_p2, blue_red_p3, blue_green_p3, blue_blue_p3, blue_red_p4, blue_green_p4, blue_blue_p4, blue_red_p5, blue_green_p5, blue_blue_p5, blue_red_p6, blue_green_p6, blue_blue_p6, blue_red_p7, blue_green_p7, blue_blue_p7, blue_red_p8, blue_green_p8, blue_blue_p8, blue_red_p9, blue_green_p9, blue_blue_p9, blue_red_p10, blue_green_p10, blue_blue_p10,
+                 ):
+    
+    x = np.linspace(0, 1, 11)
+    oby_r_y = [float(oby_red_p0), float(oby_red_p1), float(oby_red_p2), float(oby_red_p3), float(oby_red_p4), float(oby_red_p5), float(oby_red_p6), float(oby_red_p7), float(oby_red_p8), float(oby_red_p9), float(oby_red_p10)]
+    oby_g_y = [float(oby_green_p0), float(oby_green_p1), float(oby_green_p2), float(oby_green_p3), float(oby_green_p4), float(oby_green_p5), float(oby_green_p6), float(oby_green_p7), float(oby_green_p8), float(oby_green_p9), float(oby_green_p10)]
+    oby_b_y = [float(oby_blue_p0), float(oby_blue_p1), float(oby_blue_p2), float(oby_blue_p3), float(oby_blue_p4), float(oby_blue_p5), float(oby_blue_p6), float(oby_blue_p7), float(oby_blue_p8), float(oby_blue_p9), float(oby_blue_p10)]
+    achro_r_y = [float(achro_red_p0), float(achro_red_p1), float(achro_red_p2), float(achro_red_p3), float(achro_red_p4), float(achro_red_p5), float(achro_red_p6), float(achro_red_p7), float(achro_red_p8), float(achro_red_p9), float(achro_red_p10)]
+    achro_g_y = [float(achro_green_p0), float(achro_green_p1), float(achro_green_p2), float(achro_green_p3), float(achro_green_p4), float(achro_green_p5), float(achro_green_p6), float(achro_green_p7), float(achro_green_p8), float(achro_green_p9), float(achro_green_p10)]
+    achro_b_y = [float(achro_blue_p0), float(achro_blue_p1), float(achro_blue_p2), float(achro_blue_p3), float(achro_blue_p4), float(achro_blue_p5), float(achro_blue_p6), float(achro_blue_p7), float(achro_blue_p8), float(achro_blue_p9), float(achro_blue_p10)]
+    pp_r_y = [float(pp_red_p0), float(pp_red_p1), float(pp_red_p2), float(pp_red_p3), float(pp_red_p4), float(pp_red_p5), float(pp_red_p6), float(pp_red_p7), float(pp_red_p8), float(pp_red_p9), float(pp_red_p10)]
+    pp_g_y = [float(pp_green_p0), float(pp_green_p1), float(pp_green_p2), float(pp_green_p3), float(pp_green_p4), float(pp_green_p5), float(pp_green_p6), float(pp_green_p7), float(pp_green_p8), float(pp_green_p9), float(pp_green_p10)]
+    pp_b_y = [float(pp_blue_p0), float(pp_blue_p1), float(pp_blue_p2), float(pp_blue_p3), float(pp_blue_p4), float(pp_blue_p5), float(pp_blue_p6), float(pp_blue_p7), float(pp_blue_p8), float(pp_blue_p9), float(pp_blue_p10)]
+    red_r_y = [float(red_red_p0), float(red_red_p1), float(red_red_p2), float(red_red_p3), float(red_red_p4), float(red_red_p5), float(red_red_p6), float(red_red_p7), float(red_red_p8), float(red_red_p9), float(red_red_p10)]
+    red_g_y = [float(red_green_p0), float(red_green_p1), float(red_green_p2), float(red_green_p3), float(red_green_p4), float(red_green_p5), float(red_green_p6), float(red_green_p7), float(red_green_p8), float(red_green_p9), float(red_green_p10)]
+    red_b_y = [float(red_blue_p0), float(red_blue_p1), float(red_blue_p2), float(red_blue_p3), float(red_blue_p4), float(red_blue_p5), float(red_blue_p6), float(red_blue_p7), float(red_blue_p8), float(red_blue_p9), float(red_blue_p10)]
+    green_r_y = [float(green_red_p0), float(green_red_p1), float(green_red_p2), float(green_red_p3), float(green_red_p4), float(green_red_p5), float(green_red_p6), float(green_red_p7), float(green_red_p8), float(green_red_p9), float(green_red_p10)]
+    green_g_y = [float(green_green_p0), float(green_green_p1), float(green_green_p2), float(green_green_p3), float(green_green_p4), float(green_green_p5), float(green_green_p6), float(green_green_p7), float(green_green_p8), float(green_green_p9), float(green_green_p10)]
+    green_b_y = [float(green_blue_p0), float(green_blue_p1), float(green_blue_p2), float(green_blue_p3), float(green_blue_p4), float(green_blue_p5), float(green_blue_p6), float(green_blue_p7), float(green_blue_p8), float(green_blue_p9), float(green_blue_p10)]
+    blue_r_y = [float(blue_red_p0), float(blue_red_p1), float(blue_red_p2), float(blue_red_p3), float(blue_red_p4), float(blue_red_p5), float(blue_red_p6), float(blue_red_p7), float(blue_red_p8), float(blue_red_p9), float(blue_red_p10)]
+    blue_g_y = [float(blue_green_p0), float(blue_green_p1), float(blue_green_p2), float(blue_green_p3), float(blue_green_p4), float(blue_green_p5), float(blue_green_p6), float(blue_green_p7), float(blue_green_p8), float(blue_green_p9), float(blue_green_p10)]
+    blue_b_y = [float(blue_blue_p0), float(blue_blue_p1), float(blue_blue_p2), float(blue_blue_p3), float(blue_blue_p4), float(blue_blue_p5), float(blue_blue_p6), float(blue_blue_p7), float(blue_blue_p8), float(blue_blue_p9), float(blue_blue_p10)]
+
+    oby_y = torch.concatenate([torch.tensor(np.array([oby_r_y, x]).T).unsqueeze(0), torch.tensor(np.array([oby_g_y, x]).T).unsqueeze(0), torch.tensor(np.array([oby_b_y, x]).T).unsqueeze(0)], dim=0).unsqueeze(0).to(device)
+    achro_y = torch.concatenate([torch.tensor(np.array([achro_r_y, x]).T).unsqueeze(0), torch.tensor(np.array([achro_g_y, x]).T).unsqueeze(0), torch.tensor(np.array([achro_b_y, x]).T).unsqueeze(0)], dim=0).unsqueeze(0).to(device)
+    pp_y = torch.concatenate([torch.tensor(np.array([pp_r_y, x]).T).unsqueeze(0), torch.tensor(np.array([pp_g_y, x]).T).unsqueeze(0), torch.tensor(np.array([pp_b_y, x]).T).unsqueeze(0)], dim=0).unsqueeze(0).to(device)
+    red_y = torch.concatenate([torch.tensor(np.array([red_r_y, x]).T).unsqueeze(0), torch.tensor(np.array([red_g_y, x]).T).unsqueeze(0), torch.tensor(np.array([red_b_y, x]).T).unsqueeze(0)], dim=0).unsqueeze(0).to(device)
+    green_y = torch.concatenate([torch.tensor(np.array([green_r_y, x]).T).unsqueeze(0), torch.tensor(np.array([green_g_y, x]).T).unsqueeze(0), torch.tensor(np.array([green_b_y, x]).T).unsqueeze(0)], dim=0).unsqueeze(0).to(device)
+    blue_y = torch.concatenate([torch.tensor(np.array([blue_r_y, x]).T).unsqueeze(0), torch.tensor(np.array([blue_g_y, x]).T).unsqueeze(0), torch.tensor(np.array([blue_b_y, x]).T).unsqueeze(0)], dim=0).unsqueeze(0).to(device)
+
+    control_points = [oby_y, achro_y, pp_y, red_y, green_y, blue_y]
+
+    img = np.array(image)
+    img = img / 255.
+    img = img.astype(np.float32)
+    y = torch.tensor(img).permute(2,0,1).unsqueeze(0).to(device)
+
+    with torch.no_grad():
+        enhanced_img, control_points = model(y, return_curves=True, control_points=control_points)
+        
+        img_curves = [get_named_curves(control_points_i) for control_points_i in control_points]
+        
+    enhanced_img = enhanced_img.squeeze().permute(1,2,0).clamp_(0, 1).cpu().detach().numpy()
+    enhanced_img = np.clip(enhanced_img, 0. , 1.)
+
+    enhanced_img = (enhanced_img * 255.0).round().astype(np.uint8)  # float32 to uint8
+    oby_points = control_points[0][0, :, :, 0].detach().cpu().numpy()
+    achr_points = control_points[1][0, :, :, 0].detach().cpu().numpy()
+    pp_points = control_points[2][0, :, :, 0].detach().cpu().numpy()
+    red_points = control_points[3][0, :, :, 0].detach().cpu().numpy()
+    green_points = control_points[4][0, :, :, 0].detach().cpu().numpy()
+    blue_points = control_points[5][0, :, :, 0].detach().cpu().numpy()
+
+    return img_curves[0], oby_points[0, 0], oby_points[1, 0], oby_points[2, 0], oby_points[0, 1], oby_points[1, 1], oby_points[2, 1], oby_points[0, 2], oby_points[1, 2], oby_points[2, 2], oby_points[0, 3], oby_points[1, 3], oby_points[2, 3], oby_points[0, 4], oby_points[1, 4], oby_points[2, 4], oby_points[0, 5], oby_points[1, 5], oby_points[2, 5], oby_points[0, 6], oby_points[1, 6], oby_points[2, 6], oby_points[0, 7], oby_points[1, 7], oby_points[2, 7], oby_points[0, 8], oby_points[1, 8], oby_points[2, 8], oby_points[0, 9], oby_points[1, 9], oby_points[2, 9], oby_points[0, 10], oby_points[1, 10], oby_points[2, 10], img_curves[1], achr_points[0, 0], achr_points[1, 0], achr_points[2, 0], achr_points[0, 1], achr_points[1, 1], achr_points[2, 1], achr_points[0, 2], achr_points[1, 2], achr_points[2, 2], achr_points[0, 3], achr_points[1, 3], achr_points[2, 3], achr_points[0, 4], achr_points[1, 4], achr_points[2, 4], achr_points[0, 5], achr_points[1, 5], achr_points[2, 5], achr_points[0, 6], achr_points[1, 6], achr_points[2, 6], achr_points[0, 7], achr_points[1, 7], achr_points[2, 7], achr_points[0, 8], achr_points[1, 8], achr_points[2, 8], achr_points[0, 9], achr_points[1, 9], achr_points[2, 9], achr_points[0, 10], achr_points[1, 10], achr_points[2, 10], img_curves[2], pp_points[0, 0], pp_points[1, 0], pp_points[2, 0], pp_points[0, 1], pp_points[1, 1], pp_points[2, 1], pp_points[0, 2], pp_points[1, 2], pp_points[2, 2], pp_points[0, 3], pp_points[1, 3], pp_points[2, 3], pp_points[0, 4], pp_points[1, 4], pp_points[2, 4], pp_points[0, 5], pp_points[1, 5], pp_points[2, 5], pp_points[0, 6], pp_points[1, 6], pp_points[2, 6], pp_points[0, 7], pp_points[1, 7], pp_points[2, 7], pp_points[0, 8], pp_points[1, 8], pp_points[2, 8], pp_points[0, 9], pp_points[1, 9], pp_points[2, 9], pp_points[0, 10], pp_points[1, 10], pp_points[2, 10], img_curves[3], red_points[0, 0], red_points[1, 0], red_points[2, 0], red_points[0, 1], red_points[1, 1], red_points[2, 1], red_points[0, 2], red_points[1, 2], red_points[2, 2], red_points[0, 3], red_points[1, 3], red_points[2, 3], red_points[0, 4], red_points[1, 4], red_points[2, 4], red_points[0, 5], red_points[1, 5], red_points[2, 5], red_points[0, 6], red_points[1, 6], red_points[2, 6], red_points[0, 7], red_points[1, 7], red_points[2, 7], red_points[0, 8], red_points[1, 8], red_points[2, 8], red_points[0, 9], red_points[1, 9], red_points[2, 9], red_points[0, 10], red_points[1, 10], red_points[2, 10], img_curves[4], green_points[0, 0], green_points[1, 0], green_points[2, 0], green_points[0, 1], green_points[1, 1], green_points[2, 1], green_points[0, 2], green_points[1, 2], green_points[2, 2], green_points[0, 3], green_points[1, 3], green_points[2, 3], green_points[0, 4], green_points[1, 4], green_points[2, 4], green_points[0, 5], green_points[1, 5], green_points[2, 5], green_points[0, 6], green_points[1, 6], green_points[2, 6], green_points[0, 7], green_points[1, 7], green_points[2, 7], green_points[0, 8], green_points[1, 8], green_points[2, 8], green_points[0, 9], green_points[1, 9], green_points[2, 9], green_points[0, 10], green_points[1, 10], green_points[2, 10], img_curves[5], blue_points[0, 0], blue_points[1, 0], blue_points[2, 0], blue_points[0, 1], blue_points[1, 1], blue_points[2, 1], blue_points[0, 2], blue_points[1, 2], blue_points[2, 2], blue_points[0, 3], blue_points[1, 3], blue_points[2, 3], blue_points[0, 4], blue_points[1, 4], blue_points[2, 4], blue_points[0, 5], blue_points[1, 5], blue_points[2, 5], blue_points[0, 6], blue_points[1, 6], blue_points[2, 6], blue_points[0, 7], blue_points[1, 7], blue_points[2, 7], blue_points[0, 8], blue_points[1, 8], blue_points[2, 8], blue_points[0, 9], blue_points[1, 9], blue_points[2, 9], blue_points[0, 10], blue_points[1, 10], blue_points[2, 10], Image.fromarray(enhanced_img)
+        
+        
+        
+        
+title = "NamedCurves🌈🤗"
+description = '''
+'''
+
+article = "<p style='text-align: center'><a href='https://github.com/davidserra9/namedcurves' target='_blank'>NamedCurves: Learned Image Enhancement via Color Naming</a></p>"
+
+#### Image,Prompts examples
+#examples = [['assets/a4957-input.png']]
+
+css = """
+    .image-frame img, .image-container img {
+        width: auto;
+        height: auto;
+        max-width: none;
+    }
+"""
+
+with gr.Blocks() as demo:
+    gr.Markdown("""
+    ## [NamedCurves](https://namedcurves.github.io/): Learned Image Enhancement via Color Naming
+    [David Serrano-Lozano](https://davidserra9.github.io/), [Luis Herranz](https://www.lherranz.org/), [Michael S. Brown](https://www.eecs.yorku.ca/~mbrown/), [Javier Vazquez-Corral](https://jvazquezcorral.github.io/)
+    Computer Vision Center, Universitat Autònoma de Barcelona, Universidad Autónoma de Madrid, York University
+    
+    **NamedCurves decomposes an image into a small set of named colors and enhances them using a learned set of tone curves.** By making this decomposition, we improve the interactivity of the model as the user can modify the tone curves assigned to color name to manipulate only a certain color of the image.
+    
+    * Upload an image and click "Run" to automatically enhance the image. 
+    * Then, you can adjust the tone curves for each color name to make the retouched version to your liking. Manipulate the sliders corresponding to the control points that define the tone curves. Note that for simplicity, we show the intensity values of the control points instead of the RGB values of each control point.
+                    
+    """)
+    with gr.Row():
+        with gr.Column():
+            image = gr.Image(type="pil", label="Input")
+            run_btn = gr.Button("Run")
+            examples = gr.Examples(['assets/a4957-input.png', 'assets/a4996-input.png', 'assets/a4998-input.png', 'assets/a5000-input.png',
+                                    'assets/a4986-input.png', 'assets/a4988-input.png', 'assets/a4990-input.png', 'assets/a4993-input.png'], inputs=[image])
+
+        with gr.Tabs() as input_tabs:
+                with gr.Tab(label="Orange", id=0) as oby_curves_tab:
+                    oby_curves = gr.Image(label="Orange-Brown-Yellow Curves", type="pil")
+                    with gr.Tabs() as channel_tabs:
+                        with gr.Tab(label="R", id=0) as oby_red_curves_tab:
+                            oby_red_p0 = gr.Slider(0, 1, label="R-P0", interactive=True)
+                            oby_red_p1 = gr.Slider(0, 1, label="R-P1", interactive=True)
+                            oby_red_p2 = gr.Slider(0, 1, label="R-P2", interactive=True)
+                            oby_red_p3 = gr.Slider(0, 1, label="R-P3", interactive=True)
+                            oby_red_p4 = gr.Slider(0, 1, label="R-P4", interactive=True)
+                            oby_red_p5 = gr.Slider(0, 1, label="R-P5", interactive=True)
+                            oby_red_p6 = gr.Slider(0, 1, label="R-P6", interactive=True)
+                            oby_red_p7 = gr.Slider(0, 1, label="R-P7", interactive=True)
+                            oby_red_p8 = gr.Slider(0, 1, label="R-P8", interactive=True)
+                            oby_red_p9 = gr.Slider(0, 1, label="R-P9", interactive=True)
+                            oby_red_p10 = gr.Slider(0, 1, label="R-P10", interactive=True)
+                        with gr.Tab(label="G", id=1) as oby_green_curves_tab:
+                            oby_green_p0 = gr.Slider(0, 1, label="G-P0", interactive=True)
+                            oby_green_p1 = gr.Slider(0, 1, label="G-P1", interactive=True)
+                            oby_green_p2 = gr.Slider(0, 1, label="G-P2", interactive=True)
+                            oby_green_p3 = gr.Slider(0, 1, label="G-P3", interactive=True)
+                            oby_green_p4 = gr.Slider(0, 1, label="G-P4", interactive=True)
+                            oby_green_p5 = gr.Slider(0, 1, label="G-P5", interactive=True)
+                            oby_green_p6 = gr.Slider(0, 1, label="G-P6", interactive=True)
+                            oby_green_p7 = gr.Slider(0, 1, label="G-P7", interactive=True)
+                            oby_green_p8 = gr.Slider(0, 1, label="G-P8", interactive=True)
+                            oby_green_p9 = gr.Slider(0, 1, label="G-P9", interactive=True)
+                            oby_green_p10 = gr.Slider(0, 1, label="G-P10", interactive=True)
+                        with gr.Tab(label="B", id=2) as oby_blue_curves_tab:
+                            oby_blue_p0 = gr.Slider(0, 1, label="B-P0", interactive=True)
+                            oby_blue_p1 = gr.Slider(0, 1, label="B-P1", interactive=True)
+                            oby_blue_p2 = gr.Slider(0, 1, label="B-P2", interactive=True)
+                            oby_blue_p3 = gr.Slider(0, 1, label="B-P3", interactive=True)
+                            oby_blue_p4 = gr.Slider(0, 1, label="B-P4", interactive=True)
+                            oby_blue_p5 = gr.Slider(0, 1, label="B-P5", interactive=True)
+                            oby_blue_p6 = gr.Slider(0, 1, label="B-P6", interactive=True)
+                            oby_blue_p7 = gr.Slider(0, 1, label="B-P7", interactive=True)
+                            oby_blue_p8 = gr.Slider(0, 1, label="B-P8", interactive=True)
+                            oby_blue_p9 = gr.Slider(0, 1, label="B-P9", interactive=True)
+                            oby_blue_p10 = gr.Slider(0, 1, label="B-P10", interactive=True)
+
+                with gr.Tab(label="Achr", id=1) as achro_curves_tab:
+                    achro_curves = gr.Image(label="Achromatic Curves", type="pil")
+                    with gr.Tabs() as channel_tabs:
+                        with gr.Tab(label="R", id=0) as achro_red_curves_tab:
+                            achro_red_p0 = gr.Slider(0, 1, label="R-P0", interactive=True)
+                            achro_red_p1 = gr.Slider(0, 1, label="R-P1", interactive=True)
+                            achro_red_p2 = gr.Slider(0, 1, label="R-P2", interactive=True)
+                            achro_red_p3 = gr.Slider(0, 1, label="R-P3", interactive=True)
+                            achro_red_p4 = gr.Slider(0, 1, label="R-P4", interactive=True)
+                            achro_red_p5 = gr.Slider(0, 1, label="R-P5", interactive=True)
+                            achro_red_p6 = gr.Slider(0, 1, label="R-P6", interactive=True)
+                            achro_red_p7 = gr.Slider(0, 1, label="R-P7", interactive=True)
+                            achro_red_p8 = gr.Slider(0, 1, label="R-P8", interactive=True)
+                            achro_red_p9 = gr.Slider(0, 1, label="R-P9", interactive=True)
+                            achro_red_p10 = gr.Slider(0, 1, label="R-P10", interactive=True)
+                        with gr.Tab(label="G", id=1) as achro_green_curves_tab:
+                            achro_green_p0 = gr.Slider(0, 1, label="G-P0", interactive=True)
+                            achro_green_p1 = gr.Slider(0, 1, label="G-P1", interactive=True)
+                            achro_green_p2 = gr.Slider(0, 1, label="G-P2", interactive=True)
+                            achro_green_p3 = gr.Slider(0, 1, label="G-P3", interactive=True)
+                            achro_green_p4 = gr.Slider(0, 1, label="G-P4", interactive=True)
+                            achro_green_p5 = gr.Slider(0, 1, label="G-P5", interactive=True)
+                            achro_green_p6 = gr.Slider(0, 1, label="G-P6", interactive=True)
+                            achro_green_p7 = gr.Slider(0, 1, label="G-P7", interactive=True)
+                            achro_green_p8 = gr.Slider(0, 1, label="G-P8", interactive=True)
+                            achro_green_p9 = gr.Slider(0, 1, label="G-P9", interactive=True)
+                            achro_green_p10 = gr.Slider(0, 1, label="G-P10", interactive=True)
+                        with gr.Tab(label="B", id=2) as achro_blue_curves_tab:
+                            achro_blue_p0 = gr.Slider(0, 1, label="B-P0", interactive=True)
+                            achro_blue_p1 = gr.Slider(0, 1, label="B-P1", interactive=True)
+                            achro_blue_p2 = gr.Slider(0, 1, label="B-P2", interactive=True)
+                            achro_blue_p3 = gr.Slider(0, 1, label="B-P3", interactive=True)
+                            achro_blue_p4 = gr.Slider(0, 1, label="B-P4", interactive=True)
+                            achro_blue_p5 = gr.Slider(0, 1, label="B-P5", interactive=True)
+                            achro_blue_p6 = gr.Slider(0, 1, label="B-P6", interactive=True)
+                            achro_blue_p7 = gr.Slider(0, 1, label="B-P7", interactive=True)
+                            achro_blue_p8 = gr.Slider(0, 1, label="B-P8", interactive=True)
+                            achro_blue_p9 = gr.Slider(0, 1, label="B-P9", interactive=True)
+                            achro_blue_p10 = gr.Slider(0, 1, label="B-P10", interactive=True)
+
+                with gr.Tab(label="Pink", id=2) as pink_purple_curves_tab:
+                    pink_purple_curves = gr.Image(label="Pink-Purple Curves", type="pil")
+                    with gr.Tabs() as channel_tabs:
+                        with gr.Tab(label="R", id=0) as pp_red_curves_tab:
+                            pp_red_p0 = gr.Slider(0, 1, label="R-P0", interactive=True)
+                            pp_red_p1 = gr.Slider(0, 1, label="R-P1", interactive=True)
+                            pp_red_p2 = gr.Slider(0, 1, label="R-P2", interactive=True)
+                            pp_red_p3 = gr.Slider(0, 1, label="R-P3", interactive=True)
+                            pp_red_p4 = gr.Slider(0, 1, label="R-P4", interactive=True)
+                            pp_red_p5 = gr.Slider(0, 1, label="R-P5", interactive=True)
+                            pp_red_p6 = gr.Slider(0, 1, label="R-P6", interactive=True)
+                            pp_red_p7 = gr.Slider(0, 1, label="R-P7", interactive=True)
+                            pp_red_p8 = gr.Slider(0, 1, label="R-P8", interactive=True)
+                            pp_red_p9 = gr.Slider(0, 1, label="R-P9", interactive=True)
+                            pp_red_p10 = gr.Slider(0, 1, label="R-P10", interactive=True)
+                        with gr.Tab(label="G", id=1) as pp_green_curves_tab:
+                            pp_green_p0 = gr.Slider(0, 1, label="G-P0", interactive=True)
+                            pp_green_p1 = gr.Slider(0, 1, label="G-P1", interactive=True)
+                            pp_green_p2 = gr.Slider(0, 1, label="G-P2", interactive=True)
+                            pp_green_p3 = gr.Slider(0, 1, label="G-P3", interactive=True)
+                            pp_green_p4 = gr.Slider(0, 1, label="G-P4", interactive=True)
+                            pp_green_p5 = gr.Slider(0, 1, label="G-P5", interactive=True)
+                            pp_green_p6 = gr.Slider(0, 1, label="G-P6", interactive=True)
+                            pp_green_p7 = gr.Slider(0, 1, label="G-P7", interactive=True)
+                            pp_green_p8 = gr.Slider(0, 1, label="G-P8", interactive=True)
+                            pp_green_p9 = gr.Slider(0, 1, label="G-P9", interactive=True)
+                            pp_green_p10 = gr.Slider(0, 1, label="G-P10", interactive=True)
+                        with gr.Tab(label="B", id=2) as pp_blue_curves_tab:
+                            pp_blue_p0 = gr.Slider(0, 1, label="B-P0", interactive=True)
+                            pp_blue_p1 = gr.Slider(0, 1, label="B-P1", interactive=True)
+                            pp_blue_p2 = gr.Slider(0, 1, label="B-P2", interactive=True)
+                            pp_blue_p3 = gr.Slider(0, 1, label="B-P3", interactive=True)
+                            pp_blue_p4 = gr.Slider(0, 1, label="B-P4", interactive=True)
+                            pp_blue_p5 = gr.Slider(0, 1, label="B-P5", interactive=True)
+                            pp_blue_p6 = gr.Slider(0, 1, label="B-P6", interactive=True)
+                            pp_blue_p7 = gr.Slider(0, 1, label="B-P7", interactive=True)
+                            pp_blue_p8 = gr.Slider(0, 1, label="B-P8", interactive=True)
+                            pp_blue_p9 = gr.Slider(0, 1, label="B-P9", interactive=True)
+                            pp_blue_p10 = gr.Slider(0, 1, label="B-P10", interactive=True)
+
+                with gr.Tab(label="Red", id=3) as red_curves_tab:
+                    red_curves = gr.Image(label="Red Curves", type="pil")
+                    with gr.Tabs() as channel_tabs:
+                        with gr.Tab(label="R", id=0) as red_red_curves_tab:
+                            red_red_p0 = gr.Slider(0, 1, label="R-P0", interactive=True)
+                            red_red_p1 = gr.Slider(0, 1, label="R-P1", interactive=True)
+                            red_red_p2 = gr.Slider(0, 1, label="R-P2", interactive=True)
+                            red_red_p3 = gr.Slider(0, 1, label="R-P3", interactive=True)
+                            red_red_p4 = gr.Slider(0, 1, label="R-P4", interactive=True)
+                            red_red_p5 = gr.Slider(0, 1, label="R-P5", interactive=True)
+                            red_red_p6 = gr.Slider(0, 1, label="R-P6", interactive=True)
+                            red_red_p7 = gr.Slider(0, 1, label="R-P7", interactive=True)
+                            red_red_p8 = gr.Slider(0, 1, label="R-P8", interactive=True)
+                            red_red_p9 = gr.Slider(0, 1, label="R-P9", interactive=True)
+                            red_red_p10 = gr.Slider(0, 1, label="R-P10", interactive=True)
+                        with gr.Tab(label="G", id=1) as red_green_curves_tab:
+                            red_green_p0 = gr.Slider(0, 1, label="G-P0", interactive=True)
+                            red_green_p1 = gr.Slider(0, 1, label="G-P1", interactive=True)
+                            red_green_p2 = gr.Slider(0, 1, label="G-P2", interactive=True)
+                            red_green_p3 = gr.Slider(0, 1, label="G-P3", interactive=True)
+                            red_green_p4 = gr.Slider(0, 1, label="G-P4", interactive=True)
+                            red_green_p5 = gr.Slider(0, 1, label="G-P5", interactive=True)
+                            red_green_p6 = gr.Slider(0, 1, label="G-P6", interactive=True)
+                            red_green_p7 = gr.Slider(0, 1, label="G-P7", interactive=True)
+                            red_green_p8 = gr.Slider(0, 1, label="G-P8", interactive=True)
+                            red_green_p9 = gr.Slider(0, 1, label="G-P9", interactive=True)
+                            red_green_p10 = gr.Slider(0, 1, label="G-P10", interactive=True)
+                        with gr.Tab(label="B", id=2) as red_blue_curves_tab:
+                            red_blue_p0 = gr.Slider(0, 1, label="B-P0", interactive=True)
+                            red_blue_p1 = gr.Slider(0, 1, label="B-P1", interactive=True)
+                            red_blue_p2 = gr.Slider(0, 1, label="B-P2", interactive=True)
+                            red_blue_p3 = gr.Slider(0, 1, label="B-P3", interactive=True)
+                            red_blue_p4 = gr.Slider(0, 1, label="B-P4", interactive=True)
+                            red_blue_p5 = gr.Slider(0, 1, label="B-P5", interactive=True)
+                            red_blue_p6 = gr.Slider(0, 1, label="B-P6", interactive=True)
+                            red_blue_p7 = gr.Slider(0, 1, label="B-P7", interactive=True)
+                            red_blue_p8 = gr.Slider(0, 1, label="B-P8", interactive=True)
+                            red_blue_p9 = gr.Slider(0, 1, label="B-P9", interactive=True)
+                            red_blue_p10 = gr.Slider(0, 1, label="B-P10", interactive=True)
+
+                with gr.Tab(label="Green", id=4) as green_curves_tab:
+                    green_curves = gr.Image(label="Green Curves", type="pil")
+                    with gr.Tabs() as channel_tabs:
+                        with gr.Tab(label="R", id=0) as green_red_curves_tab:
+                            green_red_p0 = gr.Slider(0, 1, label="R-P0", interactive=True)
+                            green_red_p1 = gr.Slider(0, 1, label="R-P1", interactive=True)
+                            green_red_p2 = gr.Slider(0, 1, label="R-P2", interactive=True)
+                            green_red_p3 = gr.Slider(0, 1, label="R-P3", interactive=True)
+                            green_red_p4 = gr.Slider(0, 1, label="R-P4", interactive=True)
+                            green_red_p5 = gr.Slider(0, 1, label="R-P5", interactive=True)
+                            green_red_p6 = gr.Slider(0, 1, label="R-P6", interactive=True)
+                            green_red_p7 = gr.Slider(0, 1, label="R-P7", interactive=True)
+                            green_red_p8 = gr.Slider(0, 1, label="R-P8", interactive=True)
+                            green_red_p9 = gr.Slider(0, 1, label="R-P9", interactive=True)
+                            green_red_p10 = gr.Slider(0, 1, label="R-P10", interactive=True)
+                        with gr.Tab(label="G", id=1) as green_green_curves_tab:
+                            green_green_p0 = gr.Slider(0, 1, label="G-P0", interactive=True)
+                            green_green_p1 = gr.Slider(0, 1, label="G-P1", interactive=True)
+                            green_green_p2 = gr.Slider(0, 1, label="G-P2", interactive=True)
+                            green_green_p3 = gr.Slider(0, 1, label="G-P3", interactive=True)
+                            green_green_p4 = gr.Slider(0, 1, label="G-P4", interactive=True)
+                            green_green_p5 = gr.Slider(0, 1, label="G-P5", interactive=True)
+                            green_green_p6 = gr.Slider(0, 1, label="G-P6", interactive=True)
+                            green_green_p7 = gr.Slider(0, 1, label="G-P7", interactive=True)
+                            green_green_p8 = gr.Slider(0, 1, label="G-P8", interactive=True)
+                            green_green_p9 = gr.Slider(0, 1, label="G-P9", interactive=True)
+                            green_green_p10 = gr.Slider(0, 1, label="G-P10", interactive=True)
+                        with gr.Tab(label="B", id=2) as green_blue_curves_tab:
+                            green_blue_p0 = gr.Slider(0, 1, label="B-P0", interactive=True)
+                            green_blue_p1 = gr.Slider(0, 1, label="B-P1", interactive=True)
+                            green_blue_p2 = gr.Slider(0, 1, label="B-P2", interactive=True)
+                            green_blue_p3 = gr.Slider(0, 1, label="B-P3", interactive=True)
+                            green_blue_p4 = gr.Slider(0, 1, label="B-P4", interactive=True)
+                            green_blue_p5 = gr.Slider(0, 1, label="B-P5", interactive=True)
+                            green_blue_p6 = gr.Slider(0, 1, label="B-P6", interactive=True)
+                            green_blue_p7 = gr.Slider(0, 1, label="B-P7", interactive=True)
+                            green_blue_p8 = gr.Slider(0, 1, label="B-P8", interactive=True)
+                            green_blue_p9 = gr.Slider(0, 1, label="B-P9", interactive=True)
+                            green_blue_p10 = gr.Slider(0, 1, label="B-P10", interactive=True)
+
+                with gr.Tab(label="Blue", id=5) as blue_curves_tab:
+                    blue_curves = gr.Image(label="Blue Curves", type="pil")
+                    with gr.Tabs() as channel_tabs:
+                        with gr.Tab(label="R", id=0) as blue_red_curves_tab:
+                            blue_red_p0 = gr.Slider(0, 1, label="R-P0", interactive=True)
+                            blue_red_p1 = gr.Slider(0, 1, label="R-P1", interactive=True)
+                            blue_red_p2 = gr.Slider(0, 1, label="R-P2", interactive=True)
+                            blue_red_p3 = gr.Slider(0, 1, label="R-P3", interactive=True)
+                            blue_red_p4 = gr.Slider(0, 1, label="R-P4", interactive=True)
+                            blue_red_p5 = gr.Slider(0, 1, label="R-P5", interactive=True)
+                            blue_red_p6 = gr.Slider(0, 1, label="R-P6", interactive=True)
+                            blue_red_p7 = gr.Slider(0, 1, label="R-P7", interactive=True)
+                            blue_red_p8 = gr.Slider(0, 1, label="R-P8", interactive=True)
+                            blue_red_p9 = gr.Slider(0, 1, label="R-P9", interactive=True)
+                            blue_red_p10 = gr.Slider(0, 1, label="R-P10", interactive=True)
+                        with gr.Tab(label="G", id=1) as blue_green_curves_tab:
+                            blue_green_p0 = gr.Slider(0, 1, label="G-P0", interactive=True)
+                            blue_green_p1 = gr.Slider(0, 1, label="G-P1", interactive=True)
+                            blue_green_p2 = gr.Slider(0, 1, label="G-P2", interactive=True)
+                            blue_green_p3 = gr.Slider(0, 1, label="G-P3", interactive=True)
+                            blue_green_p4 = gr.Slider(0, 1, label="G-P4", interactive=True)
+                            blue_green_p5 = gr.Slider(0, 1, label="G-P5", interactive=True)
+                            blue_green_p6 = gr.Slider(0, 1, label="G-P6", interactive=True)
+                            blue_green_p7 = gr.Slider(0, 1, label="G-P7", interactive=True)
+                            blue_green_p8 = gr.Slider(0, 1, label="G-P8", interactive=True)
+                            blue_green_p9 = gr.Slider(0, 1, label="G-P9", interactive=True)
+                            blue_green_p10 = gr.Slider(0, 1, label="G-P10", interactive=True)
+                        with gr.Tab(label="B", id=2) as blue_blue_curves_tab:
+                            blue_blue_p0 = gr.Slider(0, 1, label="B-P0", interactive=True)
+                            blue_blue_p1 = gr.Slider(0, 1, label="B-P1", interactive=True)
+                            blue_blue_p2 = gr.Slider(0, 1, label="B-P2", interactive=True)
+                            blue_blue_p3 = gr.Slider(0, 1, label="B-P3", interactive=True)
+                            blue_blue_p4 = gr.Slider(0, 1, label="B-P4", interactive=True)
+                            blue_blue_p5 = gr.Slider(0, 1, label="B-P5", interactive=True)
+                            blue_blue_p6 = gr.Slider(0, 1, label="B-P6", interactive=True)
+                            blue_blue_p7 = gr.Slider(0, 1, label="B-P7", interactive=True)
+                            blue_blue_p8 = gr.Slider(0, 1, label="B-P8", interactive=True)
+                            blue_blue_p9 = gr.Slider(0, 1, label="B-P9", interactive=True)
+                            blue_blue_p10 = gr.Slider(0, 1, label="B-P10", interactive=True)
+
+        with gr.Column():
+            out_image = gr.Image(type="pil", label="Output")
+            recompute_btn = gr.Button("Recompute")
+
+
+    run_btn.click(
+        process_img,
+        inputs=[image],
+        outputs=[oby_curves, oby_red_p0, oby_green_p0, oby_blue_p0, oby_red_p1, oby_green_p1, oby_blue_p1, oby_red_p2, oby_green_p2, oby_blue_p2, oby_red_p3, oby_green_p3, oby_blue_p3, oby_red_p4, oby_green_p4, oby_blue_p4, oby_red_p5, oby_green_p5, oby_blue_p5, oby_red_p6, oby_green_p6, oby_blue_p6, oby_red_p7, oby_green_p7, oby_blue_p7, oby_red_p8, oby_green_p8, oby_blue_p8, oby_red_p9, oby_green_p9, oby_blue_p9, oby_red_p10, oby_green_p10, oby_blue_p10,
+                 achro_curves, achro_red_p0, achro_green_p0, achro_blue_p0, achro_red_p1, achro_green_p1, achro_blue_p1, achro_red_p2, achro_green_p2, achro_blue_p2, achro_red_p3, achro_green_p3, achro_blue_p3, achro_red_p4, achro_green_p4, achro_blue_p4, achro_red_p5, achro_green_p5, achro_blue_p5, achro_red_p6, achro_green_p6, achro_blue_p6, achro_red_p7, achro_green_p7, achro_blue_p7, achro_red_p8, achro_green_p8, achro_blue_p8, achro_red_p9, achro_green_p9, achro_blue_p9, achro_red_p10, achro_green_p10, achro_blue_p10,
+                 pink_purple_curves, pp_red_p0, pp_green_p0, pp_blue_p0, pp_red_p1, pp_green_p1, pp_blue_p1, pp_red_p2, pp_green_p2, pp_blue_p2, pp_red_p3, pp_green_p3, pp_blue_p3, pp_red_p4, pp_green_p4, pp_blue_p4, pp_red_p5, pp_green_p5, pp_blue_p5, pp_red_p6, pp_green_p6, pp_blue_p6, pp_red_p7, pp_green_p7, pp_blue_p7, pp_red_p8, pp_green_p8, pp_blue_p8, pp_red_p9, pp_green_p9, pp_blue_p9, pp_red_p10, pp_green_p10, pp_blue_p10,
+                 red_curves, red_red_p0, red_green_p0, red_blue_p0, red_red_p1, red_green_p1, red_blue_p1, red_red_p2, red_green_p2, red_blue_p2, red_red_p3, red_green_p3, red_blue_p3, red_red_p4, red_green_p4, red_blue_p4, red_red_p5, red_green_p5, red_blue_p5, red_red_p6, red_green_p6, red_blue_p6, red_red_p7, red_green_p7, red_blue_p7, red_red_p8, red_green_p8, red_blue_p8, red_red_p9, red_green_p9, red_blue_p9, red_red_p10, red_green_p10, red_blue_p10,
+                 green_curves, green_red_p0, green_green_p0, green_blue_p0, green_red_p1, green_green_p1, green_blue_p1, green_red_p2, green_green_p2, green_blue_p2, green_red_p3, green_green_p3, green_blue_p3, green_red_p4, green_green_p4, green_blue_p4, green_red_p5, green_green_p5, green_blue_p5, green_red_p6, green_green_p6, green_blue_p6, green_red_p7, green_green_p7, green_blue_p7, green_red_p8, green_green_p8, green_blue_p8, green_red_p9, green_green_p9, green_blue_p9, green_red_p10, green_green_p10, green_blue_p10,
+                 blue_curves, blue_red_p0, blue_green_p0, blue_blue_p0, blue_red_p1, blue_green_p1, blue_blue_p1, blue_red_p2, blue_green_p2, blue_blue_p2, blue_red_p3, blue_green_p3, blue_blue_p3, blue_red_p4, blue_green_p4, blue_blue_p4, blue_red_p5, blue_green_p5, blue_blue_p5, blue_red_p6, blue_green_p6, blue_blue_p6, blue_red_p7, blue_green_p7, blue_blue_p7, blue_red_p8, blue_green_p8, blue_blue_p8, blue_red_p9, blue_green_p9, blue_blue_p9, blue_red_p10, blue_green_p10, blue_blue_p10,
+                 out_image],
+    )
+
+    recompute_btn.click(
+        process_img_with_sliders,
+        inputs=[image,
+                oby_red_p0, oby_green_p0, oby_blue_p0, oby_red_p1, oby_green_p1, oby_blue_p1, oby_red_p2, oby_green_p2, oby_blue_p2, oby_red_p3, oby_green_p3, oby_blue_p3, oby_red_p4, oby_green_p4, oby_blue_p4, oby_red_p5, oby_green_p5, oby_blue_p5, oby_red_p6, oby_green_p6, oby_blue_p6, oby_red_p7, oby_green_p7, oby_blue_p7, oby_red_p8, oby_green_p8, oby_blue_p8, oby_red_p9, oby_green_p9, oby_blue_p9, oby_red_p10, oby_green_p10, oby_blue_p10,
+                achro_red_p0, achro_green_p0, achro_blue_p0, achro_red_p1, achro_green_p1, achro_blue_p1, achro_red_p2, achro_green_p2, achro_blue_p2, achro_red_p3, achro_green_p3, achro_blue_p3, achro_red_p4, achro_green_p4, achro_blue_p4, achro_red_p5, achro_green_p5, achro_blue_p5, achro_red_p6, achro_green_p6, achro_blue_p6, achro_red_p7, achro_green_p7, achro_blue_p7, achro_red_p8, achro_green_p8, achro_blue_p8, achro_red_p9, achro_green_p9, achro_blue_p9, achro_red_p10, achro_green_p10, achro_blue_p10,
+                pp_red_p0, pp_green_p0, pp_blue_p0, pp_red_p1, pp_green_p1, pp_blue_p1, pp_red_p2, pp_green_p2, pp_blue_p2, pp_red_p3, pp_green_p3, pp_blue_p3, pp_red_p4, pp_green_p4, pp_blue_p4, pp_red_p5, pp_green_p5, pp_blue_p5, pp_red_p6, pp_green_p6, pp_blue_p6, pp_red_p7, pp_green_p7, pp_blue_p7, pp_red_p8, pp_green_p8, pp_blue_p8, pp_red_p9, pp_green_p9, pp_blue_p9, pp_red_p10, pp_green_p10, pp_blue_p10,
+                red_red_p0, red_green_p0, red_blue_p0, red_red_p1, red_green_p1, red_blue_p1, red_red_p2, red_green_p2, red_blue_p2, red_red_p3, red_green_p3, red_blue_p3, red_red_p4, red_green_p4, red_blue_p4, red_red_p5, red_green_p5, red_blue_p5, red_red_p6, red_green_p6, red_blue_p6, red_red_p7, red_green_p7, red_blue_p7, red_red_p8, red_green_p8, red_blue_p8, red_red_p9, red_green_p9, red_blue_p9, red_red_p10, red_green_p10, red_blue_p10,
+                green_red_p0, green_green_p0, green_blue_p0, green_red_p1, green_green_p1, green_blue_p1, green_red_p2, green_green_p2, green_blue_p2, green_red_p3, green_green_p3, green_blue_p3, green_red_p4, green_green_p4, green_blue_p4, green_red_p5, green_green_p5, green_blue_p5, green_red_p6, green_green_p6, green_blue_p6, green_red_p7, green_green_p7, green_blue_p7, green_red_p8, green_green_p8, green_blue_p8, green_red_p9, green_green_p9, green_blue_p9, green_red_p10, green_green_p10, green_blue_p10,
+                blue_red_p0, blue_green_p0, blue_blue_p0, blue_red_p1, blue_green_p1, blue_blue_p1, blue_red_p2, blue_green_p2, blue_blue_p2, blue_red_p3, blue_green_p3, blue_blue_p3, blue_red_p4, blue_green_p4, blue_blue_p4, blue_red_p5, blue_green_p5, blue_blue_p5, blue_red_p6, blue_green_p6, blue_blue_p6, blue_red_p7, blue_green_p7, blue_blue_p7, blue_red_p8, blue_green_p8, blue_blue_p8, blue_red_p9, blue_green_p9, blue_blue_p9, blue_red_p10, blue_green_p10, blue_blue_p10],
+        outputs=[oby_curves, oby_red_p0, oby_green_p0, oby_blue_p0, oby_red_p1, oby_green_p1, oby_blue_p1, oby_red_p2, oby_green_p2, oby_blue_p2, oby_red_p3, oby_green_p3, oby_blue_p3, oby_red_p4, oby_green_p4, oby_blue_p4, oby_red_p5, oby_green_p5, oby_blue_p5, oby_red_p6, oby_green_p6, oby_blue_p6, oby_red_p7, oby_green_p7, oby_blue_p7, oby_red_p8, oby_green_p8, oby_blue_p8, oby_red_p9, oby_green_p9, oby_blue_p9, oby_red_p10, oby_green_p10, oby_blue_p10,
+                 achro_curves, achro_red_p0, achro_green_p0, achro_blue_p0, achro_red_p1, achro_green_p1, achro_blue_p1, achro_red_p2, achro_green_p2, achro_blue_p2, achro_red_p3, achro_green_p3, achro_blue_p3, achro_red_p4, achro_green_p4, achro_blue_p4, achro_red_p5, achro_green_p5, achro_blue_p5, achro_red_p6, achro_green_p6, achro_blue_p6, achro_red_p7, achro_green_p7, achro_blue_p7, achro_red_p8, achro_green_p8, achro_blue_p8, achro_red_p9, achro_green_p9, achro_blue_p9, achro_red_p10, achro_green_p10, achro_blue_p10,
+                 pink_purple_curves, pp_red_p0, pp_green_p0, pp_blue_p0, pp_red_p1, pp_green_p1, pp_blue_p1, pp_red_p2, pp_green_p2, pp_blue_p2, pp_red_p3, pp_green_p3, pp_blue_p3, pp_red_p4, pp_green_p4, pp_blue_p4, pp_red_p5, pp_green_p5, pp_blue_p5, pp_red_p6, pp_green_p6, pp_blue_p6, pp_red_p7, pp_green_p7, pp_blue_p7, pp_red_p8, pp_green_p8, pp_blue_p8, pp_red_p9, pp_green_p9, pp_blue_p9, pp_red_p10, pp_green_p10, pp_blue_p10,
+                 red_curves, red_red_p0, red_green_p0, red_blue_p0, red_red_p1, red_green_p1, red_blue_p1, red_red_p2, red_green_p2, red_blue_p2, red_red_p3, red_green_p3, red_blue_p3, red_red_p4, red_green_p4, red_blue_p4, red_red_p5, red_green_p5, red_blue_p5, red_red_p6, red_green_p6, red_blue_p6, red_red_p7, red_green_p7, red_blue_p7, red_red_p8, red_green_p8, red_blue_p8, red_red_p9, red_green_p9, red_blue_p9, red_red_p10, red_green_p10, red_blue_p10,
+                 green_curves, green_red_p0, green_green_p0, green_blue_p0, green_red_p1, green_green_p1, green_blue_p1, green_red_p2, green_green_p2, green_blue_p2, green_red_p3, green_green_p3, green_blue_p3, green_red_p4, green_green_p4, green_blue_p4, green_red_p5, green_green_p5, green_blue_p5, green_red_p6, green_green_p6, green_blue_p6, green_red_p7, green_green_p7, green_blue_p7, green_red_p8, green_green_p8, green_blue_p8, green_red_p9, green_green_p9, green_blue_p9, green_red_p10, green_green_p10, green_blue_p10,
+                 blue_curves, blue_red_p0, blue_green_p0, blue_blue_p0, blue_red_p1, blue_green_p1, blue_blue_p1, blue_red_p2, blue_green_p2, blue_blue_p2, blue_red_p3, blue_green_p3, blue_blue_p3, blue_red_p4, blue_green_p4, blue_blue_p4, blue_red_p5, blue_green_p5, blue_blue_p5, blue_red_p6, blue_green_p6, blue_blue_p6, blue_red_p7, blue_green_p7, blue_blue_p7, blue_red_p8, blue_green_p8, blue_blue_p8, blue_red_p9, blue_green_p9, blue_blue_p9, blue_red_p10, blue_green_p10, blue_blue_p10,
+                 out_image],
+    )
+
+if __name__ == "__main__":
+    demo.launch()

configs/mit5k_dpe_config.yaml

@@ -0,0 +1,96 @@
+model:
+    ckpt_path: ~
+    backbone:
+        params:
+            input_channels: 3
+            output_channels: 3
+            encoder_dims: [4, 8, 16]
+            decoder_dims: [8, 4]
+    color_naming:
+        num_categories: 6
+    bezier_control_points_estimator:
+        params:
+            num_categories: ${model.color_naming.num_categories}
+            num_control_points: 10
+    local_fusion:
+        params:
+            att_in_dim: 3
+            num_categories: ${model.color_naming.num_categories}
+            max_pool_ksize1: 4
+            max_pool_ksize2: 2
+            encoder_dims: [8, 16]
+
+data:
+    train:
+        target: mit5k
+        params:
+            input_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/input
+            target_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/expertC_gt
+            img_ids_filepath: mit5k_ids_filepath/dpe/images_train.txt
+            transform:
+                - type: RandomCrop
+                  params:
+                    size: [ 256, 256 ]
+                - type: Resize
+                  params:
+                    size: 256
+                - type: RandomHorizontalFlip
+                  params:
+                    p: 0.5
+                - type: RandomVerticalFlip
+                  params:
+                    p: 0.5
+    valid:
+        target: mit5k
+        params:
+            input_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/input
+            target_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/expertC_gt
+            img_ids_filepath: mit5k_ids_filepath/dpe/images_test.txt
+
+    test:
+        target: mit5k
+        params:
+            input_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/input
+            target_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/expertC_gt
+            img_ids_filepath: mit5k_ids_filepath/dpe/images_test.txt
+
+train:
+    cuda_visible_device: 0
+    batch_size: 8
+    epochs: 100
+    valid_every: 1
+    optimizer:
+        type: Adam
+        params:
+            lr: 1e-4
+            betas: [ 0.9, 0.999 ]
+            eps: 1e-8
+    criterion:
+        type: backbone-L2-SSIM
+        params:
+            alpha: 0.5
+            ssim_window_size: 5
+eval:
+    metrics:
+        - type: PSNR
+          params:
+              data_range: 1.0
+        - type: SSIM
+          params:
+              kernel_size: 11
+        - type: LPIPS
+          params:
+              net: vgg
+              version: 0.1
+        - type: deltaE00
+        - type: deltaEab
+    metric_to_save: PSNR
+
+
+
+
+
+
+
+
+

configs/mit5k_upe_config.yaml

@@ -0,0 +1,90 @@
+model:
+    ckpt_path: ~
+    backbone:
+        params:
+            input_channels: 3
+            output_channels: 3
+            encoder_dims: [4, 8, 16]
+            decoder_dims: [8, 4]
+    color_naming:
+        num_categories: 6
+    bezier_control_points_estimator:
+        params:
+            num_categories: ${model.color_naming.num_categories}
+            num_control_points: 10
+    local_fusion:
+        params:
+            att_in_dim: 3
+            num_categories: ${model.color_naming.num_categories}
+            max_pool_ksize1: 4
+            max_pool_ksize2: 2
+            encoder_dims: [8, 16]
+
+data:
+    train:
+        target: mit5k
+        params:
+            input_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/input
+            target_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/expertC_gt
+            img_ids_filepath: /home/dserrano/Workspace/DeepLPF/adobe5k_upe/images_train.txt
+            transform:
+                - type: RandomCrop
+                  params:
+                    size: [ 256, 256 ]
+                - type: Resize
+                  params:
+                    size: 256
+                - type: RandomHorizontalFlip
+                  params:
+                    p: 0.5
+                - type: RandomVerticalFlip
+                  params:
+                    p: 0.5
+
+    test:
+        target: mit5k
+        params:
+            input_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/input
+            target_path: /home/dserrano/Documents/datasets/FiveK-UEGAN/expertC_gt
+            img_ids_filepath: /home/dserrano/Workspace/DeepLPF/adobe5k_upe/images_train.txt
+
+train:
+    cuda_visible_device: 0
+    batch_size: 8
+    epochs: 100
+    valid_every: 1
+    optimizer:
+        type: Adam
+        params:
+            lr: 1e-4
+            betas: [ 0.9, 0.999 ]
+            eps: 1e-8
+    criterion:
+        type: backbone-L2-SSIM
+        params:
+            alpha: 0.5
+            ssim_window_size: 5
+eval:
+    metrics:
+        - type: PSNR
+          params:
+              data_range: 1.0
+        - type: SSIM
+          params:
+              kernel_size: 11
+        - type: LPIPS
+          params:
+              net: vgg
+              version: 0.1
+        - type: deltaE00
+        - type: deltaEab
+    metric_to_save: PSNR
+
+
+
+
+
+
+
+
+

data/datasets.py

@@ -0,0 +1,109 @@
+import os
+import logging
+import numpy as np
+from glob import glob
+from PIL import Image
+from torch.utils.data import Dataset
+from torchvision import transforms
+from torchvision.transforms import functional as TF
+from torchvision.transforms.functional import to_tensor
+from data.image_transformations import get_transforms
+
+class MIT5KDataset(Dataset):
+    def __init__(self, input_path, target_path, img_ids_filepath, transform=None):
+        self.input_path = input_path
+        self.target_path = target_path
+        self.transform = transform
+        self.img_ids = self._read_img_ids(img_ids_filepath)
+        self.data = self._create_data_list()
+        if transform is not None:
+            self.image_transforms = get_transforms(transform)
+        else:
+            self.image_transforms = None
+
+    def _read_img_ids(self, img_ids_filepath):
+        # Read the image IDs from the txt file
+        with open(img_ids_filepath, 'r') as f:
+            img_ids = [line.strip() for line in f.readlines()]
+        return img_ids
+
+    def _create_data_list(self):
+        # Create a list of dictionaries with 'input_path', 'target_path' and 'name'
+        data_list = []
+        for input_file in glob(os.path.join(self.input_path, "*")):
+            img_id = os.path.basename(input_file).split('-')[0]
+            if img_id in self.img_ids:
+                target_file = os.path.join(self.target_path, os.path.basename(os.path.basename(input_file)))
+                if not os.path.exists(target_file):
+                    raise FileNotFoundError(f"Target file {target_file} not found. While input file {input_file} was found.")
+                data_list.append({'input_path': input_file, 'target_path': target_file, 'name': img_id})
+
+        return data_list
+
+    def __len__(self):
+        return len(self.data)
+
+    def __getitem__(self, idx):
+        data = self.data[idx]
+        input_image, target_image = self._load_image_pair(data['input_path'], data['target_path'])
+
+        return {'input_image': input_image, 'target_image': target_image, 'name':data['name']}
+
+    def _load_image_pair(self, img1_path, img2_path):
+        img1_tensor = to_tensor(np.array(Image.open(img1_path).convert('RGB')))
+        img2_tensor = to_tensor(np.array(Image.open(img2_path).convert('RGB')))
+
+        if self.image_transforms is not None:
+            for image_transform in self.image_transforms:
+                img1_tensor, img2_tensor = image_transform(img1_tensor, img2_tensor)
+
+        return img1_tensor, img2_tensor
+
+#class PPR10KDataset(Dataset):
+
+
+def get_single_dataset(type, params):
+    if type == 'mit5k':
+        return MIT5KDataset(**params)
+    elif type == 'ppr10k':
+    	# TODO:
+        return PPR10KDataset(**params)
+    else:
+        raise ValueError(f"Unsupported dataset type: {type}")
+
+
+def get_datasets(config):
+    """Returns the datsaets based on the configuration file."""
+
+    if len(config) == 2:
+        train_dataset = get_single_dataset(config.train.target, config.train.params)
+        test_dataset = get_single_dataset(config.test.target, config.test.params)
+        return train_dataset, None, test_dataset
+
+    elif len(config) == 3:
+        train_dataset = get_single_dataset(config.train.target, config.train.params)
+        val_dataset = get_single_dataset(config.valid.target, config.valid.params)
+        test_dataset = get_single_dataset(config.test.target, config.test.params)
+        return train_dataset, val_dataset, test_dataset
+
+    else:
+        raise ValueError("The number of datasets should be 2 (train/test) or 3 (train/valid/test).")
+
+if __name__ == "__main__":
+    from omegaconf import OmegaConf
+    config = OmegaConf.load("../configs/mit5k_upe_config.yaml")
+
+    dataset = MIT5KDataset(**config.data.train.params)
+    input_img, target_img, name = dataset[0]
+
+    import matplotlib.pyplot as plt
+    plt.subplot(1, 2, 1)
+    plt.imshow(input_img.squeeze().permute(1, 2, 0).numpy())
+    plt.title("Input Image")
+    plt.subplot(1, 2, 2)
+    plt.imshow(target_img.squeeze().permute(1, 2, 0).numpy())
+    plt.title("Target Image")
+    plt.show()
+
+
+

data/image_transformations.py

@@ -0,0 +1,61 @@
+import random
+import torchvision.transforms.functional as F
+from torchvision import transforms
+
+class RandomCropPair:
+    def __init__(self, size):
+        self.size = size
+
+    def __call__(self, img1, img2):
+        i, j, h, w = transforms.RandomCrop.get_params(img1, self.size)
+        img1 = F.crop(img1, i, j, h, w)
+        img2 = F.crop(img2, i, j, h, w)
+        return img1, img2
+
+class ResizePair:
+    def __init__(self, size):
+        self.size = size
+
+    def __call__(self, img1, img2):
+        # antialias=True is used to avoid torchvision warning
+        img1 = F.resize(img1, self.size, antialias=True)
+        img2 = F.resize(img2, self.size, antialias=True)
+        return img1, img2
+
+class RandomHorizontalFlipPair:
+    def __init__(self, p=0.5):
+        self.p = p
+
+    def __call__(self, img1, img2):
+        if random.random() < self.p:
+            img1 = F.hflip(img1)
+            img2 = F.hflip(img2)
+        return img1, img2
+
+class RandomVerticalFlipPair:
+    def __init__(self, p=0.5):
+        self.p = p
+
+    def __call__(self, img1, img2):
+        if random.random() < self.p:
+            img1 = F.vflip(img1)
+            img2 = F.vflip(img2)
+        return img1, img2
+
+def get_transforms(transforms_config):
+    transform_list = []
+    for transform in transforms_config:
+        transform_type = transform['type']
+        params = transform['params']
+        if transform_type == 'RandomCrop':
+            transform_list.append(RandomCropPair(**params))
+        elif transform_type == 'Resize':
+            transform_list.append(ResizePair(**params))
+        elif transform_type == 'RandomHorizontalFlip':
+            transform_list.append(RandomHorizontalFlipPair(**params))
+        elif transform_type == 'RandomVerticalFlip':
+            transform_list.append(RandomVerticalFlipPair(**params))
+        else:
+            raise ValueError(f"Unsupported transform type: {transform_type}")
+
+    return transform_list

mit5k_ids_filepath/dpe/images_test.txt

@@ -0,0 +1,50 @@
+a4501
+a4502
+a4503
+a4504
+a4505
+a4506
+a4507
+a4508
+a4509
+a4510
+a4511
+a4512
+a4513
+a4514
+a4515
+a4516
+a4517
+a4518
+a4519
+a4520
+a4521
+a4522
+a4523
+a4524
+a4525
+a4526
+a4527
+a4528
+a4529
+a4530
+a4531
+a4532
+a4533
+a4534
+a4535
+a4536
+a4537
+a4538
+a4539
+a4540
+a4541
+a4542
+a4543
+a4544
+a4545
+a4546
+a4547
+a4548
+a4549
+a4550

mit5k_ids_filepath/dpe/images_train.txt

@@ -0,0 +1,62 @@
+a0001
+a0002
+a0003
+a0004
+a0005
+a0006
+a0007
+a0008
+a0009
+a0010
+a0011
+a0012
+a0013
+a0014
+a0015
+a0016
+a0017
+a0018
+a0019
+a0020
+a0021
+a0022
+a0023
+a0024
+a0025
+a0026
+a0027
+a0028
+a0029
+a0030
+a0031
+a0032
+a0033
+a0034
+a0035
+a0036
+a0037
+a0038
+a0039
+a0040
+a0041
+a0042
+a0043
+a0044
+a0045
+a0046
+a0047
+a0048
+a0049
+a0050
+a0051
+a0052
+a0053
+a0054
+a0055
+a0056
+a0057
+a0058
+a0059
+a0060
+a0061
+a0062

mit5k_ids_filepath/dpe/images_valid.txt

@@ -0,0 +1,2250 @@
+a2251
+a2252
+a2253
+a2254
+a2255
+a2256
+a2257
+a2258
+a2259
+a2260
+a2261
+a2262
+a2263
+a2264
+a2265
+a2266
+a2267
+a2268
+a2269
+a2270
+a2271
+a2272
+a2273
+a2274
+a2275
+a2276
+a2277
+a2278
+a2279
+a2280
+a2281
+a2282
+a2283
+a2284
+a2285
+a2286
+a2287
+a2288
+a2289
+a2290
+a2291
+a2292
+a2293
+a2294
+a2295
+a2296
+a2297
+a2298
+a2299
+a2300
+a2301
+a2302
+a2303
+a2304
+a2305
+a2306
+a2307
+a2308
+a2309
+a2310
+a2311
+a2312
+a2313
+a2314
+a2315
+a2316
+a2317
+a2318
+a2319
+a2320
+a2321
+a2322
+a2323
+a2324
+a2325
+a2326
+a2327
+a2328
+a2329
+a2330
+a2331
+a2332
+a2333
+a2334
+a2335
+a2336
+a2337
+a2338
+a2339
+a2340
+a2341
+a2342
+a2343
+a2344
+a2345
+a2346
+a2347
+a2348
+a2349
+a2350
+a2351
+a2352
+a2353
+a2354
+a2355
+a2356
+a2357
+a2358
+a2359
+a2360
+a2361
+a2362
+a2363
+a2364
+a2365
+a2366
+a2367
+a2368
+a2369
+a2370
+a2371
+a2372
+a2373
+a2374
+a2375
+a2376
+a2377
+a2378
+a2379
+a2380
+a2381
+a2382
+a2383
+a2384
+a2385
+a2386
+a2387
+a2388
+a2389
+a2390
+a2391
+a2392
+a2393
+a2394
+a2395
+a2396
+a2397
+a2398
+a2399
+a2400
+a2401
+a2402
+a2403
+a2404
+a2405
+a2406
+a2407
+a2408
+a2409
+a2410
+a2411
+a2412
+a2413
+a2414
+a2415
+a2416
+a2417
+a2418
+a2419
+a2420
+a2421
+a2422
+a2423
+a2424
+a2425
+a2426
+a2427
+a2428
+a2429
+a2430
+a2431
+a2432
+a2433
+a2434
+a2435
+a2436
+a2437
+a2438
+a2439
+a2440
+a2441
+a2442
+a2443
+a2444
+a2445
+a2446
+a2447
+a2448
+a2449
+a2450
+a2451
+a2452
+a2453
+a2454
+a2455
+a2456
+a2457
+a2458
+a2459
+a2460
+a2461
+a2462
+a2463
+a2464
+a2465
+a2466
+a2467
+a2468
+a2469
+a2470
+a2471
+a2472
+a2473
+a2474
+a2475
+a2476
+a2477
+a2478
+a2479
+a2480
+a2481
+a2482
+a2483
+a2484
+a2485
+a2486
+a2487
+a2488
+a2489
+a2490
+a2491
+a2492
+a2493
+a2494
+a2495
+a2496
+a2497
+a2498
+a2499
+a2500
+a2501
+a2502
+a2503
+a2504
+a2505
+a2506
+a2507
+a2508
+a2509
+a2510
+a2511
+a2512
+a2513
+a2514
+a2515
+a2516
+a2517
+a2518
+a2519
+a2520
+a2521
+a2522
+a2523
+a2524
+a2525
+a2526
+a2527
+a2528
+a2529
+a2530
+a2531
+a2532
+a2533
+a2534
+a2535
+a2536
+a2537
+a2538
+a2539
+a2540
+a2541
+a2542
+a2543
+a2544
+a2545
+a2546
+a2547
+a2548
+a2549
+a2550
+a2551
+a2552
+a2553
+a2554
+a2555
+a2556
+a2557
+a2558
+a2559
+a2560
+a2561
+a2562
+a2563
+a2564
+a2565
+a2566
+a2567
+a2568
+a2569
+a2570
+a2571
+a2572
+a2573
+a2574
+a2575
+a2576
+a2577
+a2578
+a2579
+a2580
+a2581
+a2582
+a2583
+a2584
+a2585
+a2586
+a2587
+a2588
+a2589
+a2590
+a2591
+a2592
+a2593
+a2594
+a2595
+a2596
+a2597
+a2598
+a2599
+a2600
+a2601
+a2602
+a2603
+a2604
+a2605
+a2606
+a2607
+a2608
+a2609
+a2610
+a2611
+a2612
+a2613
+a2614
+a2615
+a2616
+a2617
+a2618
+a2619
+a2620
+a2621
+a2622
+a2623
+a2624
+a2625
+a2626
+a2627
+a2628
+a2629
+a2630
+a2631
+a2632
+a2633
+a2634
+a2635
+a2636
+a2637
+a2638
+a2639
+a2640
+a2641
+a2642
+a2643
+a2644
+a2645
+a2646
+a2647
+a2648
+a2649
+a2650
+a2651
+a2652
+a2653
+a2654
+a2655
+a2656
+a2657
+a2658
+a2659
+a2660
+a2661
+a2662
+a2663
+a2664
+a2665
+a2666
+a2667
+a2668
+a2669
+a2670
+a2671
+a2672
+a2673
+a2674
+a2675
+a2676
+a2677
+a2678
+a2679
+a2680
+a2681
+a2682
+a2683
+a2684
+a2685
+a2686
+a2687
+a2688
+a2689
+a2690
+a2691
+a2692
+a2693
+a2694
+a2695
+a2696
+a2697
+a2698
+a2699
+a2700
+a2701
+a2702
+a2703
+a2704
+a2705
+a2706
+a2707
+a2708
+a2709
+a2710
+a2711
+a2712
+a2713
+a2714
+a2715
+a2716
+a2717
+a2718
+a2719
+a2720
+a2721
+a2722
+a2723
+a2724
+a2725
+a2726
+a2727
+a2728
+a2729
+a2730
+a2731
+a2732
+a2733
+a2734
+a2735
+a2736
+a2737
+a2738
+a2739
+a2740
+a2741
+a2742
+a2743
+a2744
+a2745
+a2746
+a2747
+a2748
+a2749
+a2750
+a2751
+a2752
+a2753
+a2754
+a2755
+a2756
+a2757
+a2758
+a2759
+a2760
+a2761
+a2762
+a2763
+a2764
+a2765
+a2766
+a2767
+a2768
+a2769
+a2770
+a2771
+a2772
+a2773
+a2774
+a2775
+a2776
+a2777
+a2778
+a2779
+a2780
+a2781
+a2782
+a2783
+a2784
+a2785
+a2786
+a2787
+a2788
+a2789
+a2790
+a2791
+a2792
+a2793
+a2794
+a2795
+a2796
+a2797
+a2798
+a2799
+a2800
+a2801
+a2802
+a2803
+a2804
+a2805
+a2806
+a2807
+a2808
+a2809
+a2810
+a2811
+a2812
+a2813
+a2814
+a2815
+a2816
+a2817
+a2818
+a2819
+a2820
+a2821
+a2822
+a2823
+a2824
+a2825
+a2826
+a2827
+a2828
+a2829
+a2830
+a2831
+a2832
+a2833
+a2834
+a2835
+a2836
+a2837
+a2838
+a2839
+a2840
+a2841
+a2842
+a2843
+a2844
+a2845
+a2846
+a2847
+a2848
+a2849
+a2850
+a2851
+a2852
+a2853
+a2854
+a2855
+a2856
+a2857
+a2858
+a2859
+a2860
+a2861
+a2862
+a2863
+a2864
+a2865
+a2866
+a2867
+a2868
+a2869
+a2870
+a2871
+a2872
+a2873
+a2874
+a2875
+a2876
+a2877
+a2878
+a2879
+a2880
+a2881
+a2882
+a2883
+a2884
+a2885
+a2886
+a2887
+a2888
+a2889
+a2890
+a2891
+a2892
+a2893
+a2894
+a2895
+a2896
+a2897
+a2898
+a2899
+a2900
+a2901
+a2902
+a2903
+a2904
+a2905
+a2906
+a2907
+a2908
+a2909
+a2910
+a2911
+a2912
+a2913
+a2914
+a2915
+a2916
+a2917
+a2918
+a2919
+a2920
+a2921
+a2922
+a2923
+a2924
+a2925
+a2926
+a2927
+a2928
+a2929
+a2930
+a2931
+a2932
+a2933
+a2934
+a2935
+a2936
+a2937
+a2938
+a2939
+a2940
+a2941
+a2942
+a2943
+a2944
+a2945
+a2946
+a2947
+a2948
+a2949
+a2950
+a2951
+a2952
+a2953
+a2954
+a2955
+a2956
+a2957
+a2958
+a2959
+a2960
+a2961
+a2962
+a2963
+a2964
+a2965
+a2966
+a2967
+a2968
+a2969
+a2970
+a2971
+a2972
+a2973
+a2974
+a2975
+a2976
+a2977
+a2978
+a2979
+a2980
+a2981
+a2982
+a2983
+a2984
+a2985
+a2986
+a2987
+a2988
+a2989
+a2990
+a2991
+a2992
+a2993
+a2994
+a2995
+a2996
+a2997
+a2998
+a2999
+a3000
+a3001
+a3002
+a3003
+a3004
+a3005
+a3006
+a3007
+a3008
+a3009
+a3010
+a3011
+a3012
+a3013
+a3014
+a3015
+a3016
+a3017
+a3018
+a3019
+a3020
+a3021
+a3022
+a3023
+a3024
+a3025
+a3026
+a3027
+a3028
+a3029
+a3030
+a3031
+a3032
+a3033
+a3034
+a3035
+a3036
+a3037
+a3038
+a3039
+a3040
+a3041
+a3042
+a3043
+a3044
+a3045
+a3046
+a3047
+a3048
+a3049
+a3050
+a3051
+a3052
+a3053
+a3054
+a3055
+a3056
+a3057
+a3058
+a3059
+a3060
+a3061
+a3062
+a3063
+a3064
+a3065
+a3066
+a3067
+a3068
+a3069
+a3070
+a3071
+a3072
+a3073
+a3074
+a3075
+a3076
+a3077
+a3078
+a3079
+a3080
+a3081
+a3082
+a3083
+a3084
+a3085
+a3086
+a3087
+a3088
+a3089
+a3090
+a3091
+a3092
+a3093
+a3094
+a3095
+a3096
+a3097
+a3098
+a3099
+a3100
+a3101
+a3102
+a3103
+a3104
+a3105
+a3106
+a3107
+a3108
+a3109
+a3110
+a3111
+a3112
+a3113
+a3114
+a3115
+a3116
+a3117
+a3118
+a3119
+a3120
+a3121
+a3122
+a3123
+a3124
+a3125
+a3126
+a3127
+a3128
+a3129
+a3130
+a3131
+a3132
+a3133
+a3134
+a3135
+a3136
+a3137
+a3138
+a3139
+a3140
+a3141
+a3142
+a3143
+a3144
+a3145
+a3146
+a3147
+a3148
+a3149
+a3150
+a3151
+a3152
+a3153
+a3154
+a3155
+a3156
+a3157
+a3158
+a3159
+a3160
+a3161
+a3162
+a3163
+a3164
+a3165
+a3166
+a3167
+a3168
+a3169
+a3170
+a3171
+a3172
+a3173
+a3174
+a3175
+a3176
+a3177
+a3178
+a3179
+a3180
+a3181
+a3182
+a3183
+a3184
+a3185
+a3186
+a3187
+a3188
+a3189
+a3190
+a3191
+a3192
+a3193
+a3194
+a3195
+a3196
+a3197
+a3198
+a3199
+a3200
+a3201
+a3202
+a3203
+a3204
+a3205
+a3206
+a3207
+a3208
+a3209
+a3210
+a3211
+a3212
+a3213
+a3214
+a3215
+a3216
+a3217
+a3218
+a3219
+a3220
+a3221
+a3222
+a3223
+a3224
+a3225
+a3226
+a3227
+a3228
+a3229
+a3230
+a3231
+a3232
+a3233
+a3234
+a3235
+a3236
+a3237
+a3238
+a3239
+a3240
+a3241
+a3242
+a3243
+a3244
+a3245
+a3246
+a3247
+a3248
+a3249
+a3250
+a3251
+a3252
+a3253
+a3254
+a3255
+a3256
+a3257
+a3258
+a3259
+a3260
+a3261
+a3262
+a3263
+a3264
+a3265
+a3266
+a3267
+a3268
+a3269
+a3270
+a3271
+a3272
+a3273
+a3274
+a3275
+a3276
+a3277
+a3278
+a3279
+a3280
+a3281
+a3282
+a3283
+a3284
+a3285
+a3286
+a3287
+a3288
+a3289
+a3290
+a3291
+a3292
+a3293
+a3294
+a3295
+a3296
+a3297
+a3298
+a3299
+a3300
+a3301
+a3302
+a3303
+a3304
+a3305
+a3306
+a3307
+a3308
+a3309
+a3310
+a3311
+a3312
+a3313
+a3314
+a3315
+a3316
+a3317
+a3318
+a3319
+a3320
+a3321
+a3322
+a3323
+a3324
+a3325
+a3326
+a3327
+a3328
+a3329
+a3330
+a3331
+a3332
+a3333
+a3334
+a3335
+a3336
+a3337
+a3338
+a3339
+a3340
+a3341
+a3342
+a3343
+a3344
+a3345
+a3346
+a3347
+a3348
+a3349
+a3350
+a3351
+a3352
+a3353
+a3354
+a3355
+a3356
+a3357
+a3358
+a3359
+a3360
+a3361
+a3362
+a3363
+a3364
+a3365
+a3366
+a3367
+a3368
+a3369
+a3370
+a3371
+a3372
+a3373
+a3374
+a3375
+a3376
+a3377
+a3378
+a3379
+a3380
+a3381
+a3382
+a3383
+a3384
+a3385
+a3386
+a3387
+a3388
+a3389
+a3390
+a3391
+a3392
+a3393
+a3394
+a3395
+a3396
+a3397
+a3398
+a3399
+a3400
+a3401
+a3402
+a3403
+a3404
+a3405
+a3406
+a3407
+a3408
+a3409
+a3410
+a3411
+a3412
+a3413
+a3414
+a3415
+a3416
+a3417
+a3418
+a3419
+a3420
+a3421
+a3422
+a3423
+a3424
+a3425
+a3426
+a3427
+a3428
+a3429
+a3430
+a3431
+a3432
+a3433
+a3434
+a3435
+a3436
+a3437
+a3438
+a3439
+a3440
+a3441
+a3442
+a3443
+a3444
+a3445
+a3446
+a3447
+a3448
+a3449
+a3450
+a3451
+a3452
+a3453
+a3454
+a3455
+a3456
+a3457
+a3458
+a3459
+a3460
+a3461
+a3462
+a3463
+a3464
+a3465
+a3466
+a3467
+a3468
+a3469
+a3470
+a3471
+a3472
+a3473
+a3474
+a3475
+a3476
+a3477
+a3478
+a3479
+a3480
+a3481
+a3482
+a3483
+a3484
+a3485
+a3486
+a3487
+a3488
+a3489
+a3490
+a3491
+a3492
+a3493
+a3494
+a3495
+a3496
+a3497
+a3498
+a3499
+a3500
+a3501
+a3502
+a3503
+a3504
+a3505
+a3506
+a3507
+a3508
+a3509
+a3510
+a3511
+a3512
+a3513
+a3514
+a3515
+a3516
+a3517
+a3518
+a3519
+a3520
+a3521
+a3522
+a3523
+a3524
+a3525
+a3526
+a3527
+a3528
+a3529
+a3530
+a3531
+a3532
+a3533
+a3534
+a3535
+a3536
+a3537
+a3538
+a3539
+a3540
+a3541
+a3542
+a3543
+a3544
+a3545
+a3546
+a3547
+a3548
+a3549
+a3550
+a3551
+a3552
+a3553
+a3554
+a3555
+a3556
+a3557
+a3558
+a3559
+a3560
+a3561
+a3562
+a3563
+a3564
+a3565
+a3566
+a3567
+a3568
+a3569
+a3570
+a3571
+a3572
+a3573
+a3574
+a3575
+a3576
+a3577
+a3578
+a3579
+a3580
+a3581
+a3582
+a3583
+a3584
+a3585
+a3586
+a3587
+a3588
+a3589
+a3590
+a3591
+a3592
+a3593
+a3594
+a3595
+a3596
+a3597
+a3598
+a3599
+a3600
+a3601
+a3602
+a3603
+a3604
+a3605
+a3606
+a3607
+a3608
+a3609
+a3610
+a3611
+a3612
+a3613
+a3614
+a3615
+a3616
+a3617
+a3618
+a3619
+a3620
+a3621
+a3622
+a3623
+a3624
+a3625
+a3626
+a3627
+a3628
+a3629
+a3630
+a3631
+a3632
+a3633
+a3634
+a3635
+a3636
+a3637
+a3638
+a3639
+a3640
+a3641
+a3642
+a3643
+a3644
+a3645
+a3646
+a3647
+a3648
+a3649
+a3650
+a3651
+a3652
+a3653
+a3654
+a3655
+a3656
+a3657
+a3658
+a3659
+a3660
+a3661
+a3662
+a3663
+a3664
+a3665
+a3666
+a3667
+a3668
+a3669
+a3670
+a3671
+a3672
+a3673
+a3674
+a3675
+a3676
+a3677
+a3678
+a3679
+a3680
+a3681
+a3682
+a3683
+a3684
+a3685
+a3686
+a3687
+a3688
+a3689
+a3690
+a3691
+a3692
+a3693
+a3694
+a3695
+a3696
+a3697
+a3698
+a3699
+a3700
+a3701
+a3702
+a3703
+a3704
+a3705
+a3706
+a3707
+a3708
+a3709
+a3710
+a3711
+a3712
+a3713
+a3714
+a3715
+a3716
+a3717
+a3718
+a3719
+a3720
+a3721
+a3722
+a3723
+a3724
+a3725
+a3726
+a3727
+a3728
+a3729
+a3730
+a3731
+a3732
+a3733
+a3734
+a3735
+a3736
+a3737
+a3738
+a3739
+a3740
+a3741
+a3742
+a3743
+a3744
+a3745
+a3746
+a3747
+a3748
+a3749
+a3750
+a3751
+a3752
+a3753
+a3754
+a3755
+a3756
+a3757
+a3758
+a3759
+a3760
+a3761
+a3762
+a3763
+a3764
+a3765
+a3766
+a3767
+a3768
+a3769
+a3770
+a3771
+a3772
+a3773
+a3774
+a3775
+a3776
+a3777
+a3778
+a3779
+a3780
+a3781
+a3782
+a3783
+a3784
+a3785
+a3786
+a3787
+a3788
+a3789
+a3790
+a3791
+a3792
+a3793
+a3794
+a3795
+a3796
+a3797
+a3798
+a3799
+a3800
+a3801
+a3802
+a3803
+a3804
+a3805
+a3806
+a3807
+a3808
+a3809
+a3810
+a3811
+a3812
+a3813
+a3814
+a3815
+a3816
+a3817
+a3818
+a3819
+a3820
+a3821
+a3822
+a3823
+a3824
+a3825
+a3826
+a3827
+a3828
+a3829
+a3830
+a3831
+a3832
+a3833
+a3834
+a3835
+a3836
+a3837
+a3838
+a3839
+a3840
+a3841
+a3842
+a3843
+a3844
+a3845
+a3846
+a3847
+a3848
+a3849
+a3850
+a3851
+a3852
+a3853
+a3854
+a3855
+a3856
+a3857
+a3858
+a3859
+a3860
+a3861
+a3862
+a3863
+a3864
+a3865
+a3866
+a3867
+a3868
+a3869
+a3870
+a3871
+a3872
+a3873
+a3874
+a3875
+a3876
+a3877
+a3878
+a3879
+a3880
+a3881
+a3882
+a3883
+a3884
+a3885
+a3886
+a3887
+a3888
+a3889
+a3890
+a3891
+a3892
+a3893
+a3894
+a3895
+a3896
+a3897
+a3898
+a3899
+a3900
+a3901
+a3902
+a3903
+a3904
+a3905
+a3906
+a3907
+a3908
+a3909
+a3910
+a3911
+a3912
+a3913
+a3914
+a3915
+a3916
+a3917
+a3918
+a3919
+a3920
+a3921
+a3922
+a3923
+a3924
+a3925
+a3926
+a3927
+a3928
+a3929
+a3930
+a3931
+a3932
+a3933
+a3934
+a3935
+a3936
+a3937
+a3938
+a3939
+a3940
+a3941
+a3942
+a3943
+a3944
+a3945
+a3946
+a3947
+a3948
+a3949
+a3950
+a3951
+a3952
+a3953
+a3954
+a3955
+a3956
+a3957
+a3958
+a3959
+a3960
+a3961
+a3962
+a3963
+a3964
+a3965
+a3966
+a3967
+a3968
+a3969
+a3970
+a3971
+a3972
+a3973
+a3974
+a3975
+a3976
+a3977
+a3978
+a3979
+a3980
+a3981
+a3982
+a3983
+a3984
+a3985
+a3986
+a3987
+a3988
+a3989
+a3990
+a3991
+a3992
+a3993
+a3994
+a3995
+a3996
+a3997
+a3998
+a3999
+a4000
+a4001
+a4002
+a4003
+a4004
+a4005
+a4006
+a4007
+a4008
+a4009
+a4010
+a4011
+a4012
+a4013
+a4014
+a4015
+a4016
+a4017
+a4018
+a4019
+a4020
+a4021
+a4022
+a4023
+a4024
+a4025
+a4026
+a4027
+a4028
+a4029
+a4030
+a4031
+a4032
+a4033
+a4034
+a4035
+a4036
+a4037
+a4038
+a4039
+a4040
+a4041
+a4042
+a4043
+a4044
+a4045
+a4046
+a4047
+a4048
+a4049
+a4050
+a4051
+a4052
+a4053
+a4054
+a4055
+a4056
+a4057
+a4058
+a4059
+a4060
+a4061
+a4062
+a4063
+a4064
+a4065
+a4066
+a4067
+a4068
+a4069
+a4070
+a4071
+a4072
+a4073
+a4074
+a4075
+a4076
+a4077
+a4078
+a4079
+a4080
+a4081
+a4082
+a4083
+a4084
+a4085
+a4086
+a4087
+a4088
+a4089
+a4090
+a4091
+a4092
+a4093
+a4094
+a4095
+a4096
+a4097
+a4098
+a4099
+a4100
+a4101
+a4102
+a4103
+a4104
+a4105
+a4106
+a4107
+a4108
+a4109
+a4110
+a4111
+a4112
+a4113
+a4114
+a4115
+a4116
+a4117
+a4118
+a4119
+a4120
+a4121
+a4122
+a4123
+a4124
+a4125
+a4126
+a4127
+a4128
+a4129
+a4130
+a4131
+a4132
+a4133
+a4134
+a4135
+a4136
+a4137
+a4138
+a4139
+a4140
+a4141
+a4142
+a4143
+a4144
+a4145
+a4146
+a4147
+a4148
+a4149
+a4150
+a4151
+a4152
+a4153
+a4154
+a4155
+a4156
+a4157
+a4158
+a4159
+a4160
+a4161
+a4162
+a4163
+a4164
+a4165
+a4166
+a4167
+a4168
+a4169
+a4170
+a4171
+a4172
+a4173
+a4174
+a4175
+a4176
+a4177
+a4178
+a4179
+a4180
+a4181
+a4182
+a4183
+a4184
+a4185
+a4186
+a4187
+a4188
+a4189
+a4190
+a4191
+a4192
+a4193
+a4194
+a4195
+a4196
+a4197
+a4198
+a4199
+a4200
+a4201
+a4202
+a4203
+a4204
+a4205
+a4206
+a4207
+a4208
+a4209
+a4210
+a4211
+a4212
+a4213
+a4214
+a4215
+a4216
+a4217
+a4218
+a4219
+a4220
+a4221
+a4222
+a4223
+a4224
+a4225
+a4226
+a4227
+a4228
+a4229
+a4230
+a4231
+a4232
+a4233
+a4234
+a4235
+a4236
+a4237
+a4238
+a4239
+a4240
+a4241
+a4242
+a4243
+a4244
+a4245
+a4246
+a4247
+a4248
+a4249
+a4250
+a4251
+a4252
+a4253
+a4254
+a4255
+a4256
+a4257
+a4258
+a4259
+a4260
+a4261
+a4262
+a4263
+a4264
+a4265
+a4266
+a4267
+a4268
+a4269
+a4270
+a4271
+a4272
+a4273
+a4274
+a4275
+a4276
+a4277
+a4278
+a4279
+a4280
+a4281
+a4282
+a4283
+a4284
+a4285
+a4286
+a4287
+a4288
+a4289
+a4290
+a4291
+a4292
+a4293
+a4294
+a4295
+a4296
+a4297
+a4298
+a4299
+a4300
+a4301
+a4302
+a4303
+a4304
+a4305
+a4306
+a4307
+a4308
+a4309
+a4310
+a4311
+a4312
+a4313
+a4314
+a4315
+a4316
+a4317
+a4318
+a4319
+a4320
+a4321
+a4322
+a4323
+a4324
+a4325
+a4326
+a4327
+a4328
+a4329
+a4330
+a4331
+a4332
+a4333
+a4334
+a4335
+a4336
+a4337
+a4338
+a4339
+a4340
+a4341
+a4342
+a4343
+a4344
+a4345
+a4346
+a4347
+a4348
+a4349
+a4350
+a4351
+a4352
+a4353
+a4354
+a4355
+a4356
+a4357
+a4358
+a4359
+a4360
+a4361
+a4362
+a4363
+a4364
+a4365
+a4366
+a4367
+a4368
+a4369
+a4370
+a4371
+a4372
+a4373
+a4374
+a4375
+a4376
+a4377
+a4378
+a4379
+a4380
+a4381
+a4382
+a4383
+a4384
+a4385
+a4386
+a4387
+a4388
+a4389
+a4390
+a4391
+a4392
+a4393
+a4394
+a4395
+a4396
+a4397
+a4398
+a4399
+a4400
+a4401
+a4402
+a4403
+a4404
+a4405
+a4406
+a4407
+a4408
+a4409
+a4410
+a4411
+a4412
+a4413
+a4414
+a4415
+a4416
+a4417
+a4418
+a4419
+a4420
+a4421
+a4422
+a4423
+a4424
+a4425
+a4426
+a4427
+a4428
+a4429
+a4430
+a4431
+a4432
+a4433
+a4434
+a4435
+a4436
+a4437
+a4438
+a4439
+a4440
+a4441
+a4442
+a4443
+a4444
+a4445
+a4446
+a4447
+a4448
+a4449
+a4450
+a4451
+a4452
+a4453
+a4454
+a4455
+a4456
+a4457
+a4458
+a4459
+a4460
+a4461
+a4462
+a4463
+a4464
+a4465
+a4466
+a4467
+a4468
+a4469
+a4470
+a4471
+a4472
+a4473
+a4474
+a4475
+a4476
+a4477
+a4478
+a4479
+a4480
+a4481
+a4482
+a4483
+a4484
+a4485
+a4486
+a4487
+a4488
+a4489
+a4490
+a4491
+a4492
+a4493
+a4494
+a4495
+a4496
+a4497
+a4498
+a4499
+a4500

mit5k_ids_filepath/upe_uegan/images_test.txt

@@ -0,0 +1,500 @@
+a4501
+a4502
+a4503
+a4504
+a4505
+a4506
+a4507
+a4508
+a4509
+a4510
+a4511
+a4512
+a4513
+a4514
+a4515
+a4516
+a4517
+a4518
+a4519
+a4520
+a4521
+a4522
+a4523
+a4524
+a4525
+a4526
+a4527
+a4528
+a4529
+a4530
+a4531
+a4532
+a4533
+a4534
+a4535
+a4536
+a4537
+a4538
+a4539
+a4540
+a4541
+a4542
+a4543
+a4544
+a4545
+a4546
+a4547
+a4548
+a4549
+a4550
+a4551
+a4552
+a4553
+a4554
+a4555
+a4556
+a4557
+a4558
+a4559
+a4560
+a4561
+a4562
+a4563
+a4564
+a4565
+a4566
+a4567
+a4568
+a4569
+a4570
+a4571
+a4572
+a4573
+a4574
+a4575
+a4576
+a4577
+a4578
+a4579
+a4580
+a4581
+a4582
+a4583
+a4584
+a4585
+a4586
+a4587
+a4588
+a4589
+a4590
+a4591
+a4592
+a4593
+a4594
+a4595
+a4596
+a4597
+a4598
+a4599
+a4600
+a4601
+a4602
+a4603
+a4604
+a4605
+a4606
+a4607
+a4608
+a4609
+a4610
+a4611
+a4612
+a4613
+a4614
+a4615
+a4616
+a4617
+a4618
+a4619
+a4620
+a4621
+a4622
+a4623
+a4624
+a4625
+a4626
+a4627
+a4628
+a4629
+a4630
+a4631
+a4632
+a4633
+a4634
+a4635
+a4636
+a4637
+a4638
+a4639
+a4640
+a4641
+a4642
+a4643
+a4644
+a4645
+a4646
+a4647
+a4648
+a4649
+a4650
+a4651
+a4652
+a4653
+a4654
+a4655
+a4656
+a4657
+a4658
+a4659
+a4660
+a4661
+a4662
+a4663
+a4664
+a4665
+a4666
+a4667
+a4668
+a4669
+a4670
+a4671
+a4672
+a4673
+a4674
+a4675
+a4676
+a4677
+a4678
+a4679
+a4680
+a4681
+a4682
+a4683
+a4684
+a4685
+a4686
+a4687
+a4688
+a4689
+a4690
+a4691
+a4692
+a4693
+a4694
+a4695
+a4696
+a4697
+a4698
+a4699
+a4700
+a4701
+a4702
+a4703
+a4704
+a4705
+a4706
+a4707
+a4708
+a4709
+a4710
+a4711
+a4712
+a4713
+a4714
+a4715
+a4716
+a4717
+a4718
+a4719
+a4720
+a4721
+a4722
+a4723
+a4724
+a4725
+a4726
+a4727
+a4728
+a4729
+a4730
+a4731
+a4732
+a4733
+a4734
+a4735
+a4736
+a4737
+a4738
+a4739
+a4740
+a4741
+a4742
+a4743
+a4744
+a4745
+a4746
+a4747
+a4748
+a4749
+a4750
+a4751
+a4752
+a4753
+a4754
+a4755
+a4756
+a4757
+a4758
+a4759
+a4760
+a4761
+a4762
+a4763
+a4764
+a4765
+a4766
+a4767
+a4768
+a4769
+a4770
+a4771
+a4772
+a4773
+a4774
+a4775
+a4776
+a4777
+a4778
+a4779
+a4780
+a4781
+a4782
+a4783
+a4784
+a4785
+a4786
+a4787
+a4788
+a4789
+a4790
+a4791
+a4792
+a4793
+a4794
+a4795
+a4796
+a4797
+a4798
+a4799
+a4800
+a4801
+a4802
+a4803
+a4804
+a4805
+a4806
+a4807
+a4808
+a4809
+a4810
+a4811
+a4812
+a4813
+a4814
+a4815
+a4816
+a4817
+a4818
+a4819
+a4820
+a4821
+a4822
+a4823
+a4824
+a4825
+a4826
+a4827
+a4828
+a4829
+a4830
+a4831
+a4832
+a4833
+a4834
+a4835
+a4836
+a4837
+a4838
+a4839
+a4840
+a4841
+a4842
+a4843
+a4844
+a4845
+a4846
+a4847
+a4848
+a4849
+a4850
+a4851
+a4852
+a4853
+a4854
+a4855
+a4856
+a4857
+a4858
+a4859
+a4860
+a4861
+a4862
+a4863
+a4864
+a4865
+a4866
+a4867
+a4868
+a4869
+a4870
+a4871
+a4872
+a4873
+a4874
+a4875
+a4876
+a4877
+a4878
+a4879
+a4880
+a4881
+a4882
+a4883
+a4884
+a4885
+a4886
+a4887
+a4888
+a4889
+a4890
+a4891
+a4892
+a4893
+a4894
+a4895
+a4896
+a4897
+a4898
+a4899
+a4900
+a4901
+a4902
+a4903
+a4904
+a4905
+a4906
+a4907
+a4908
+a4909
+a4910
+a4911
+a4912
+a4913
+a4914
+a4915
+a4916
+a4917
+a4918
+a4919
+a4920
+a4921
+a4922
+a4923
+a4924
+a4925
+a4926
+a4927
+a4928
+a4929
+a4930
+a4931
+a4932
+a4933
+a4934
+a4935
+a4936
+a4937
+a4938
+a4939
+a4940
+a4941
+a4942
+a4943
+a4944
+a4945
+a4946
+a4947
+a4948
+a4949
+a4950
+a4951
+a4952
+a4953
+a4954
+a4955
+a4956
+a4957
+a4958
+a4959
+a4960
+a4961
+a4962
+a4963
+a4964
+a4965
+a4966
+a4967
+a4968
+a4969
+a4970
+a4971
+a4972
+a4973
+a4974
+a4975
+a4976
+a4977
+a4978
+a4979
+a4980
+a4981
+a4982
+a4983
+a4984
+a4985
+a4986
+a4987
+a4988
+a4989
+a4990
+a4991
+a4992
+a4993
+a4994
+a4995
+a4996
+a4997
+a4998
+a4999
+a5000

mit5k_ids_filepath/upe_uegan/images_train.txt

@@ -0,0 +1,4500 @@
+a0001
+a0002
+a0003
+a0004
+a0005
+a0006
+a0007
+a0008
+a0009
+a0010
+a0011
+a0012
+a0013
+a0014
+a0015
+a0016
+a0017
+a0018
+a0019
+a0020
+a0021
+a0022
+a0023
+a0024
+a0025
+a0026
+a0027
+a0028
+a0029
+a0030
+a0031
+a0032
+a0033
+a0034
+a0035
+a0036
+a0037
+a0038
+a0039
+a0040
+a0041
+a0042
+a0043
+a0044
+a0045
+a0046
+a0047
+a0048
+a0049
+a0050
+a0051
+a0052
+a0053
+a0054
+a0055
+a0056
+a0057
+a0058
+a0059
+a0060
+a0061
+a0062
+a0063
+a0064
+a0065
+a0066
+a0067
+a0068
+a0069
+a0070
+a0071
+a0072
+a0073
+a0074
+a0075
+a0076
+a0077
+a0078
+a0079
+a0080
+a0081
+a0082
+a0083
+a0084
+a0085
+a0086
+a0087
+a0088
+a0089
+a0090
+a0091
+a0092
+a0093
+a0094
+a0095
+a0096
+a0097
+a0098
+a0099
+a0100
+a0101
+a0102
+a0103
+a0104
+a0105
+a0106
+a0107
+a0108
+a0109
+a0110
+a0111
+a0112
+a0113
+a0114
+a0115
+a0116
+a0117
+a0118
+a0119
+a0120
+a0121
+a0122
+a0123
+a0124
+a0125
+a0126
+a0127
+a0128
+a0129
+a0130
+a0131
+a0132
+a0133
+a0134
+a0135
+a0136
+a0137
+a0138
+a0139
+a0140
+a0141
+a0142
+a0143
+a0144
+a0145
+a0146
+a0147
+a0148
+a0149
+a0150
+a0151
+a0152
+a0153
+a0154
+a0155
+a0156
+a0157
+a0158
+a0159
+a0160
+a0161
+a0162
+a0163
+a0164
+a0165
+a0166
+a0167
+a0168
+a0169
+a0170
+a0171
+a0172
+a0173
+a0174
+a0175
+a0176
+a0177
+a0178
+a0179
+a0180
+a0181
+a0182
+a0183
+a0184
+a0185
+a0186
+a0187
+a0188
+a0189
+a0190
+a0191
+a0192
+a0193
+a0194
+a0195
+a0196
+a0197
+a0198
+a0199
+a0200
+a0201
+a0202
+a0203
+a0204
+a0205
+a0206
+a0207
+a0208
+a0209
+a0210
+a0211
+a0212
+a0213
+a0214
+a0215
+a0216
+a0217
+a0218
+a0219
+a0220
+a0221
+a0222
+a0223
+a0224
+a0225
+a0226
+a0227
+a0228
+a0229
+a0230
+a0231
+a0232
+a0233
+a0234
+a0235
+a0236
+a0237
+a0238
+a0239
+a0240
+a0241
+a0242
+a0243
+a0244
+a0245
+a0246
+a0247
+a0248
+a0249
+a0250
+a0251
+a0252
+a0253
+a0254
+a0255
+a0256
+a0257
+a0258
+a0259
+a0260
+a0261
+a0262
+a0263
+a0264
+a0265
+a0266
+a0267
+a0268
+a0269
+a0270
+a0271
+a0272
+a0273
+a0274
+a0275
+a0276
+a0277
+a0278
+a0279
+a0280
+a0281
+a0282
+a0283
+a0284
+a0285
+a0286
+a0287
+a0288
+a0289
+a0290
+a0291
+a0292
+a0293
+a0294
+a0295
+a0296
+a0297
+a0298
+a0299
+a0300
+a0301
+a0302
+a0303
+a0304
+a0305
+a0306
+a0307
+a0308
+a0309
+a0310
+a0311
+a0312
+a0313
+a0314
+a0315
+a0316
+a0317
+a0318
+a0319
+a0320
+a0321
+a0322
+a0323
+a0324
+a0325
+a0326
+a0327
+a0328
+a0329
+a0330
+a0331
+a0332
+a0333
+a0334
+a0335
+a0336
+a0337
+a0338
+a0339
+a0340
+a0341
+a0342
+a0343
+a0344
+a0345
+a0346
+a0347
+a0348
+a0349
+a0350
+a0351
+a0352
+a0353
+a0354
+a0355
+a0356
+a0357
+a0358
+a0359
+a0360
+a0361
+a0362
+a0363
+a0364
+a0365
+a0366
+a0367
+a0368
+a0369
+a0370
+a0371
+a0372
+a0373
+a0374
+a0375
+a0376
+a0377
+a0378
+a0379
+a0380
+a0381
+a0382
+a0383
+a0384
+a0385
+a0386
+a0387
+a0388
+a0389
+a0390
+a0391
+a0392
+a0393
+a0394
+a0395
+a0396
+a0397
+a0398
+a0399
+a0400
+a0401
+a0402
+a0403
+a0404
+a0405
+a0406
+a0407
+a0408
+a0409
+a0410
+a0411
+a0412
+a0413
+a0414
+a0415
+a0416
+a0417
+a0418
+a0419
+a0420
+a0421
+a0422
+a0423
+a0424
+a0425
+a0426
+a0427
+a0428
+a0429
+a0430
+a0431
+a0432
+a0433
+a0434
+a0435
+a0436
+a0437
+a0438
+a0439
+a0440
+a0441
+a0442
+a0443
+a0444
+a0445
+a0446
+a0447
+a0448
+a0449
+a0450
+a0451
+a0452
+a0453
+a0454
+a0455
+a0456
+a0457
+a0458
+a0459
+a0460
+a0461
+a0462
+a0463
+a0464
+a0465
+a0466
+a0467
+a0468
+a0469
+a0470
+a0471
+a0472
+a0473
+a0474
+a0475
+a0476
+a0477
+a0478
+a0479
+a0480
+a0481
+a0482
+a0483
+a0484
+a0485
+a0486
+a0487
+a0488
+a0489
+a0490
+a0491
+a0492
+a0493
+a0494
+a0495
+a0496
+a0497
+a0498
+a0499
+a0500
+a0501
+a0502
+a0503
+a0504
+a0505
+a0506
+a0507
+a0508
+a0509
+a0510
+a0511
+a0512
+a0513
+a0514
+a0515
+a0516
+a0517
+a0518
+a0519
+a0520
+a0521
+a0522
+a0523
+a0524
+a0525
+a0526
+a0527
+a0528
+a0529
+a0530
+a0531
+a0532
+a0533
+a0534
+a0535
+a0536
+a0537
+a0538
+a0539
+a0540
+a0541
+a0542
+a0543
+a0544
+a0545
+a0546
+a0547
+a0548
+a0549
+a0550
+a0551
+a0552
+a0553
+a0554
+a0555
+a0556
+a0557
+a0558
+a0559
+a0560
+a0561
+a0562
+a0563
+a0564
+a0565
+a0566
+a0567
+a0568
+a0569
+a0570
+a0571
+a0572
+a0573
+a0574
+a0575
+a0576
+a0577
+a0578
+a0579
+a0580
+a0581
+a0582
+a0583
+a0584
+a0585
+a0586
+a0587
+a0588
+a0589
+a0590
+a0591
+a0592
+a0593
+a0594
+a0595
+a0596
+a0597
+a0598
+a0599
+a0600
+a0601
+a0602
+a0603
+a0604
+a0605
+a0606
+a0607
+a0608
+a0609
+a0610
+a0611
+a0612
+a0613
+a0614
+a0615
+a0616
+a0617
+a0618
+a0619
+a0620
+a0621
+a0622
+a0623
+a0624
+a0625
+a0626
+a0627
+a0628
+a0629
+a0630
+a0631
+a0632
+a0633
+a0634
+a0635
+a0636
+a0637
+a0638
+a0639
+a0640
+a0641
+a0642
+a0643
+a0644
+a0645
+a0646
+a0647
+a0648
+a0649
+a0650
+a0651
+a0652
+a0653
+a0654
+a0655
+a0656
+a0657
+a0658
+a0659
+a0660
+a0661
+a0662
+a0663
+a0664
+a0665
+a0666
+a0667
+a0668
+a0669
+a0670
+a0671
+a0672
+a0673
+a0674
+a0675
+a0676
+a0677
+a0678
+a0679
+a0680
+a0681
+a0682
+a0683
+a0684
+a0685
+a0686
+a0687
+a0688
+a0689
+a0690
+a0691
+a0692
+a0693
+a0694
+a0695
+a0696
+a0697
+a0698
+a0699
+a0700
+a0701
+a0702
+a0703
+a0704
+a0705
+a0706
+a0707
+a0708
+a0709
+a0710
+a0711
+a0712
+a0713
+a0714
+a0715
+a0716
+a0717
+a0718
+a0719
+a0720
+a0721
+a0722
+a0723
+a0724
+a0725
+a0726
+a0727
+a0728
+a0729
+a0730
+a0731
+a0732
+a0733
+a0734
+a0735
+a0736
+a0737
+a0738
+a0739
+a0740
+a0741
+a0742
+a0743
+a0744
+a0745
+a0746
+a0747
+a0748
+a0749
+a0750
+a0751
+a0752
+a0753
+a0754
+a0755
+a0756
+a0757
+a0758
+a0759
+a0760
+a0761
+a0762
+a0763
+a0764
+a0765
+a0766
+a0767
+a0768
+a0769
+a0770
+a0771
+a0772
+a0773
+a0774
+a0775
+a0776
+a0777
+a0778
+a0779
+a0780
+a0781
+a0782
+a0783
+a0784
+a0785
+a0786
+a0787
+a0788
+a0789
+a0790
+a0791
+a0792
+a0793
+a0794
+a0795
+a0796
+a0797
+a0798
+a0799
+a0800
+a0801
+a0802
+a0803
+a0804
+a0805
+a0806
+a0807
+a0808
+a0809
+a0810
+a0811
+a0812
+a0813
+a0814
+a0815
+a0816
+a0817
+a0818
+a0819
+a0820
+a0821
+a0822
+a0823
+a0824
+a0825
+a0826
+a0827
+a0828
+a0829
+a0830
+a0831
+a0832
+a0833
+a0834
+a0835
+a0836
+a0837
+a0838
+a0839
+a0840
+a0841
+a0842
+a0843
+a0844
+a0845
+a0846
+a0847
+a0848
+a0849
+a0850
+a0851
+a0852
+a0853
+a0854
+a0855
+a0856
+a0857
+a0858
+a0859
+a0860
+a0861
+a0862
+a0863
+a0864
+a0865
+a0866
+a0867
+a0868
+a0869
+a0870
+a0871
+a0872
+a0873
+a0874
+a0875
+a0876
+a0877
+a0878
+a0879
+a0880
+a0881
+a0882
+a0883
+a0884
+a0885
+a0886
+a0887
+a0888
+a0889
+a0890
+a0891
+a0892
+a0893
+a0894
+a0895
+a0896
+a0897
+a0898
+a0899
+a0900
+a0901
+a0902
+a0903
+a0904
+a0905
+a0906
+a0907
+a0908
+a0909
+a0910
+a0911
+a0912
+a0913
+a0914
+a0915
+a0916
+a0917
+a0918
+a0919
+a0920
+a0921
+a0922
+a0923
+a0924
+a0925
+a0926
+a0927
+a0928
+a0929
+a0930
+a0931
+a0932
+a0933
+a0934
+a0935
+a0936
+a0937
+a0938
+a0939
+a0940
+a0941
+a0942
+a0943
+a0944
+a0945
+a0946
+a0947
+a0948
+a0949
+a0950
+a0951
+a0952
+a0953
+a0954
+a0955
+a0956
+a0957
+a0958
+a0959
+a0960
+a0961
+a0962
+a0963
+a0964
+a0965
+a0966
+a0967
+a0968
+a0969
+a0970
+a0971
+a0972
+a0973
+a0974
+a0975
+a0976
+a0977
+a0978
+a0979
+a0980
+a0981
+a0982
+a0983
+a0984
+a0985
+a0986
+a0987
+a0988
+a0989
+a0990
+a0991
+a0992
+a0993
+a0994
+a0995
+a0996
+a0997
+a0998
+a0999
+a1000
+a1001
+a1002
+a1003
+a1004
+a1005
+a1006
+a1007
+a1008
+a1009
+a1010
+a1011
+a1012
+a1013
+a1014
+a1015
+a1016
+a1017
+a1018
+a1019
+a1020
+a1021
+a1022
+a1023
+a1024
+a1025
+a1026
+a1027
+a1028
+a1029
+a1030
+a1031
+a1032
+a1033
+a1034
+a1035
+a1036
+a1037
+a1038
+a1039
+a1040
+a1041
+a1042
+a1043
+a1044
+a1045
+a1046
+a1047
+a1048
+a1049
+a1050
+a1051
+a1052
+a1053
+a1054
+a1055
+a1056
+a1057
+a1058
+a1059
+a1060
+a1061
+a1062
+a1063
+a1064
+a1065
+a1066
+a1067
+a1068
+a1069
+a1070
+a1071
+a1072
+a1073
+a1074
+a1075
+a1076
+a1077
+a1078
+a1079
+a1080
+a1081
+a1082
+a1083
+a1084
+a1085
+a1086
+a1087
+a1088
+a1089
+a1090
+a1091
+a1092
+a1093
+a1094
+a1095
+a1096
+a1097
+a1098
+a1099
+a1100
+a1101
+a1102
+a1103
+a1104
+a1105
+a1106
+a1107
+a1108
+a1109
+a1110
+a1111
+a1112
+a1113
+a1114
+a1115
+a1116
+a1117
+a1118
+a1119
+a1120
+a1121
+a1122
+a1123
+a1124
+a1125
+a1126
+a1127
+a1128
+a1129
+a1130
+a1131
+a1132
+a1133
+a1134
+a1135
+a1136
+a1137
+a1138
+a1139
+a1140
+a1141
+a1142
+a1143
+a1144
+a1145
+a1146
+a1147
+a1148
+a1149
+a1150
+a1151
+a1152
+a1153
+a1154
+a1155
+a1156
+a1157
+a1158
+a1159
+a1160
+a1161
+a1162
+a1163
+a1164
+a1165
+a1166
+a1167
+a1168
+a1169
+a1170
+a1171
+a1172
+a1173
+a1174
+a1175
+a1176
+a1177
+a1178
+a1179
+a1180
+a1181
+a1182
+a1183
+a1184
+a1185
+a1186
+a1187
+a1188
+a1189
+a1190
+a1191
+a1192
+a1193
+a1194
+a1195
+a1196
+a1197
+a1198
+a1199
+a1200
+a1201
+a1202
+a1203
+a1204
+a1205
+a1206
+a1207
+a1208
+a1209
+a1210
+a1211
+a1212
+a1213
+a1214
+a1215
+a1216
+a1217
+a1218
+a1219
+a1220
+a1221
+a1222
+a1223
+a1224
+a1225
+a1226
+a1227
+a1228
+a1229
+a1230
+a1231
+a1232
+a1233
+a1234
+a1235
+a1236
+a1237
+a1238
+a1239
+a1240
+a1241
+a1242
+a1243
+a1244
+a1245
+a1246
+a1247
+a1248
+a1249
+a1250
+a1251
+a1252
+a1253
+a1254
+a1255
+a1256
+a1257
+a1258
+a1259
+a1260
+a1261
+a1262
+a1263
+a1264
+a1265
+a1266
+a1267
+a1268
+a1269
+a1270
+a1271
+a1272
+a1273
+a1274
+a1275
+a1276
+a1277
+a1278
+a1279
+a1280
+a1281
+a1282
+a1283
+a1284
+a1285
+a1286
+a1287
+a1288
+a1289
+a1290
+a1291
+a1292
+a1293
+a1294
+a1295
+a1296
+a1297
+a1298
+a1299
+a1300
+a1301
+a1302
+a1303
+a1304
+a1305
+a1306
+a1307
+a1308
+a1309
+a1310
+a1311
+a1312
+a1313
+a1314
+a1315
+a1316
+a1317
+a1318
+a1319
+a1320
+a1321
+a1322
+a1323
+a1324
+a1325
+a1326
+a1327
+a1328
+a1329
+a1330
+a1331
+a1332
+a1333
+a1334
+a1335
+a1336
+a1337
+a1338
+a1339
+a1340
+a1341
+a1342
+a1343
+a1344
+a1345
+a1346
+a1347
+a1348
+a1349
+a1350
+a1351
+a1352
+a1353
+a1354
+a1355
+a1356
+a1357
+a1358
+a1359
+a1360
+a1361
+a1362
+a1363
+a1364
+a1365
+a1366
+a1367
+a1368
+a1369
+a1370
+a1371
+a1372
+a1373
+a1374
+a1375
+a1376
+a1377
+a1378
+a1379
+a1380
+a1381
+a1382
+a1383
+a1384
+a1385
+a1386
+a1387
+a1388
+a1389
+a1390
+a1391
+a1392
+a1393
+a1394
+a1395
+a1396
+a1397
+a1398
+a1399
+a1400
+a1401
+a1402
+a1403
+a1404
+a1405
+a1406
+a1407
+a1408
+a1409
+a1410
+a1411
+a1412
+a1413
+a1414
+a1415
+a1416
+a1417
+a1418
+a1419
+a1420
+a1421
+a1422
+a1423
+a1424
+a1425
+a1426
+a1427
+a1428
+a1429
+a1430
+a1431
+a1432
+a1433
+a1434
+a1435
+a1436
+a1437
+a1438
+a1439
+a1440
+a1441
+a1442
+a1443
+a1444
+a1445
+a1446
+a1447
+a1448
+a1449
+a1450
+a1451
+a1452
+a1453
+a1454
+a1455
+a1456
+a1457
+a1458
+a1459
+a1460
+a1461
+a1462
+a1463
+a1464
+a1465
+a1466
+a1467
+a1468
+a1469
+a1470
+a1471
+a1472
+a1473
+a1474
+a1475
+a1476
+a1477
+a1478
+a1479
+a1480
+a1481
+a1482
+a1483
+a1484
+a1485
+a1486
+a1487
+a1488
+a1489
+a1490
+a1491
+a1492
+a1493
+a1494
+a1495
+a1496
+a1497
+a1498
+a1499
+a1500
+a1501
+a1502
+a1503
+a1504
+a1505
+a1506
+a1507
+a1508
+a1509
+a1510
+a1511
+a1512
+a1513
+a1514
+a1515
+a1516
+a1517
+a1518
+a1519
+a1520
+a1521
+a1522
+a1523
+a1524
+a1525
+a1526
+a1527
+a1528
+a1529
+a1530
+a1531
+a1532
+a1533
+a1534
+a1535
+a1536
+a1537
+a1538
+a1539
+a1540
+a1541
+a1542
+a1543
+a1544
+a1545
+a1546
+a1547
+a1548
+a1549
+a1550
+a1551
+a1552
+a1553
+a1554
+a1555
+a1556
+a1557
+a1558
+a1559
+a1560
+a1561
+a1562
+a1563
+a1564
+a1565
+a1566
+a1567
+a1568
+a1569
+a1570
+a1571
+a1572
+a1573
+a1574
+a1575
+a1576
+a1577
+a1578
+a1579
+a1580
+a1581
+a1582
+a1583
+a1584
+a1585
+a1586
+a1587
+a1588
+a1589
+a1590
+a1591
+a1592
+a1593
+a1594
+a1595
+a1596
+a1597
+a1598
+a1599
+a1600
+a1601
+a1602
+a1603
+a1604
+a1605
+a1606
+a1607
+a1608
+a1609
+a1610
+a1611
+a1612
+a1613
+a1614
+a1615
+a1616
+a1617
+a1618
+a1619
+a1620
+a1621
+a1622
+a1623
+a1624
+a1625
+a1626
+a1627
+a1628
+a1629
+a1630
+a1631
+a1632
+a1633
+a1634
+a1635
+a1636
+a1637
+a1638
+a1639
+a1640
+a1641
+a1642
+a1643
+a1644
+a1645
+a1646
+a1647
+a1648
+a1649
+a1650
+a1651
+a1652
+a1653
+a1654
+a1655
+a1656
+a1657
+a1658
+a1659
+a1660
+a1661
+a1662
+a1663
+a1664
+a1665
+a1666
+a1667
+a1668
+a1669
+a1670
+a1671
+a1672
+a1673
+a1674
+a1675
+a1676
+a1677
+a1678
+a1679
+a1680
+a1681
+a1682
+a1683
+a1684
+a1685
+a1686
+a1687
+a1688
+a1689
+a1690
+a1691
+a1692
+a1693
+a1694
+a1695
+a1696
+a1697
+a1698
+a1699
+a1700
+a1701
+a1702
+a1703
+a1704
+a1705
+a1706
+a1707
+a1708
+a1709
+a1710
+a1711
+a1712
+a1713
+a1714
+a1715
+a1716
+a1717
+a1718
+a1719
+a1720
+a1721
+a1722
+a1723
+a1724
+a1725
+a1726
+a1727
+a1728
+a1729
+a1730
+a1731
+a1732
+a1733
+a1734
+a1735
+a1736
+a1737
+a1738
+a1739
+a1740
+a1741
+a1742
+a1743
+a1744
+a1745
+a1746
+a1747
+a1748
+a1749
+a1750
+a1751
+a1752
+a1753
+a1754
+a1755
+a1756
+a1757
+a1758
+a1759
+a1760
+a1761
+a1762
+a1763
+a1764
+a1765
+a1766
+a1767
+a1768
+a1769
+a1770
+a1771
+a1772
+a1773
+a1774
+a1775
+a1776
+a1777
+a1778
+a1779
+a1780
+a1781
+a1782
+a1783
+a1784
+a1785
+a1786
+a1787
+a1788
+a1789
+a1790
+a1791
+a1792
+a1793
+a1794
+a1795
+a1796
+a1797
+a1798
+a1799
+a1800
+a1801
+a1802
+a1803
+a1804
+a1805
+a1806
+a1807
+a1808
+a1809
+a1810
+a1811
+a1812
+a1813
+a1814
+a1815
+a1816
+a1817
+a1818
+a1819
+a1820
+a1821
+a1822
+a1823
+a1824
+a1825
+a1826
+a1827
+a1828
+a1829
+a1830
+a1831
+a1832
+a1833
+a1834
+a1835
+a1836
+a1837
+a1838
+a1839
+a1840
+a1841
+a1842
+a1843
+a1844
+a1845
+a1846
+a1847
+a1848
+a1849
+a1850
+a1851
+a1852
+a1853
+a1854
+a1855
+a1856
+a1857
+a1858
+a1859
+a1860
+a1861
+a1862
+a1863
+a1864
+a1865
+a1866
+a1867
+a1868
+a1869
+a1870
+a1871
+a1872
+a1873
+a1874
+a1875
+a1876
+a1877
+a1878
+a1879
+a1880
+a1881
+a1882
+a1883
+a1884
+a1885
+a1886
+a1887
+a1888
+a1889
+a1890
+a1891
+a1892
+a1893
+a1894
+a1895
+a1896
+a1897
+a1898
+a1899
+a1900
+a1901
+a1902
+a1903
+a1904
+a1905
+a1906
+a1907
+a1908
+a1909
+a1910
+a1911
+a1912
+a1913
+a1914
+a1915
+a1916
+a1917
+a1918
+a1919
+a1920
+a1921
+a1922
+a1923
+a1924
+a1925
+a1926
+a1927
+a1928
+a1929
+a1930
+a1931
+a1932
+a1933
+a1934
+a1935
+a1936
+a1937
+a1938
+a1939
+a1940
+a1941
+a1942
+a1943
+a1944
+a1945
+a1946
+a1947
+a1948
+a1949
+a1950
+a1951
+a1952
+a1953
+a1954
+a1955
+a1956
+a1957
+a1958
+a1959
+a1960
+a1961
+a1962
+a1963
+a1964
+a1965
+a1966
+a1967
+a1968
+a1969
+a1970
+a1971
+a1972
+a1973
+a1974
+a1975
+a1976
+a1977
+a1978
+a1979
+a1980
+a1981
+a1982
+a1983
+a1984
+a1985
+a1986
+a1987
+a1988
+a1989
+a1990
+a1991
+a1992
+a1993
+a1994
+a1995
+a1996
+a1997
+a1998
+a1999
+a2000
+a2001
+a2002
+a2003
+a2004
+a2005
+a2006
+a2007
+a2008
+a2009
+a2010
+a2011
+a2012
+a2013
+a2014
+a2015
+a2016
+a2017
+a2018
+a2019
+a2020
+a2021
+a2022
+a2023
+a2024
+a2025
+a2026
+a2027
+a2028
+a2029
+a2030
+a2031
+a2032
+a2033
+a2034
+a2035
+a2036
+a2037
+a2038
+a2039
+a2040
+a2041
+a2042
+a2043
+a2044
+a2045
+a2046
+a2047
+a2048
+a2049
+a2050
+a2051
+a2052
+a2053
+a2054
+a2055
+a2056
+a2057
+a2058
+a2059
+a2060
+a2061
+a2062
+a2063
+a2064
+a2065
+a2066
+a2067
+a2068
+a2069
+a2070
+a2071
+a2072
+a2073
+a2074
+a2075
+a2076
+a2077
+a2078
+a2079
+a2080
+a2081
+a2082
+a2083
+a2084
+a2085
+a2086
+a2087
+a2088
+a2089
+a2090
+a2091
+a2092
+a2093
+a2094
+a2095
+a2096
+a2097
+a2098
+a2099
+a2100
+a2101
+a2102
+a2103
+a2104
+a2105
+a2106
+a2107
+a2108
+a2109
+a2110
+a2111
+a2112
+a2113
+a2114
+a2115
+a2116
+a2117
+a2118
+a2119
+a2120
+a2121
+a2122
+a2123
+a2124
+a2125
+a2126
+a2127
+a2128
+a2129
+a2130
+a2131
+a2132
+a2133
+a2134
+a2135
+a2136
+a2137
+a2138
+a2139
+a2140
+a2141
+a2142
+a2143
+a2144
+a2145
+a2146
+a2147
+a2148
+a2149
+a2150
+a2151
+a2152
+a2153
+a2154
+a2155
+a2156
+a2157
+a2158
+a2159
+a2160
+a2161
+a2162
+a2163
+a2164
+a2165
+a2166
+a2167
+a2168
+a2169
+a2170
+a2171
+a2172
+a2173
+a2174
+a2175
+a2176
+a2177
+a2178
+a2179
+a2180
+a2181
+a2182
+a2183
+a2184
+a2185
+a2186
+a2187
+a2188
+a2189
+a2190
+a2191
+a2192
+a2193
+a2194
+a2195
+a2196
+a2197
+a2198
+a2199
+a2200
+a2201
+a2202
+a2203
+a2204
+a2205
+a2206
+a2207
+a2208
+a2209
+a2210
+a2211
+a2212
+a2213
+a2214
+a2215
+a2216
+a2217
+a2218
+a2219
+a2220
+a2221
+a2222
+a2223
+a2224
+a2225
+a2226
+a2227
+a2228
+a2229
+a2230
+a2231
+a2232
+a2233
+a2234
+a2235
+a2236
+a2237
+a2238
+a2239
+a2240
+a2241
+a2242
+a2243
+a2244
+a2245
+a2246
+a2247
+a2248
+a2249
+a2250
+a2251
+a2252
+a2253
+a2254
+a2255
+a2256
+a2257
+a2258
+a2259
+a2260
+a2261
+a2262
+a2263
+a2264
+a2265
+a2266
+a2267
+a2268
+a2269
+a2270
+a2271
+a2272
+a2273
+a2274
+a2275
+a2276
+a2277
+a2278
+a2279
+a2280
+a2281
+a2282
+a2283
+a2284
+a2285
+a2286
+a2287
+a2288
+a2289
+a2290
+a2291
+a2292
+a2293
+a2294
+a2295
+a2296
+a2297
+a2298
+a2299
+a2300
+a2301
+a2302
+a2303
+a2304
+a2305
+a2306
+a2307
+a2308
+a2309
+a2310
+a2311
+a2312
+a2313
+a2314
+a2315
+a2316
+a2317
+a2318
+a2319
+a2320
+a2321
+a2322
+a2323
+a2324
+a2325
+a2326
+a2327
+a2328
+a2329
+a2330
+a2331
+a2332
+a2333
+a2334
+a2335
+a2336
+a2337
+a2338
+a2339
+a2340
+a2341
+a2342
+a2343
+a2344
+a2345
+a2346
+a2347
+a2348
+a2349
+a2350
+a2351
+a2352
+a2353
+a2354
+a2355
+a2356
+a2357
+a2358
+a2359
+a2360
+a2361
+a2362
+a2363
+a2364
+a2365
+a2366
+a2367
+a2368
+a2369
+a2370
+a2371
+a2372
+a2373
+a2374
+a2375
+a2376
+a2377
+a2378
+a2379
+a2380
+a2381
+a2382
+a2383
+a2384
+a2385
+a2386
+a2387
+a2388
+a2389
+a2390
+a2391
+a2392
+a2393
+a2394
+a2395
+a2396
+a2397
+a2398
+a2399
+a2400
+a2401
+a2402
+a2403
+a2404
+a2405
+a2406
+a2407
+a2408
+a2409
+a2410
+a2411
+a2412
+a2413
+a2414
+a2415
+a2416
+a2417
+a2418
+a2419
+a2420
+a2421
+a2422
+a2423
+a2424
+a2425
+a2426
+a2427
+a2428
+a2429
+a2430
+a2431
+a2432
+a2433
+a2434
+a2435
+a2436
+a2437
+a2438
+a2439
+a2440
+a2441
+a2442
+a2443
+a2444
+a2445
+a2446
+a2447
+a2448
+a2449
+a2450
+a2451
+a2452
+a2453
+a2454
+a2455
+a2456
+a2457
+a2458
+a2459
+a2460
+a2461
+a2462
+a2463
+a2464
+a2465
+a2466
+a2467
+a2468
+a2469
+a2470
+a2471
+a2472
+a2473
+a2474
+a2475
+a2476
+a2477
+a2478
+a2479
+a2480
+a2481
+a2482
+a2483
+a2484
+a2485
+a2486
+a2487
+a2488
+a2489
+a2490
+a2491
+a2492
+a2493
+a2494
+a2495
+a2496
+a2497
+a2498
+a2499
+a2500
+a2501
+a2502
+a2503
+a2504
+a2505
+a2506
+a2507
+a2508
+a2509
+a2510
+a2511
+a2512
+a2513
+a2514
+a2515
+a2516
+a2517
+a2518
+a2519
+a2520
+a2521
+a2522
+a2523
+a2524
+a2525
+a2526
+a2527
+a2528
+a2529
+a2530
+a2531
+a2532
+a2533
+a2534
+a2535
+a2536
+a2537
+a2538
+a2539
+a2540
+a2541
+a2542
+a2543
+a2544
+a2545
+a2546
+a2547
+a2548
+a2549
+a2550
+a2551
+a2552
+a2553
+a2554
+a2555
+a2556
+a2557
+a2558
+a2559
+a2560
+a2561
+a2562
+a2563
+a2564
+a2565
+a2566
+a2567
+a2568
+a2569
+a2570
+a2571
+a2572
+a2573
+a2574
+a2575
+a2576
+a2577
+a2578
+a2579
+a2580
+a2581
+a2582
+a2583
+a2584
+a2585
+a2586
+a2587
+a2588
+a2589
+a2590
+a2591
+a2592
+a2593
+a2594
+a2595
+a2596
+a2597
+a2598
+a2599
+a2600
+a2601
+a2602
+a2603
+a2604
+a2605
+a2606
+a2607
+a2608
+a2609
+a2610
+a2611
+a2612
+a2613
+a2614
+a2615
+a2616
+a2617
+a2618
+a2619
+a2620
+a2621
+a2622
+a2623
+a2624
+a2625
+a2626
+a2627
+a2628
+a2629
+a2630
+a2631
+a2632
+a2633
+a2634
+a2635
+a2636
+a2637
+a2638
+a2639
+a2640
+a2641
+a2642
+a2643
+a2644
+a2645
+a2646
+a2647
+a2648
+a2649
+a2650
+a2651
+a2652
+a2653
+a2654
+a2655
+a2656
+a2657
+a2658
+a2659
+a2660
+a2661
+a2662
+a2663
+a2664
+a2665
+a2666
+a2667
+a2668
+a2669
+a2670
+a2671
+a2672
+a2673
+a2674
+a2675
+a2676
+a2677
+a2678
+a2679
+a2680
+a2681
+a2682
+a2683
+a2684
+a2685
+a2686
+a2687
+a2688
+a2689
+a2690
+a2691
+a2692
+a2693
+a2694
+a2695
+a2696
+a2697
+a2698
+a2699
+a2700
+a2701
+a2702
+a2703
+a2704
+a2705
+a2706
+a2707
+a2708
+a2709
+a2710
+a2711
+a2712
+a2713
+a2714
+a2715
+a2716
+a2717
+a2718
+a2719
+a2720
+a2721
+a2722
+a2723
+a2724
+a2725
+a2726
+a2727
+a2728
+a2729
+a2730
+a2731
+a2732
+a2733
+a2734
+a2735
+a2736
+a2737
+a2738
+a2739
+a2740
+a2741
+a2742
+a2743
+a2744
+a2745
+a2746
+a2747
+a2748
+a2749
+a2750
+a2751
+a2752
+a2753
+a2754
+a2755
+a2756
+a2757
+a2758
+a2759
+a2760
+a2761
+a2762
+a2763
+a2764
+a2765
+a2766
+a2767
+a2768
+a2769
+a2770
+a2771
+a2772
+a2773
+a2774
+a2775
+a2776
+a2777
+a2778
+a2779
+a2780
+a2781
+a2782
+a2783
+a2784
+a2785
+a2786
+a2787
+a2788
+a2789
+a2790
+a2791
+a2792
+a2793
+a2794
+a2795
+a2796
+a2797
+a2798
+a2799
+a2800
+a2801
+a2802
+a2803
+a2804
+a2805
+a2806
+a2807
+a2808
+a2809
+a2810
+a2811
+a2812
+a2813
+a2814
+a2815
+a2816
+a2817
+a2818
+a2819
+a2820
+a2821
+a2822
+a2823
+a2824
+a2825
+a2826
+a2827
+a2828
+a2829
+a2830
+a2831
+a2832
+a2833
+a2834
+a2835
+a2836
+a2837
+a2838
+a2839
+a2840
+a2841
+a2842
+a2843
+a2844
+a2845
+a2846
+a2847
+a2848
+a2849
+a2850
+a2851
+a2852
+a2853
+a2854
+a2855
+a2856
+a2857
+a2858
+a2859
+a2860
+a2861
+a2862
+a2863
+a2864
+a2865
+a2866
+a2867
+a2868
+a2869
+a2870
+a2871
+a2872
+a2873
+a2874
+a2875
+a2876
+a2877
+a2878
+a2879
+a2880
+a2881
+a2882
+a2883
+a2884
+a2885
+a2886
+a2887
+a2888
+a2889
+a2890
+a2891
+a2892
+a2893
+a2894
+a2895
+a2896
+a2897
+a2898
+a2899
+a2900
+a2901
+a2902
+a2903
+a2904
+a2905
+a2906
+a2907
+a2908
+a2909
+a2910
+a2911
+a2912
+a2913
+a2914
+a2915
+a2916
+a2917
+a2918
+a2919
+a2920
+a2921
+a2922
+a2923
+a2924
+a2925
+a2926
+a2927
+a2928
+a2929
+a2930
+a2931
+a2932
+a2933
+a2934
+a2935
+a2936
+a2937
+a2938
+a2939
+a2940
+a2941
+a2942
+a2943
+a2944
+a2945
+a2946
+a2947
+a2948
+a2949
+a2950
+a2951
+a2952
+a2953
+a2954
+a2955
+a2956
+a2957
+a2958
+a2959
+a2960
+a2961
+a2962
+a2963
+a2964
+a2965
+a2966
+a2967
+a2968
+a2969
+a2970
+a2971
+a2972
+a2973
+a2974
+a2975
+a2976
+a2977
+a2978
+a2979
+a2980
+a2981
+a2982
+a2983
+a2984
+a2985
+a2986
+a2987
+a2988
+a2989
+a2990
+a2991
+a2992
+a2993
+a2994
+a2995
+a2996
+a2997
+a2998
+a2999
+a3000
+a3001
+a3002
+a3003
+a3004
+a3005
+a3006
+a3007
+a3008
+a3009
+a3010
+a3011
+a3012
+a3013
+a3014
+a3015
+a3016
+a3017
+a3018
+a3019
+a3020
+a3021
+a3022
+a3023
+a3024
+a3025
+a3026
+a3027
+a3028
+a3029
+a3030
+a3031
+a3032
+a3033
+a3034
+a3035
+a3036
+a3037
+a3038
+a3039
+a3040
+a3041
+a3042
+a3043
+a3044
+a3045
+a3046
+a3047
+a3048
+a3049
+a3050
+a3051
+a3052
+a3053
+a3054
+a3055
+a3056
+a3057
+a3058
+a3059
+a3060
+a3061
+a3062
+a3063
+a3064
+a3065
+a3066
+a3067
+a3068
+a3069
+a3070
+a3071
+a3072
+a3073
+a3074
+a3075
+a3076
+a3077
+a3078
+a3079
+a3080
+a3081
+a3082
+a3083
+a3084
+a3085
+a3086
+a3087
+a3088
+a3089
+a3090
+a3091
+a3092
+a3093
+a3094
+a3095
+a3096
+a3097
+a3098
+a3099
+a3100
+a3101
+a3102
+a3103
+a3104
+a3105
+a3106
+a3107
+a3108
+a3109
+a3110
+a3111
+a3112
+a3113
+a3114
+a3115
+a3116
+a3117
+a3118
+a3119
+a3120
+a3121
+a3122
+a3123
+a3124
+a3125
+a3126
+a3127
+a3128
+a3129
+a3130
+a3131
+a3132
+a3133
+a3134
+a3135
+a3136
+a3137
+a3138
+a3139
+a3140
+a3141
+a3142
+a3143
+a3144
+a3145
+a3146
+a3147
+a3148
+a3149
+a3150
+a3151
+a3152
+a3153
+a3154
+a3155
+a3156
+a3157
+a3158
+a3159
+a3160
+a3161
+a3162
+a3163
+a3164
+a3165
+a3166
+a3167
+a3168
+a3169
+a3170
+a3171
+a3172
+a3173
+a3174
+a3175
+a3176
+a3177
+a3178
+a3179
+a3180
+a3181
+a3182
+a3183
+a3184
+a3185
+a3186
+a3187
+a3188
+a3189
+a3190
+a3191
+a3192
+a3193
+a3194
+a3195
+a3196
+a3197
+a3198
+a3199
+a3200
+a3201
+a3202
+a3203
+a3204
+a3205
+a3206
+a3207
+a3208
+a3209
+a3210
+a3211
+a3212
+a3213
+a3214
+a3215
+a3216
+a3217
+a3218
+a3219
+a3220
+a3221
+a3222
+a3223
+a3224
+a3225
+a3226
+a3227
+a3228
+a3229
+a3230
+a3231
+a3232
+a3233
+a3234
+a3235
+a3236
+a3237
+a3238
+a3239
+a3240
+a3241
+a3242
+a3243
+a3244
+a3245
+a3246
+a3247
+a3248
+a3249
+a3250
+a3251
+a3252
+a3253
+a3254
+a3255
+a3256
+a3257
+a3258
+a3259
+a3260
+a3261
+a3262
+a3263
+a3264
+a3265
+a3266
+a3267
+a3268
+a3269
+a3270
+a3271
+a3272
+a3273
+a3274
+a3275
+a3276
+a3277
+a3278
+a3279
+a3280
+a3281
+a3282
+a3283
+a3284
+a3285
+a3286
+a3287
+a3288
+a3289
+a3290
+a3291
+a3292
+a3293
+a3294
+a3295
+a3296
+a3297
+a3298
+a3299
+a3300
+a3301
+a3302
+a3303
+a3304
+a3305
+a3306
+a3307
+a3308
+a3309
+a3310
+a3311
+a3312
+a3313
+a3314
+a3315
+a3316
+a3317
+a3318
+a3319
+a3320
+a3321
+a3322
+a3323
+a3324
+a3325
+a3326
+a3327
+a3328
+a3329
+a3330
+a3331
+a3332
+a3333
+a3334
+a3335
+a3336
+a3337
+a3338
+a3339
+a3340
+a3341
+a3342
+a3343
+a3344
+a3345
+a3346
+a3347
+a3348
+a3349
+a3350
+a3351
+a3352
+a3353
+a3354
+a3355
+a3356
+a3357
+a3358
+a3359
+a3360
+a3361
+a3362
+a3363
+a3364
+a3365
+a3366
+a3367
+a3368
+a3369
+a3370
+a3371
+a3372
+a3373
+a3374
+a3375
+a3376
+a3377
+a3378
+a3379
+a3380
+a3381
+a3382
+a3383
+a3384
+a3385
+a3386
+a3387
+a3388
+a3389
+a3390
+a3391
+a3392
+a3393
+a3394
+a3395
+a3396
+a3397
+a3398
+a3399
+a3400
+a3401
+a3402
+a3403
+a3404
+a3405
+a3406
+a3407
+a3408
+a3409
+a3410
+a3411
+a3412
+a3413
+a3414
+a3415
+a3416
+a3417
+a3418
+a3419
+a3420
+a3421
+a3422
+a3423
+a3424
+a3425
+a3426
+a3427
+a3428
+a3429
+a3430
+a3431
+a3432
+a3433
+a3434
+a3435
+a3436
+a3437
+a3438
+a3439
+a3440
+a3441
+a3442
+a3443
+a3444
+a3445
+a3446
+a3447
+a3448
+a3449
+a3450
+a3451
+a3452
+a3453
+a3454
+a3455
+a3456
+a3457
+a3458
+a3459
+a3460
+a3461
+a3462
+a3463
+a3464
+a3465
+a3466
+a3467
+a3468
+a3469
+a3470
+a3471
+a3472
+a3473
+a3474
+a3475
+a3476
+a3477
+a3478
+a3479
+a3480
+a3481
+a3482
+a3483
+a3484
+a3485
+a3486
+a3487
+a3488
+a3489
+a3490
+a3491
+a3492
+a3493
+a3494
+a3495
+a3496
+a3497
+a3498
+a3499
+a3500
+a3501
+a3502
+a3503
+a3504
+a3505
+a3506
+a3507
+a3508
+a3509
+a3510
+a3511
+a3512
+a3513
+a3514
+a3515
+a3516
+a3517
+a3518
+a3519
+a3520
+a3521
+a3522
+a3523
+a3524
+a3525
+a3526
+a3527
+a3528
+a3529
+a3530
+a3531
+a3532
+a3533
+a3534
+a3535
+a3536
+a3537
+a3538
+a3539
+a3540
+a3541
+a3542
+a3543
+a3544
+a3545
+a3546
+a3547
+a3548
+a3549
+a3550
+a3551
+a3552
+a3553
+a3554
+a3555
+a3556
+a3557
+a3558
+a3559
+a3560
+a3561
+a3562
+a3563
+a3564
+a3565
+a3566
+a3567
+a3568
+a3569
+a3570
+a3571
+a3572
+a3573
+a3574
+a3575
+a3576
+a3577
+a3578
+a3579
+a3580
+a3581
+a3582
+a3583
+a3584
+a3585
+a3586
+a3587
+a3588
+a3589
+a3590
+a3591
+a3592
+a3593
+a3594
+a3595
+a3596
+a3597
+a3598
+a3599
+a3600
+a3601
+a3602
+a3603
+a3604
+a3605
+a3606
+a3607
+a3608
+a3609
+a3610
+a3611
+a3612
+a3613
+a3614
+a3615
+a3616
+a3617
+a3618
+a3619
+a3620
+a3621
+a3622
+a3623
+a3624
+a3625
+a3626
+a3627
+a3628
+a3629
+a3630
+a3631
+a3632
+a3633
+a3634
+a3635
+a3636
+a3637
+a3638
+a3639
+a3640
+a3641
+a3642
+a3643
+a3644
+a3645
+a3646
+a3647
+a3648
+a3649
+a3650
+a3651
+a3652
+a3653
+a3654
+a3655
+a3656
+a3657
+a3658
+a3659
+a3660
+a3661
+a3662
+a3663
+a3664
+a3665
+a3666
+a3667
+a3668
+a3669
+a3670
+a3671
+a3672
+a3673
+a3674
+a3675
+a3676
+a3677
+a3678
+a3679
+a3680
+a3681
+a3682
+a3683
+a3684
+a3685
+a3686
+a3687
+a3688
+a3689
+a3690
+a3691
+a3692
+a3693
+a3694
+a3695
+a3696
+a3697
+a3698
+a3699
+a3700
+a3701
+a3702
+a3703
+a3704
+a3705
+a3706
+a3707
+a3708
+a3709
+a3710
+a3711
+a3712
+a3713
+a3714
+a3715
+a3716
+a3717
+a3718
+a3719
+a3720
+a3721
+a3722
+a3723
+a3724
+a3725
+a3726
+a3727
+a3728
+a3729
+a3730
+a3731
+a3732
+a3733
+a3734
+a3735
+a3736
+a3737
+a3738
+a3739
+a3740
+a3741
+a3742
+a3743
+a3744
+a3745
+a3746
+a3747
+a3748
+a3749
+a3750
+a3751
+a3752
+a3753
+a3754
+a3755
+a3756
+a3757
+a3758
+a3759
+a3760
+a3761
+a3762
+a3763
+a3764
+a3765
+a3766
+a3767
+a3768
+a3769
+a3770
+a3771
+a3772
+a3773
+a3774
+a3775
+a3776
+a3777
+a3778
+a3779
+a3780
+a3781
+a3782
+a3783
+a3784
+a3785
+a3786
+a3787
+a3788
+a3789
+a3790
+a3791
+a3792
+a3793
+a3794
+a3795
+a3796
+a3797
+a3798
+a3799
+a3800
+a3801
+a3802
+a3803
+a3804
+a3805
+a3806
+a3807
+a3808
+a3809
+a3810
+a3811
+a3812
+a3813
+a3814
+a3815
+a3816
+a3817
+a3818
+a3819
+a3820
+a3821
+a3822
+a3823
+a3824
+a3825
+a3826
+a3827
+a3828
+a3829
+a3830
+a3831
+a3832
+a3833
+a3834
+a3835
+a3836
+a3837
+a3838
+a3839
+a3840
+a3841
+a3842
+a3843
+a3844
+a3845
+a3846
+a3847
+a3848
+a3849
+a3850
+a3851
+a3852
+a3853
+a3854
+a3855
+a3856
+a3857
+a3858
+a3859
+a3860
+a3861
+a3862
+a3863
+a3864
+a3865
+a3866
+a3867
+a3868
+a3869
+a3870
+a3871
+a3872
+a3873
+a3874
+a3875
+a3876
+a3877
+a3878
+a3879
+a3880
+a3881
+a3882
+a3883
+a3884
+a3885
+a3886
+a3887
+a3888
+a3889
+a3890
+a3891
+a3892
+a3893
+a3894
+a3895
+a3896
+a3897
+a3898
+a3899
+a3900
+a3901
+a3902
+a3903
+a3904
+a3905
+a3906
+a3907
+a3908
+a3909
+a3910
+a3911
+a3912
+a3913
+a3914
+a3915
+a3916
+a3917
+a3918
+a3919
+a3920
+a3921
+a3922
+a3923
+a3924
+a3925
+a3926
+a3927
+a3928
+a3929
+a3930
+a3931
+a3932
+a3933
+a3934
+a3935
+a3936
+a3937
+a3938
+a3939
+a3940
+a3941
+a3942
+a3943
+a3944
+a3945
+a3946
+a3947
+a3948
+a3949
+a3950
+a3951
+a3952
+a3953
+a3954
+a3955
+a3956
+a3957
+a3958
+a3959
+a3960
+a3961
+a3962
+a3963
+a3964
+a3965
+a3966
+a3967
+a3968
+a3969
+a3970
+a3971
+a3972
+a3973
+a3974
+a3975
+a3976
+a3977
+a3978
+a3979
+a3980
+a3981
+a3982
+a3983
+a3984
+a3985
+a3986
+a3987
+a3988
+a3989
+a3990
+a3991
+a3992
+a3993
+a3994
+a3995
+a3996
+a3997
+a3998
+a3999
+a4000
+a4001
+a4002
+a4003
+a4004
+a4005
+a4006
+a4007
+a4008
+a4009
+a4010
+a4011
+a4012
+a4013
+a4014
+a4015
+a4016
+a4017
+a4018
+a4019
+a4020
+a4021
+a4022
+a4023
+a4024
+a4025
+a4026
+a4027
+a4028
+a4029
+a4030
+a4031
+a4032
+a4033
+a4034
+a4035
+a4036
+a4037
+a4038
+a4039
+a4040
+a4041
+a4042
+a4043
+a4044
+a4045
+a4046
+a4047
+a4048
+a4049
+a4050
+a4051
+a4052
+a4053
+a4054
+a4055
+a4056
+a4057
+a4058
+a4059
+a4060
+a4061
+a4062
+a4063
+a4064
+a4065
+a4066
+a4067
+a4068
+a4069
+a4070
+a4071
+a4072
+a4073
+a4074
+a4075
+a4076
+a4077
+a4078
+a4079
+a4080
+a4081
+a4082
+a4083
+a4084
+a4085
+a4086
+a4087
+a4088
+a4089
+a4090
+a4091
+a4092
+a4093
+a4094
+a4095
+a4096
+a4097
+a4098
+a4099
+a4100
+a4101
+a4102
+a4103
+a4104
+a4105
+a4106
+a4107
+a4108
+a4109
+a4110
+a4111
+a4112
+a4113
+a4114
+a4115
+a4116
+a4117
+a4118
+a4119
+a4120
+a4121
+a4122
+a4123
+a4124
+a4125
+a4126
+a4127
+a4128
+a4129
+a4130
+a4131
+a4132
+a4133
+a4134
+a4135
+a4136
+a4137
+a4138
+a4139
+a4140
+a4141
+a4142
+a4143
+a4144
+a4145
+a4146
+a4147
+a4148
+a4149
+a4150
+a4151
+a4152
+a4153
+a4154
+a4155
+a4156
+a4157
+a4158
+a4159
+a4160
+a4161
+a4162
+a4163
+a4164
+a4165
+a4166
+a4167
+a4168
+a4169
+a4170
+a4171
+a4172
+a4173
+a4174
+a4175
+a4176
+a4177
+a4178
+a4179
+a4180
+a4181
+a4182
+a4183
+a4184
+a4185
+a4186
+a4187
+a4188
+a4189
+a4190
+a4191
+a4192
+a4193
+a4194
+a4195
+a4196
+a4197
+a4198
+a4199
+a4200
+a4201
+a4202
+a4203
+a4204
+a4205
+a4206
+a4207
+a4208
+a4209
+a4210
+a4211
+a4212
+a4213
+a4214
+a4215
+a4216
+a4217
+a4218
+a4219
+a4220
+a4221
+a4222
+a4223
+a4224
+a4225
+a4226
+a4227
+a4228
+a4229
+a4230
+a4231
+a4232
+a4233
+a4234
+a4235
+a4236
+a4237
+a4238
+a4239
+a4240
+a4241
+a4242
+a4243
+a4244
+a4245
+a4246
+a4247
+a4248
+a4249
+a4250
+a4251
+a4252
+a4253
+a4254
+a4255
+a4256
+a4257
+a4258
+a4259
+a4260
+a4261
+a4262
+a4263
+a4264
+a4265
+a4266
+a4267
+a4268
+a4269
+a4270
+a4271
+a4272
+a4273
+a4274
+a4275
+a4276
+a4277
+a4278
+a4279
+a4280
+a4281
+a4282
+a4283
+a4284
+a4285
+a4286
+a4287
+a4288
+a4289
+a4290
+a4291
+a4292
+a4293
+a4294
+a4295
+a4296
+a4297
+a4298
+a4299
+a4300
+a4301
+a4302
+a4303
+a4304
+a4305
+a4306
+a4307
+a4308
+a4309
+a4310
+a4311
+a4312
+a4313
+a4314
+a4315
+a4316
+a4317
+a4318
+a4319
+a4320
+a4321
+a4322
+a4323
+a4324
+a4325
+a4326
+a4327
+a4328
+a4329
+a4330
+a4331
+a4332
+a4333
+a4334
+a4335
+a4336
+a4337
+a4338
+a4339
+a4340
+a4341
+a4342
+a4343
+a4344
+a4345
+a4346
+a4347
+a4348
+a4349
+a4350
+a4351
+a4352
+a4353
+a4354
+a4355
+a4356
+a4357
+a4358
+a4359
+a4360
+a4361
+a4362
+a4363
+a4364
+a4365
+a4366
+a4367
+a4368
+a4369
+a4370
+a4371
+a4372
+a4373
+a4374
+a4375
+a4376
+a4377
+a4378
+a4379
+a4380
+a4381
+a4382
+a4383
+a4384
+a4385
+a4386
+a4387
+a4388
+a4389
+a4390
+a4391
+a4392
+a4393
+a4394
+a4395
+a4396
+a4397
+a4398
+a4399
+a4400
+a4401
+a4402
+a4403
+a4404
+a4405
+a4406
+a4407
+a4408
+a4409
+a4410
+a4411
+a4412
+a4413
+a4414
+a4415
+a4416
+a4417
+a4418
+a4419
+a4420
+a4421
+a4422
+a4423
+a4424
+a4425
+a4426
+a4427
+a4428
+a4429
+a4430
+a4431
+a4432
+a4433
+a4434
+a4435
+a4436
+a4437
+a4438
+a4439
+a4440
+a4441
+a4442
+a4443
+a4444
+a4445
+a4446
+a4447
+a4448
+a4449
+a4450
+a4451
+a4452
+a4453
+a4454
+a4455
+a4456
+a4457
+a4458
+a4459
+a4460
+a4461
+a4462
+a4463
+a4464
+a4465
+a4466
+a4467
+a4468
+a4469
+a4470
+a4471
+a4472
+a4473
+a4474
+a4475
+a4476
+a4477
+a4478
+a4479
+a4480
+a4481
+a4482
+a4483
+a4484
+a4485
+a4486
+a4487
+a4488
+a4489
+a4490
+a4491
+a4492
+a4493
+a4494
+a4495
+a4496
+a4497
+a4498
+a4499
+a4500

models/attention_fusion.py

@@ -0,0 +1,108 @@
+import torch
+from torch import nn
+
+class LocalFusion(nn.Module):
+    def __init__(self, att_in_dim=3, num_categories=6, max_pool_ksize1=4, max_pool_ksize2=2, encoder_dims=[8, 16]):
+        super().__init__()
+        self.num_categories = num_categories
+        self.att_in_dim = att_in_dim
+
+        self.attention_fusion = nn.ModuleList([Self_Attn(in_dim=att_in_dim, max_pool_ksize1=max_pool_ksize1, max_pool_ksize2=max_pool_ksize2, encoder_dims=encoder_dims) for _ in range(num_categories)])
+
+    def forward(self, x, color_naming_probs=None, q=None):
+
+        # Using the average to compute the blending
+        if color_naming_probs is None:
+            # Using the same input tensor for query, key, and value
+            if q is None:
+                return torch.mean(torch.stack([att(x_color, q=x) for att, x_color in zip(self.attention_fusion, x)], dim=0))
+            else:
+                return torch.mean(torch.stack([att(x_color, q=q) for att, x_color in zip(self.attention_fusion, x)], dim=0))
+
+        # Using the color naming probabilities to compute the blending. Weighted average with color naming probs as
+        # weights.
+        else:
+            color_naming_probs = (color_naming_probs > 0.20).float()
+            color_naming_avg = torch.sum(color_naming_probs, dim=0).unsqueeze(1).repeat(1, 3, 1, 1)
+            color_naming_probs = color_naming_probs.unsqueeze(2).repeat(1, 1, 3, 1, 1)
+
+            # Using the same input tensor for query, key, and value
+            if q is None:
+                out = torch.stack([att(x_color, q=x) for att, x_color in zip(self.attention_fusion, x)], dim=0)
+            else:
+                out = torch.stack([att(x_color, q=q) for att, x_color in zip(self.attention_fusion, x)], dim=0)
+
+            out = torch.sum(out * color_naming_probs, dim=0) / color_naming_avg
+            return torch.clip(out, 0, 1)
+
+class Self_Attn(nn.Module):
+    def __init__(self, in_dim, max_pool_ksize1=4, max_pool_ksize2=2, encoder_dims=[8, 16]):
+        super(Self_Attn, self).__init__()
+        self.chanel_in = in_dim
+        self.max_pool_ksize1 = max_pool_ksize1
+        self.max_pool_ksize2 = max_pool_ksize2
+        self.down_ratio = max_pool_ksize1 * max_pool_ksize2
+
+        self.query_conv = nn.Sequential(
+            nn.Conv2d(in_channels=in_dim, out_channels=encoder_dims[0], kernel_size=1),
+            nn.ReLU(),
+            nn.MaxPool2d(4, 4),
+            nn.Conv2d(in_channels=encoder_dims[0], out_channels=encoder_dims[1], kernel_size=1),
+            nn.ReLU(),
+            nn.MaxPool2d(2, 2))
+
+        self.key_conv = nn.Sequential(
+            nn.Conv2d(in_channels=in_dim, out_channels=encoder_dims[0], kernel_size=1),
+            nn.ReLU(),
+            nn.MaxPool2d(4, 4),
+            nn.Conv2d(in_channels=encoder_dims[0], out_channels=encoder_dims[1], kernel_size=1),
+            nn.ReLU(),
+            nn.MaxPool2d(2, 2))
+
+        self.value_conv = nn.Sequential(
+            nn.Conv2d(in_channels=in_dim, out_channels=encoder_dims[0], kernel_size=1),
+            nn.ReLU(),
+            nn.MaxPool2d(4, 4),
+            nn.Conv2d(in_channels=encoder_dims[0], out_channels=encoder_dims[1], kernel_size=1),
+            nn.ReLU(),
+            nn.MaxPool2d(2, 2))
+
+        self.upsample = nn.Sequential(
+            nn.Conv2d(in_channels=encoder_dims[1], out_channels=encoder_dims[0], kernel_size=1),
+            nn.ReLU(),
+            nn.UpsamplingNearest2d(scale_factor=4),
+            nn.Conv2d(in_channels=encoder_dims[0], out_channels=encoder_dims[0], kernel_size=1),
+            nn.ReLU(),
+            nn.Conv2d(in_channels=encoder_dims[0], out_channels=3, kernel_size=1),
+            nn.ReLU())
+
+        self.last_conv = nn.Conv2d(in_channels=3, out_channels=3, kernel_size=1)
+
+        self.gamma = nn.Parameter(torch.zeros(1))
+
+        self.max_pool = nn.MaxPool2d(2, 2)
+        self.softmax = nn.Softmax(dim=-1)
+
+    def forward(self, x, q=None):
+
+        if q is None:
+            q = x
+
+        m_batch_size, C, width, height = x.size()
+        proj_query = self.query_conv(q).view(m_batch_size, -1, int((width//self.down_ratio)*(height//self.down_ratio))).permute(0, 2, 1)
+        proj_key = self.key_conv(x).view(m_batch_size, -1, int((width//self.down_ratio)*(height//self.down_ratio)))
+        energy = torch.bmm(proj_query, proj_key)
+        attention = self.softmax(energy)
+        proj_value = self.value_conv(x).view(m_batch_size, -1, int((width//self.down_ratio)*(height//self.down_ratio)))
+
+        out = torch.bmm(proj_value, attention.permute(0, 2, 1))
+        out = out.view(m_batch_size, 16, int(width//self.down_ratio), int(height//self.down_ratio))
+
+        out = self.upsample(out)
+        upsampled_layer = nn.Upsample(size=x.size()[2:], mode='bilinear', align_corners=False)
+        out = upsampled_layer(out)
+
+        out = self.last_conv(out)
+
+        out = out + x
+        return out

models/backbone.py

@@ -0,0 +1,234 @@
+"""
+backbone.py - Contains the backbone of the model.
+(It is based on LPIENet and CURL's backbone)
+
+Perceptual Image Enhancement for Smartphone Real-Time Applications
+https://github.com/mv-lab/AISP
+
+CURL: Neural Curve Layers for Global Image Enhancement
+https://github.com/sjmoran/CURL
+
+David Serrano (dserrano@cvc.uab.cat)
+May 2024
+"""
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from typing import List
+
+
+class AttentionBlock(nn.Module):
+    def __init__(self, dim: int):
+        super(AttentionBlock, self).__init__()
+        self._spatial_attention_conv = nn.Conv2d(2, dim, kernel_size=3, padding=1)
+
+        # Channel attention MLP
+        self._channel_attention_conv0 = nn.Conv2d(1, dim, kernel_size=1, padding=0)
+        self._channel_attention_conv1 = nn.Conv2d(dim, dim, kernel_size=1, padding=0)
+
+        self._out_conv = nn.Conv2d(2 * dim, dim, kernel_size=1, padding=0)
+
+    def forward(self, x: torch.Tensor):
+        if len(x.shape) != 4:
+            raise ValueError(f"Expected [B, C, H, W] input, got {x.shape}.")
+
+        # Spatial attention
+        mean = torch.mean(x, dim=1, keepdim=True)  # Mean/Max on C axis
+        max, _ = torch.max(x, dim=1, keepdim=True)
+        spatial_attention = torch.cat([mean, max], dim=1)  # [B, 2, H, W]
+        spatial_attention = self._spatial_attention_conv(spatial_attention)
+        spatial_attention = torch.sigmoid(spatial_attention) * x
+
+        # NOTE: This differs from CBAM as it uses Channel pooling, not spatial pooling!
+        # In a way, this is 2x spatial attention
+        channel_attention = torch.relu(self._channel_attention_conv0(mean))
+        channel_attention = self._channel_attention_conv1(channel_attention)
+        channel_attention = torch.sigmoid(channel_attention) * x
+
+        attention = torch.cat([spatial_attention, channel_attention], dim=1)  # [B, 2*dim, H, W]
+        attention = self._out_conv(attention)
+        return x + attention
+
+
+class InverseBlock(nn.Module):
+    def __init__(self, input_channels: int, channels: int):
+        super(InverseBlock, self).__init__()
+
+        self._conv0 = nn.Conv2d(input_channels, channels, kernel_size=1)
+        self._dw_conv = nn.Conv2d(channels, channels, kernel_size=3, padding=1, groups=channels)
+        self._conv1 = nn.Conv2d(channels, channels, kernel_size=1)
+        self._conv2 = nn.Conv2d(input_channels, channels, kernel_size=1)
+
+    def forward(self, x: torch.Tensor):
+        features = self._conv0(x)
+        features = F.elu(self._dw_conv(features))
+        features = self._conv1(features)
+
+        x = torch.relu(self._conv2(x))
+        return x + features
+
+
+class BaseBlock(nn.Module):
+    def __init__(self, channels: int):
+        super(BaseBlock, self).__init__()
+
+        self._conv0 = nn.Conv2d(channels, channels, kernel_size=1)
+        self._dw_conv = nn.Conv2d(channels, channels, kernel_size=3, padding=1, groups=channels)
+        self._conv1 = nn.Conv2d(channels, channels, kernel_size=1)
+
+        self._conv2 = nn.Conv2d(channels, channels, kernel_size=1)
+        self._conv3 = nn.Conv2d(channels, channels, kernel_size=1)
+
+    def forward(self, x: torch.Tensor):
+        features = self._conv0(x)
+        features = F.elu(self._dw_conv(features))
+        features = self._conv1(features)
+        x = x + features
+
+        features = F.elu(self._conv2(x))
+        features = self._conv3(features)
+        return x + features
+
+
+class AttentionTail(nn.Module):
+    def __init__(self, channels: int):
+        super(AttentionTail, self).__init__()
+
+        self._conv0 = nn.Conv2d(channels, channels, kernel_size=7, padding=3)
+        self._conv1 = nn.Conv2d(channels, channels, kernel_size=5, padding=2)
+        self._conv2 = nn.Conv2d(channels, channels, kernel_size=3, padding=1)
+
+    def forward(self, x: torch.Tensor):
+        attention = torch.relu(self._conv0(x))
+        attention = torch.relu(self._conv1(attention))
+        attention = torch.sigmoid(self._conv2(attention))
+        return x * attention
+
+class Flatten(nn.Module):
+
+    def forward(self, x):
+        """Flatten a Tensor to a Vector
+
+        :param x: Tensor
+        :returns: 1D Tensor
+        :rtype: Tensor
+
+        """
+        return x.view(x.size()[0], -1)
+
+class ResidualConnection(nn.Module):
+    def __init__(self, in_channels):
+        super(ResidualConnection, self).__init__()
+
+        self.in_channels = in_channels
+
+        self.midnet2 = nn.Sequential(
+            nn.Conv2d(in_channels, 64, 3, 1, 2, 2),
+            nn.LeakyReLU(),
+            nn.Conv2d(64, 64, 3, 1, 2, 2),
+            nn.LeakyReLU()
+        )
+
+        self.midnet4 = nn.Sequential(
+            nn.Conv2d(in_channels, 64, 3, 1, 4, 4),
+            nn.LeakyReLU(),
+            nn.Conv2d(64, 64, 3, 1, 4, 4),
+            nn.LeakyReLU()
+        )
+
+        self.globnet = nn.Sequential(
+            nn.Conv2d(in_channels, 64, 3, 2, 1, 1),
+            nn.LeakyReLU(),
+            nn.MaxPool2d(kernel_size=3, stride=2, padding=1),
+            nn.Conv2d(64, 64, 3, 2, 1, 1),
+            nn.LeakyReLU(),
+            nn.MaxPool2d(kernel_size=3, stride=2, padding=1),
+            nn.Conv2d(64, 64, 3, 2, 1, 1),
+            nn.LeakyReLU(),
+            nn.AdaptiveAvgPool2d(1),
+            Flatten(),
+            nn.Dropout(0.5),
+            nn.Linear(64, 64)
+        )
+
+        self.conv_fuse = nn.Conv2d(in_channels=192+in_channels, out_channels=in_channels, kernel_size=1)
+    def forward(self, x):
+
+        x_midnet2 = self.midnet2(x)
+        x_midnet4 = self.midnet4(x)
+        x_global = self.globnet(x).unsqueeze(2).unsqueeze(3)
+        x_global = x_global.repeat(1, 1, x_midnet2.shape[2], x_midnet2.shape[3])
+
+        x_fuse = torch.cat((x, x_midnet2, x_midnet4, x_global), dim=1)
+        x_out = self.conv_fuse(x_fuse)
+
+        return x_out
+
+class Backbone(nn.Module):
+    def __init__(self, input_channels: int, output_channels: int, encoder_dims: List[int], decoder_dims: List[int]):
+        super(Backbone, self).__init__()
+
+        if len(encoder_dims) != len(decoder_dims) + 1 or len(decoder_dims) < 1:
+            raise ValueError(f"Unexpected encoder and decoder dims: {encoder_dims}, {decoder_dims}.")
+
+        if input_channels != output_channels:
+            raise NotImplementedError()
+
+        encoders = []
+        for i, encoder_dim in enumerate(encoder_dims):
+            input_dim = input_channels if i == 0 else encoder_dims[i - 1]
+            encoders.append(
+                nn.Sequential(
+                    nn.Conv2d(input_dim, encoder_dim, kernel_size=3, padding=1),
+                    BaseBlock(encoder_dim),
+                    BaseBlock(encoder_dim),
+                    AttentionBlock(encoder_dim),
+                )
+            )
+        self._encoders = nn.ModuleList(encoders)
+
+        decoders = []
+        for i, decoder_dim in enumerate(decoder_dims):
+            input_dim = encoder_dims[-1] if i == 0 else decoder_dims[i - 1] + encoder_dims[-i - 1]
+            decoders.append(
+                nn.Sequential(
+                    nn.Conv2d(input_dim, decoder_dim, kernel_size=3, padding=1),
+                    BaseBlock(decoder_dim),
+                    BaseBlock(decoder_dim),
+                    AttentionBlock(decoder_dim),
+                )
+            )
+        self._decoders = nn.ModuleList(decoders)
+
+        self._inverse_bock = InverseBlock(encoder_dims[0] + decoder_dims[-1], output_channels)
+        self._attention_tail = AttentionTail(output_channels)
+
+        residual_connections = []
+        for i, decoder_dim in enumerate(encoder_dims):
+            residual_connections.append(
+                ResidualConnection(in_channels=decoder_dim)
+            )
+        self._residual_connections = nn.ModuleList(residual_connections)
+    def forward(self, x: torch.Tensor):
+        if len(x.shape) != 4:
+            raise ValueError(f"Expected [B, C, H, W] input, got {x.shape}.")
+        global_residual = x
+
+        encoder_outputs, residual_connections = [], []
+        for i, encoder in enumerate(self._encoders):
+            x = encoder(x)
+            if i != len(self._encoders) - 1:
+                encoder_outputs.append(x)
+                residual_connections.append(self._residual_connections[i](x))
+                x = F.max_pool2d(x, kernel_size=2)
+
+        encoder_outputs.reverse()
+        residual_connections.reverse()
+        for i, decoder in enumerate(self._decoders):
+            x = decoder(x)
+            x = nn.Upsample(size=encoder_outputs[i].shape[2:], mode='bilinear', align_corners=False)(x)
+            x = torch.cat([x, residual_connections[i]], dim=1)
+
+        x = self._inverse_bock(x)
+        x = self._attention_tail(x)
+        return torch.clip(x + global_residual, 0, 1)

models/bezier_control_point_estimator.py

@@ -0,0 +1,90 @@
+"""
+bezier_control_point_estimator.py - Contains the Bezier Control Point Estimator model.
+The Bezier Control Point Estimator estimates the set of control points that define the Bezier curve for each color name.
+
+David Serrano (dserrano@cvc.uab.cat)
+May 2024
+"""
+
+import torch
+from torch import nn
+
+class ContextualFeatureExtractor(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.main = nn.Sequential(
+            nn.Conv2d(3, 8, 3, 1, 1),
+            nn.ReLU(),
+            nn.Conv2d(8, 16, 3, 1, 1),
+            nn.ReLU(),
+            nn.Conv2d(16, 32, 3, 1, 1),
+            nn.ReLU(),
+            nn.Dropout(0.2),
+            nn.Conv2d(32, 64, 3, 1, 1),
+            nn.ReLU())
+
+    def forward(self, x):
+        return self.main(x)
+
+class BezierColorBranch(nn.Module):
+    def __init__(self, num_control_points=10):
+        super().__init__()
+        self.num_control_points = num_control_points # +1, (0, 0) point
+        self.color_branch = nn.Sequential(
+            nn.Conv2d(65, 64, 3, 1, 1),
+            nn.ReLU(),
+            nn.MaxPool2d(2, 2),
+            nn.Conv2d(64, 32, 3, 1, 1),
+            nn.ReLU(),
+            nn.MaxPool2d(2, 2),
+            nn.Conv2d(32, 32, 3, 1, 1),
+            nn.ReLU(),
+            nn.Conv2d(32, 3 * self.num_control_points, 3, 1, 1),
+            nn.AdaptiveAvgPool2d((1, 1)))
+
+        self.sigmoid = nn.Sigmoid()
+
+    def create_control_points(self, x):
+        x = torch.cumsum(torch.cat([torch.zeros_like(x[..., :1]), x], dim=-1), dim=-1)
+        x = torch.stack([x, torch.linspace(0, 1, steps=self.num_control_points+1).unsqueeze(0).repeat(x.shape[0], x.shape[1], 1).cuda()], dim=-1)
+        return x
+
+    def forward(self, x):
+        x = self.color_branch(x).view(x.size(0), 3, self.num_control_points)
+        x = self.sigmoid(x)
+        x = x / torch.sum(x, dim=2)[..., None]
+        x = self.create_control_points(x)
+        return x
+
+class BCPE(nn.Module):
+    def __init__(self, num_categories=6, num_control_points=10):
+        super().__init__()
+
+        self.contextual_feature_extractor = ContextualFeatureExtractor()
+        self.color_branches = nn.ModuleList([BezierColorBranch(num_control_points) for _ in range(num_categories)])
+
+    def binomial_coefficient(self, n, k):
+        """
+        Calculate the binomial coefficient (n choose k).
+        """
+        if k < 0 or k > n:
+            return 0.0
+        result = 1.0
+        for i in range(min(k, n - k)):
+            result *= (n - i)
+            result //= (i + 1)
+        return result
+
+    def apply_cubic_bezier(self, x, control_points):
+
+        n = control_points.shape[2]
+        output = torch.zeros_like(x)
+        for j in range(n):
+            output += control_points[..., j, 0].view(control_points.shape[0], control_points.shape[1], 1, 1) * self.binomial_coefficient(n - 1, j) * (1 - x) ** (n - 1 - j) * x ** j
+        return output
+
+    def forward(self, x, cn_probs):
+        feat = self.contextual_feature_extractor(x)
+        bezier_control_points = [color_branch(torch.cat((feat, color_probs.unsqueeze(1)), dim=1).float()) for color_branch, color_probs in zip(self.color_branches, cn_probs)]
+        global_adjusted_images = torch.stack([self.apply_cubic_bezier(x, control_points) for control_points in bezier_control_points], dim=0)
+        return global_adjusted_images

models/color_naming.py

@@ -0,0 +1,70 @@
+"""
+color_naming.py - Contains the Joost van de Weijer et al. (2009) color naming model.
+
+David Serrano (dserrano@cvc.uab.cat)
+May 2024
+"""
+
+import os
+import pathlib
+from scipy.io import loadmat
+import torch
+from torch import tensor as to_tensor
+from torchvision.transforms.functional import pil_to_tensor
+
+class ColorNaming():
+    def __init__(self, matrix_path=os.path.join(str(pathlib.Path(__file__).parent.resolve()), "joost_color_naming.mat"),
+                 num_categories=6,
+                 device='cuda'):
+        """ Van de Weijer et al. (2009) Color Naming model python implementation.
+        Van De Weijer, J. et al. Learning color names for real-world applications. IEEE Transactions on Image Processing
+        The class is based on the MATLAB implementation by Van de Weijer et al. (2009) and it needs the w2c.mat original
+        file. The input RGB image is converted to a set (6 or 11) color naming probability maps.
+
+        If num_categories is 6: orange-brown-yellow, achromatic, pink-purple, red, green, blue.
+        If num_categories is 11: black, blue, brown, gray, green, orange, pink, purple, red, white, yellow.
+        """
+        self.matrix = to_tensor(loadmat(matrix_path)['w2c']).to(device)
+        self.num_categories = num_categories
+        self.device = device
+
+        if num_categories == 6:
+            self.color_categories = [[2,5,10], [0,3,9], [6,7], [8], [4], [1]]
+            self.color_categories = [torch.tensor(x).to(device) for x in self.color_categories]
+
+    def __call__(self, input_tensor):
+        """Converts an RGB image to a color naming image.
+
+        Args:
+        input_tensor: batch of RGB images (B x 3 x H x W)
+
+        Returns:
+            torch.tensor: Color naming image.
+        """
+        # Reconvert image to [0-255] range
+        input_tensor = torch.clamp(input_tensor, 0, 1)
+        img = (input_tensor * 255).int()
+
+        index_tensor = torch.floor(
+            img[:, 0, ...].view(img.shape[0], -1) / 8).long() + 32 * torch.floor(
+            img[:, 1, ...].view(img.shape[0], -1) / 8).long() + 32 * 32 * torch.floor(
+            img[:, 2, ...].view(img.shape[0], -1) / 8).long()
+
+        prob_maps = []
+        for w2cM in self.matrix.permute(*torch.arange(self.matrix.ndim-1, -1, -1)):
+            out = w2cM[index_tensor].view(input_tensor.size(0), input_tensor.size(2), input_tensor.size(3))
+            prob_maps.append(out)
+        prob_maps = torch.stack(prob_maps, dim=0)
+
+        if self.num_categories == 11:
+            return prob_maps
+
+        elif self.num_categories == 6:
+            category_probs = []  # prob maps for each color category. [0, 1]
+            for category in self.color_categories:
+                cat_tensors = torch.index_select(prob_maps, 0, category).sum(dim=0)
+                category_probs.append(cat_tensors)
+
+            category_probs = torch.stack(category_probs, dim=0)
+
+            return category_probs

models/interactive_model.py

@@ -0,0 +1,125 @@
+"""NamedCurves model with interactive functionality. This version builds upon model.py and bezier_control_point_estimator.py by incorporating additional parameters."""
+
+from models.attention_fusion import LocalFusion
+from models.color_naming import ColorNaming
+from models.backbone import Backbone
+from torch import nn
+
+from PIL import Image
+from torchvision.transforms import functional as TF
+import torch
+
+class NamedCurves(nn.Module):
+    def __init__(self, configs: dict):
+        super().__init__()
+        self.model_configs = configs
+
+        self.backbone = Backbone(**configs['backbone']['params'])
+        self.color_naming = ColorNaming(num_categories=configs['color_naming']['num_categories'])
+        self.bcpe = BCPE(**configs['bezier_control_points_estimator']['params'])
+        self.local_fusion = LocalFusion(**configs['local_fusion']['params'])
+
+    def forward(self, x, return_backbone=False, return_curves=False, control_points=None):
+        x_backbone = self.backbone(x)
+        cn_probs = self.color_naming(x_backbone)
+
+        if return_curves:
+            x_global, control_points = self.bcpe(x_backbone, cn_probs, return_control_points=return_curves, control_points=control_points)
+        else:
+            x_global = self.bcpe(x_backbone, cn_probs, control_points=control_points)
+
+        out = self.local_fusion(x_global, cn_probs, q=x_backbone)
+
+        if return_backbone:
+            return out, x_backbone
+        if return_curves:
+            return out, control_points
+        return out
+
+class ContextualFeatureExtractor(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.main = nn.Sequential(
+            nn.Conv2d(3, 8, 3, 1, 1),
+            nn.ReLU(),
+            nn.Conv2d(8, 16, 3, 1, 1),
+            nn.ReLU(),
+            nn.Conv2d(16, 32, 3, 1, 1),
+            nn.ReLU(),
+            nn.Dropout(0.2),
+            nn.Conv2d(32, 64, 3, 1, 1),
+            nn.ReLU())
+
+    def forward(self, x):
+        return self.main(x)
+
+class BezierColorBranch(nn.Module):
+    def __init__(self, num_control_points=10):
+        super().__init__()
+        self.num_control_points = num_control_points # +1, (0, 0) point
+        self.color_branch = nn.Sequential(
+            nn.Conv2d(65, 64, 3, 1, 1),
+            nn.ReLU(),
+            nn.MaxPool2d(2, 2),
+            nn.Conv2d(64, 32, 3, 1, 1),
+            nn.ReLU(),
+            nn.MaxPool2d(2, 2),
+            nn.Conv2d(32, 32, 3, 1, 1),
+            nn.ReLU(),
+            nn.Conv2d(32, 3 * self.num_control_points, 3, 1, 1),
+            nn.AdaptiveAvgPool2d((1, 1)))
+
+        self.sigmoid = nn.Sigmoid()
+
+    def create_control_points(self, x):
+        x = torch.cumsum(torch.cat([torch.zeros_like(x[..., :1]), x], dim=-1), dim=-1)
+        x = torch.stack([x, torch.linspace(0, 1, steps=self.num_control_points+1).unsqueeze(0).repeat(x.shape[0], x.shape[1], 1).cuda()], dim=-1)
+        return x
+
+    def forward(self, x):
+        x = self.color_branch(x).view(x.size(0), 3, self.num_control_points)
+        x = self.sigmoid(x)
+        x = x / torch.sum(x, dim=2)[..., None]
+        x = self.create_control_points(x)
+        return x
+
+class BCPE(nn.Module):
+    def __init__(self, num_categories=6, num_control_points=10):
+        super().__init__()
+
+        self.contextual_feature_extractor = ContextualFeatureExtractor()
+        self.color_branches = nn.ModuleList([BezierColorBranch(num_control_points) for _ in range(num_categories)])
+
+    def binomial_coefficient(self, n, k):
+        """
+        Calculate the binomial coefficient (n choose k).
+        """
+        if k < 0 or k > n:
+            return 0.0
+        result = 1.0
+        for i in range(min(k, n - k)):
+            result *= (n - i)
+            result //= (i + 1)
+        return result
+
+    def apply_cubic_bezier(self, x, control_points):
+
+        n = control_points.shape[2]
+        output = torch.zeros_like(x)
+        for j in range(n):
+            output += control_points[..., j, 0].view(control_points.shape[0], control_points.shape[1], 1, 1) * self.binomial_coefficient(n - 1, j) * (1 - x) ** (n - 1 - j) * x ** j
+        return output
+
+    def forward(self, x, cn_probs, return_control_points=False, control_points=None):
+        feat = self.contextual_feature_extractor(x)
+        bezier_control_points = [color_branch(torch.cat((feat, color_probs.unsqueeze(1)), dim=1).float()) for color_branch, color_probs in zip(self.color_branches, cn_probs)]
+        
+        if control_points is not None:
+            bezier_control_points = control_points
+
+        global_adjusted_images = torch.stack([self.apply_cubic_bezier(x, control_points) for control_points in bezier_control_points], dim=0)
+        
+        if return_control_points:
+            return global_adjusted_images, bezier_control_points
+        
+        return global_adjusted_images

models/joost_color_naming.mat

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97b1f0ecd0fbecd122de319eacf3b5e23293defb06c40aa37a0446fb11d9cf45
+size 2734783

models/model.py

@@ -0,0 +1,28 @@
+from models.attention_fusion import LocalFusion
+from models.bezier_control_point_estimator import BCPE
+from models.color_naming import ColorNaming
+from models.backbone import Backbone
+from torch import nn
+
+from PIL import Image
+from torchvision.transforms import functional as TF
+import torch
+
+class NamedCurves(nn.Module):
+    def __init__(self, configs: dict):
+        super().__init__()
+        self.model_configs = configs
+
+        self.backbone = Backbone(**configs['backbone']['params'])
+        self.color_naming = ColorNaming(num_categories=configs['color_naming']['num_categories'])
+        self.bcpe = BCPE(**configs['bezier_control_points_estimator']['params'])
+        self.local_fusion = LocalFusion(**configs['local_fusion']['params'])
+
+    def forward(self, x, return_backbone=False):
+        x_backbone = self.backbone(x)
+        cn_probs = self.color_naming(x_backbone)
+        x_global = self.bcpe(x_backbone, cn_probs)
+        out = self.local_fusion(x_global, cn_probs, q=x_backbone)
+        if return_backbone:
+            return out, x_backbone
+        return out

requirements.txt

@@ -0,0 +1,33 @@
+antlr4-python3-runtime
+certifi
+charset-normalizer
+contourpy
+cycler
+fonttools
+idna
+imageio
+importlib_resources
+kiwisolver
+lazy_loader
+lightning-utilities
+lpips==
+matplotlib
+networkx
+numpy
+omegaconf
+packaging
+pillow
+pyparsing
+python-dateutil
+PyWavelets
+PyYAML
+requests
+scikit-image
+scipy
+six
+tifffile
+torchmetrics
+tqdm
+typing_extensions
+urllib3
+zipp

scripts/download_checkpoints.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+mkdir pretrained
+wget https://github.com/davidserra9/namedcurves/releases/download/v1.0/mit5k_dpe_psnr_24.91.pth -P pretrained
+wget https://github.com/davidserra9/namedcurves/releases/download/v1.0/mit5k_uegan_psnr_25.59.pth -P pretrained
+wget https://github.com/davidserra9/namedcurves/releases/download/v1.0/ppr10k_a_psnr_26.81.pth -P pretrained
+wget https://github.com/davidserra9/namedcurves/releases/download/v1.0/ppr10k_b_psnr_25.91.pth -P pretrained
+wget https://github.com/davidserra9/namedcurves/releases/download/v1.0/ppr10k_c_psnr_25.69.pth -P pretrained
+

scripts/generate_naming_maps.py

@@ -0,0 +1,29 @@
+import torch
+import argparse
+import os.path
+from PIL import Image
+from models.color_naming import ColorNaming
+from torchvision.transforms import functional as TF
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--num_categories', type=int, default=6)
+    parser.add_argument('--image_path', type=str, default='/home/dserrano/Documents/datasets/FiveK-DPE/input/a0001-jmac_DSC1459.png')
+    parser.add_argument('--output_path', type=str)
+    return parser.parse_args()
+
+if __name__ == "__main__":
+    args = parse_args()
+    color_naming = ColorNaming(num_categories=args.num_categories)
+
+    if os.path.isfile(args.image_path):
+        image_tensor = TF.pil_to_tensor(Image.open(args.image_path).convert('RGB')).unsqueeze(0)
+        cn_probs = color_naming(image_tensor).float().repeat(1, 3, 1, 1).cpu()
+        output_images = (1 - cn_probs) * 255 * torch.ones_like(image_tensor).repeat(args.num_categories, 1, 1, 1) + cn_probs * image_tensor.repeat(args.num_categories, 1, 1, 1)
+
+        import matplotlib.pyplot as plt
+        fig = plt.subplots(1, args.num_categories, figsize=(20, 20))
+        for i in range(args.num_categories):
+            plt.subplot(1, args.num_categories, i+1)
+            plt.imshow(output_images[i].permute(1, 2, 0).numpy().astype('uint8'))
+        plt.show()

test.py

@@ -0,0 +1,41 @@
+import argparse
+from models.model import NamedCurves
+import torch
+import os
+from omegaconf import OmegaConf
+from glob import glob
+from PIL import Image
+from torchvision.transforms import functional as TF
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--input_path', type=str, default='assets/a4957-input.png')
+    parser.add_argument('--output_path', type=str, default='output/')
+    parser.add_argument('--model_path', type=str, default='/home/dserrano/Workspace/Color-Naming-Image-Enhancement/pretrained/mit5k_uegan_psnr_25.59.pth')
+    parser.add_argument('--config_path', type=str, default='configs/mit5k_dpe_config.yaml')
+    return parser.parse_args()
+
+def main():
+    args = parse_args()
+    config = OmegaConf.load(args.config_path)
+    model = NamedCurves(config.model).cuda()
+    model.load_state_dict(torch.load(args.model_path)["model_state_dict"])
+
+    if not os.path.exists(args.output_path):
+        os.makedirs(args.output_path)
+
+    #check if input_path is a folder
+    if os.path.isdir(args.input_path):
+        input_paths = glob(sorted(args.input_path + '/*'))
+    
+    else:
+        input_paths = [args.input_path]
+    
+    for input_path in input_paths:
+        input_tensor = TF.to_tensor(Image.open(input_path)).unsqueeze(0)
+        output = model(input_tensor.cuda())
+        output = TF.to_pil_image(output[0].cpu())
+        output.save(os.path.join(args.output_path, os.path.basename(input_path)))
+
+if __name__ == '__main__':
+    main()

train.py

@@ -0,0 +1,73 @@
+import torch
+import datetime
+import os
+import logging
+
+import omegaconf
+
+from utils.logger import prepare_logging
+from torch.utils.data import DataLoader
+from data.datasets import get_datasets
+from models.model import NamingEnhancementModel
+from utils.setup_optim_scheduler import get_optimizer_scheduler
+from utils.evaluator import Evaluator
+from utils.setup_criterion import get_criterion
+from utils.trainer import Trainer
+
+
+def main(config: omegaconf.DictConfig):
+
+    os.environ["CUDA_VISIBLE_DEVICES"] = str(config.train.cuda_visible_device)
+    save_path = prepare_logging()
+
+    logging.info(f"Starting training at {datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
+    logging.info(f"Saving logs to {save_path}")
+    logging.info(f"Config file: {OmegaConf.to_yaml(config)}")
+
+    train_dataset, valid_dataset, test_dataset = get_datasets(config.data)
+
+    msg = f"Training with {len(train_dataset)} image pairs"
+    if valid_dataset is not None:
+        msg += f", validating with {len(valid_dataset)} image pairs"
+    msg += f" and testing with {len(test_dataset)} image pairs."
+    logging.info(msg)
+
+    train_loader = DataLoader(train_dataset, batch_size=config.train.batch_size, shuffle=True)
+    if valid_dataset is not None:
+        valid_loader = DataLoader(valid_dataset, batch_size=1, shuffle=False)
+    else:
+        valid_loader = None
+    test_loader = DataLoader(test_dataset, batch_size=1, shuffle=False)
+
+    model = NamingEnhancementModel(config.model)
+    if config.model.ckpt_path is not None:
+        model.load_state_dict(torch.load(config.model.ckpt_path)["model_state_dict"])
+    model.cuda()
+
+    criterion = get_criterion(config.train.criterion)
+
+    optimizer, scheduler = get_optimizer_scheduler(model,
+                                                   config.train.optimizer,
+                                                   config.train.scheduler if "scheduler" in config.train else None)
+
+    if valid_loader is not None:
+        valid_evaluator = Evaluator(valid_loader, config.eval.metrics, 'valid', save_path, config.eval.metric_to_save)
+    else:
+        valid_evaluator = None
+
+    test_evaluator = Evaluator(test_loader, config.eval.metrics, 'test', save_path, config.eval.metric_to_save)
+
+
+    trainer = Trainer(model, optimizer, criterion, scheduler, train_loader, valid_evaluator, test_evaluator, config.train, config.eval)
+    trainer.train()
+
+if __name__ == "__main__":
+    from omegaconf import OmegaConf
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--config', type=str, default='configs/mit5k_dpe_config.yaml')
+    args = parser.parse_args()
+
+    config = OmegaConf.load(args.config)
+    main(config)

utils/deltaE.py

@@ -0,0 +1,125 @@
+from skimage import color
+import numpy as np
+import torch
+
+class deltaEab():
+    def __init__(self, color_chart_area=0):
+        super().__init__()
+        self.color_chart_area = color_chart_area
+
+    def __call__(self, img1, img2):
+        """ Compute the deltaE76 between two numpy RGB images
+        From M. Afifi: https://github.com/mahmoudnafifi/WB_sRGB/blob/master/WB_sRGB_Python/evaluation/calc_deltaE.py
+        :param img1: numpy RGB image or pytorch tensor
+        :param img2: numpy RGB image or pytocrh tensor
+        :return: deltaE76
+        """
+
+        if type(img1) == torch.Tensor:
+            assert img1.shape[0] == 1
+            img1 = img1.squeeze().permute(1, 2, 0).cpu().numpy()
+
+        if type(img2) == torch.Tensor:
+            assert img2.shape[0] == 1
+            img2 = img2.squeeze().permute(1, 2, 0).cpu().numpy()
+
+        # Convert to Lab
+        img1 = color.rgb2lab(img1)
+        img2 = color.rgb2lab(img2)
+
+        # reshape to 1D array
+        img1 = img1.reshape(-1, 3).astype(np.float32)
+        img2 = img2.reshape(-1, 3).astype(np.float32)
+
+        # compute deltaE76
+        de76 = np.sqrt(np.sum(np.power(img1 - img2, 2), 1))
+
+        return sum(de76) / (np.shape(de76)[0] - self.color_chart_area)
+
+
+class deltaE00():
+    def __init__(self, color_chart_area=0):
+        super().__init__()
+        self.color_chart_area = color_chart_area
+        self.kl = 1
+        self.kc = 1
+        self.kh = 1
+
+    def __call__(self, img1, img2):
+        """ Compute the deltaE00 between two numpy RGB images
+        From M. Afifi: https://github.com/mahmoudnafifi/WB_sRGB/blob/master/WB_sRGB_Python/evaluation/calc_deltaE2000.py
+        :param img1: numpy RGB image or pytocrh tensor
+        :param img2: numpy RGB image or pytocrh tensor
+        :return: deltaE00
+        """
+
+        if type(img1) == torch.Tensor:
+            assert img1.shape[0] == 1
+            img1 = img1.squeeze().permute(1, 2, 0).cpu().numpy()
+
+        if type(img2) == torch.Tensor:
+            assert img2.shape[0] == 1
+            img2 = img2.squeeze().permute(1, 2, 0).cpu().numpy()
+
+        # Convert to Lab
+        img1 = color.rgb2lab(img1)
+        img2 = color.rgb2lab(img2)
+
+        # reshape to 1D array
+        img1 = img1.reshape(-1, 3).astype(np.float32)
+        img2 = img2.reshape(-1, 3).astype(np.float32)
+
+        # compute deltaE00
+        Lstd = np.transpose(img1[:, 0])
+        astd = np.transpose(img1[:, 1])
+        bstd = np.transpose(img1[:, 2])
+        Cabstd = np.sqrt(np.power(astd, 2) + np.power(bstd, 2))
+        Lsample = np.transpose(img2[:, 0])
+        asample = np.transpose(img2[:, 1])
+        bsample = np.transpose(img2[:, 2])
+        Cabsample = np.sqrt(np.power(asample, 2) + np.power(bsample, 2))
+        Cabarithmean = (Cabstd + Cabsample) / 2
+        G = 0.5 * (1 - np.sqrt((np.power(Cabarithmean, 7)) / (np.power(
+            Cabarithmean, 7) + np.power(25, 7))))
+        apstd = (1 + G) * astd
+        apsample = (1 + G) * asample
+        Cpsample = np.sqrt(np.power(apsample, 2) + np.power(bsample, 2))
+        Cpstd = np.sqrt(np.power(apstd, 2) + np.power(bstd, 2))
+        Cpprod = (Cpsample * Cpstd)
+        zcidx = np.argwhere(Cpprod == 0)
+        hpstd = np.arctan2(bstd, apstd)
+        hpstd[np.argwhere((np.abs(apstd) + np.abs(bstd)) == 0)] = 0
+        hpsample = np.arctan2(bsample, apsample)
+        hpsample = hpsample + 2 * np.pi * (hpsample < 0)
+        hpsample[np.argwhere((np.abs(apsample) + np.abs(bsample)) == 0)] = 0
+        dL = (Lsample - Lstd)
+        dC = (Cpsample - Cpstd)
+        dhp = (hpsample - hpstd)
+        dhp = dhp - 2 * np.pi * (dhp > np.pi)
+        dhp = dhp + 2 * np.pi * (dhp < (-np.pi))
+        dhp[zcidx] = 0
+        dH = 2 * np.sqrt(Cpprod) * np.sin(dhp / 2)
+        Lp = (Lsample + Lstd) / 2
+        Cp = (Cpstd + Cpsample) / 2
+        hp = (hpstd + hpsample) / 2
+        hp = hp - (np.abs(hpstd - hpsample) > np.pi) * np.pi
+        hp = hp + (hp < 0) * 2 * np.pi
+        hp[zcidx] = hpsample[zcidx] + hpstd[zcidx]
+        Lpm502 = np.power((Lp - 50), 2)
+        Sl = 1 + 0.015 * Lpm502 / np.sqrt(20 + Lpm502)
+        Sc = 1 + 0.045 * Cp
+        T = 1 - 0.17 * np.cos(hp - np.pi / 6) + 0.24 * np.cos(2 * hp) + \
+            0.32 * np.cos(3 * hp + np.pi / 30) \
+            - 0.20 * np.cos(4 * hp - 63 * np.pi / 180)
+        Sh = 1 + 0.015 * Cp * T
+        delthetarad = (30 * np.pi / 180) * np.exp(
+            - np.power((180 / np.pi * hp - 275) / 25, 2))
+        Rc = 2 * np.sqrt((np.power(Cp, 7)) / (np.power(Cp, 7) + np.power(25, 7)))
+        RT = - np.sin(2 * delthetarad) * Rc
+        klSl = self.kl * Sl
+        kcSc = self.kc * Sc
+        khSh = self.kh * Sh
+        de00 = np.sqrt(np.power((dL / klSl), 2) + np.power((dC / kcSc), 2) +
+                       np.power((dH / khSh), 2) + RT * (dC / kcSc) * (dH / khSh))
+
+        return np.sum(de00) / (np.shape(de00)[0] - self.color_chart_area)

utils/evaluator.py

@@ -0,0 +1,69 @@
+import torch
+import logging
+from torchmetrics import PeakSignalNoiseRatio as PSNR
+from torchmetrics import StructuralSimilarityIndexMeasure as SSIM
+from lpips import LPIPS
+from utils.deltaE import deltaEab, deltaE00
+
+class Evaluator():
+    def __init__(self, dataloader, metrics, split_name, log_dirpath, best_metric):
+        self.dataloader = dataloader
+        self._create_metrics(metrics)
+        self.split_name = split_name
+        self.log_dirpath = log_dirpath
+        self.best_metric = best_metric
+        self.best_value = 0
+
+    def _create_metrics(self, metrics):
+        self.metrics = {}
+        self.cumulative_values = {}
+        for metric in metrics:
+            if metric.type == 'PSNR':
+                self.metrics['PSNR'] = PSNR(**metric.params).cuda()
+                self.cumulative_values['PSNR'] = 0
+            elif metric.type == 'SSIM':
+                self.metrics['SSIM'] = SSIM(**metric.params).cuda()
+                self.cumulative_values['SSIM'] = 0
+            elif metric.type == 'LPIPS':
+                self.metrics['LPIPS'] = LPIPS(**metric.params).cuda()
+                self.cumulative_values['LPIPS'] = 0
+            elif metric.type == 'deltaEab':
+                self.metrics['deltaEab'] = deltaEab()
+                self.cumulative_values['deltaEab'] = 0
+            elif metric.type == 'deltaE00':
+                self.metrics['deltaE00'] = deltaE00()
+                self.cumulative_values['deltaE00'] = 0
+            else:
+                raise NotImplementedError(f"Metric {metric.type} not implemented")
+
+    def _compute_metrics(self, input_image, target_image):
+        for name, metric in self.metrics.items():
+            self.cumulative_values[name] += metric(input_image, target_image)
+
+    def _compute_average_metrics(self):
+        avg_metrics = {}
+        for name, value in self.cumulative_values.items():
+            avg_metrics[name] = float(value / len(self.dataloader))
+        return avg_metrics
+
+    def _reset_metrics(self):
+        for metric in self.metrics:
+            self.cumulative_values[metric] = 0
+
+    def __call__(self, model, save_results=True):
+        model.eval()
+        self._reset_metrics()
+        with torch.no_grad():
+            for data in self.dataloader:
+                input_image, target_image, name = data['input_image'], data['target_image'], data['name']
+
+                self._compute_metrics(input_image.cuda(), target_image.cuda())
+
+        avg_metrics = self._compute_average_metrics()
+        logging.info(f"{self.split_name} metrics: " + ", ".join([f'{key}: {value:.4f}' for key, value in avg_metrics.items()]))
+
+        if (avg_metrics[self.best_metric] > self.best_value) and save_results:
+            self.best_value = avg_metrics[self.best_metric]
+            torch.save({**{'model_state_dict': model.state_dict()}, **avg_metrics},
+                       f"{self.log_dirpath}/{self.split_name}_best_model.pth")
+            logging.info(f"New best model saved at {self.log_dirpath}/{self.split_name}_best_model.pth")

utils/logger.py

@@ -0,0 +1,21 @@
+import logging
+import os
+import datetime
+
+def prepare_logging():
+    save_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "logs")
+    if not os.path.exists(save_path):
+        os.makedirs(save_path)
+
+    timestamp = datetime.datetime.now().strftime('%Y%m%d_%H%M%S')
+    log_dirpath = "log_" + timestamp
+    save_path = os.path.join(save_path, log_dirpath)
+    os.mkdir(save_path)
+
+    handlers = [logging.FileHandler(os.path.join(save_path, "log.txt")), logging.StreamHandler()]
+    logging.basicConfig(level=logging.INFO,
+                        format='%(asctime)s %(levelname)s %(message)s',
+                        handlers=handlers,
+                        datefmt='%Y-%m-%d %H:%M:%S')
+
+    return save_path

utils/setup_criterion.py

@@ -0,0 +1,26 @@
+from torch import nn
+from torchmetrics import StructuralSimilarityIndexMeasure as SSIM
+
+class BackboneL2SSIMLoss(nn.Module):
+    def __init__(self, ssim_window_size=5, alpha=0.5):
+        super(BackboneL2SSIMLoss, self).__init__()
+        self.ssim_window_size = ssim_window_size
+        self.alpha = alpha
+
+        self.ssim_loss = SSIM(kernel_size=ssim_window_size).cuda()
+        self.l2_loss = nn.MSELoss()
+
+    def forward(self, backbone, prediction, target):
+        ssim_loss_pred = (1.0 - self.ssim_loss(prediction, target))
+        l2_loss_pred = self.l2_loss(prediction, target)
+        l2_loss_backbone = self.l2_loss(backbone, target)
+
+        return self.alpha*l2_loss_backbone + l2_loss_pred + ssim_loss_pred
+
+def get_criterion(criterion_config):
+    criterion_type = criterion_config.type
+    if criterion_type == 'backbone-L2-SSIM':
+        return BackboneL2SSIMLoss(**criterion_config['params'])
+    #TODO: Add more criterion types here (L1, L2 ...)
+    else:
+        raise ValueError(f"Unsupported criterion type: {criterion_type}")

utils/setup_optim_scheduler.py

@@ -0,0 +1,10 @@
+import torch
+from torch.optim import lr_scheduler
+
+def get_optimizer_scheduler(model, optim_config, scheduler_config):
+    optimizer = getattr(torch.optim, optim_config.type)(model.parameters(), **optim_config.params)
+    if scheduler_config is not None:
+        scheduler = getattr(torch.optim.lr_scheduler, scheduler_config.type)(optimizer, **scheduler_config.params)
+    else:
+        scheduler = None
+    return optimizer, scheduler

utils/trainer.py

@@ -0,0 +1,47 @@
+import logging
+
+class Trainer():
+    def __init__(self, model, optimizer, criterion, scheduler, train_loader, valid_evaluator, test_evaluator, config_train, config_eval):
+        self.model = model
+        self.optimizer = optimizer
+        self.criterion = criterion
+        self.scheduler = scheduler
+        self.train_loader = train_loader
+        self.valid_evaluator = valid_evaluator
+        self.test_evaluator = test_evaluator
+        self.config_train = config_train
+        self.config_eval = config_eval
+
+    def _train_step(self, input_image, target_image):
+        self.optimizer.zero_grad()
+
+        prediction, x_backbone = self.model(input_image.cuda(), return_backbone=True)
+        loss = self.criterion(x_backbone, prediction, target_image.cuda())
+
+        loss.backward()
+        self.optimizer.step()
+        if self.scheduler is not None:
+            self.scheduler.step()
+
+        return loss.item()
+
+    def _train_epoch(self):
+        epoch_loss = 0
+        self.model.train()
+        for data in self.train_loader:
+            input_image, target_image, name = data['input_image'], data['target_image'], data['name']
+            loss = self._train_step(input_image, target_image)
+            epoch_loss += loss
+
+        return epoch_loss / len(self.train_loader)
+
+    def train(self):
+        for epoch in range(self.config_train.epochs):
+            epoch_loss = self._train_epoch()
+            logging.info(f"Epoch {epoch+1}/{self.config_train.epochs} | Loss: {epoch_loss}")
+
+            if self.valid_evaluator is not None and (epoch+1) % self.config_train.valid_every == 0:
+                self.valid_evaluator(self.model)
+
+        self.test_evaluator(self.model, save_results=True if self.valid_evaluator is None else False)
+        logging.info("Training finished.")