---
license: openrail
---

# This repo contains the pretrained checkpoints for Ctrl-Adapter


CTRL-Adapter is an efficient and versatile framework for adding diverse
spatial controls to any image or video diffusion model. It supports a variety of useful
applications, including video control, video control with multiple conditions, video control with
sparse frame conditions, image control, zero-shot transfer to unseen conditions, and video editing.

See also: https://github.com/jaehong31/RACCooN


<!-- <br>
<img width="800" src="assets/raccoon_teaser.png"/>
<br> -->

<br>
<br>

# Description of VPLM Dataset

Multi-Objects Description
- Train: RACCooN/VPLM/gt_train.json
- Test: RACCooN/VPLM/gt_test.json

Single-Object Layout Prediction
- Train: RACCooN/VPLM/gt_train_layouts.json
- Test: RACCooN/VPLM/gt_test_layouts.json


# Description of Model Checkpoints

## V2P

Multi-Objects Description
- RACCooN/mllm_finetuned/multi_obj_projector.bin

Single-Object Description
- RACCooN/mllm_finetuned/single_obj_projector.bin

Single-Object Layout Prediction
- RACCooN/mllm_finetuned/layout_pred_projector.bin


## P2V
- RACCooN/unet_finetuned/diffusion_pytorch_model.safetensors