metadata
license: openrail
This repo contains the pretrained checkpoints for Ctrl-Adapter
CTRL-Adapter is an efficient and versatile framework for adding diverse spatial controls to any image or video diffusion model. It supports a variety of useful applications, including video control, video control with multiple conditions, video control with sparse frame conditions, image control, zero-shot transfer to unseen conditions, and video editing.
See also: https://github.com/jaehong31/RACCooN
Description of VPLM Dataset
Multi-Objects Description
- Train: RACCooN/VPLM/gt_train.json
- Test: RACCooN/VPLM/gt_test.json
Single-Object Layout Prediction
- Train: RACCooN/VPLM/gt_train_layouts.json
- Test: RACCooN/VPLM/gt_test_layouts.json
Description of Model Checkpoints
V2P
Multi-Objects Description
- RACCooN/mllm_finetuned/multi_obj_projector.bin
Single-Object Description
- RACCooN/mllm_finetuned/single_obj_projector.bin
Single-Object Layout Prediction
- RACCooN/mllm_finetuned/layout_pred_projector.bin
P2V
- RACCooN/unet_finetuned/diffusion_pytorch_model.safetensors