|
---
|
|
license: openrail
|
|
---
|
|
|
|
# This repo contains the pretrained checkpoints for Ctrl-Adapter
|
|
|
|
|
|
CTRL-Adapter is an efficient and versatile framework for adding diverse
|
|
spatial controls to any image or video diffusion model. It supports a variety of useful
|
|
applications, including video control, video control with multiple conditions, video control with
|
|
sparse frame conditions, image control, zero-shot transfer to unseen conditions, and video editing.
|
|
|
|
See also: https://github.com/jaehong31/RACCooN
|
|
|
|
|
|
<!-- <br>
|
|
<img width="800" src="assets/raccoon_teaser.png"/>
|
|
<br> -->
|
|
|
|
<br>
|
|
<br>
|
|
|
|
# Description of VPLM Dataset
|
|
|
|
Multi-Objects Description
|
|
- Train: RACCooN/VPLM/gt_train.json
|
|
- Test: RACCooN/VPLM/gt_test.json
|
|
|
|
Single-Object Layout Prediction
|
|
- Train: RACCooN/VPLM/gt_train_layouts.json
|
|
- Test: RACCooN/VPLM/gt_test_layouts.json
|
|
|
|
|
|
# Description of Model Checkpoints
|
|
|
|
## V2P
|
|
|
|
Multi-Objects Description
|
|
- RACCooN/mllm_finetuned/multi_obj_projector.bin
|
|
|
|
Single-Object Description
|
|
- RACCooN/mllm_finetuned/single_obj_projector.bin
|
|
|
|
Single-Object Layout Prediction
|
|
- RACCooN/mllm_finetuned/layout_pred_projector.bin
|
|
|
|
|
|
## P2V
|
|
- RACCooN/unet_finetuned/diffusion_pytorch_model.safetensors
|
|
|
