XiangpengYang commited on
Commit
b5e0a87
·
1 Parent(s): 69073d3

update intro

Browse files
README.md CHANGED
@@ -8,6 +8,17 @@
8
  [![Demo Video - VideoGrain](https://img.shields.io/badge/Demo_Video-VideoGrain-red)](https://youtu.be/JKDLet618hU)
9
 
10
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  <table class="center" border="1" cellspacing="0" cellpadding="5">
13
  <tr>
@@ -92,7 +103,8 @@ VideoGrain is a training-free framework. To run the inference script, use the fo
92
 
93
  ```bash
94
  bash test.sh
95
- or accelerate launch test.py --config config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
 
96
  ```
97
 
98
  <details><summary>The result is saved at `./result` . (Click for directory structure) </summary>
 
8
  [![Demo Video - VideoGrain](https://img.shields.io/badge/Demo_Video-VideoGrain-red)](https://youtu.be/JKDLet618hU)
9
 
10
 
11
+ ## Introduction
12
+ VideoGrain is a zero-shot method for class-level, instance-level, and part-level video editing.
13
+ - **Multi-grained Video Editing**
14
+ - class-level: Editing objects within the same class (previous SOTA limited to this level)
15
+ - instance-level: Editing each individual instance to distinct object
16
+ - part-level: Adding new objects or modifying existing attributes at the part-level
17
+ - **Training-Free**
18
+ - Does not require any training/fine-tuning
19
+ - **One prompt to control multi-regions, deep investigations about cross/self attn**
20
+ - modulating cross attn weight for multi-regions control, visualization are available
21
+ - modulating self attn weight for feature decoupling, clustering function are available
22
 
23
  <table class="center" border="1" cellspacing="0" cellpadding="5">
24
  <tr>
 
103
 
104
  ```bash
105
  bash test.sh
106
+ #or
107
+ CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
108
  ```
109
 
110
  <details><summary>The result is saved at `./result` . (Click for directory structure) </summary>
annotator/dwpose/__pycache__/wholebody.cpython-310.pyc CHANGED
Binary files a/annotator/dwpose/__pycache__/wholebody.cpython-310.pyc and b/annotator/dwpose/__pycache__/wholebody.cpython-310.pyc differ
 
config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml CHANGED
@@ -1,4 +1,4 @@
1
- pretrained_model_path: "./ckpt/stable-diffusion-v1-5"
2
  logdir: ./result/part_level/run_two_man/left2spider_right2polar-sunglasses
3
 
4
  dataset_config:
 
1
+ pretrained_model_path: "ckpt/stable-diffusion-v1-5"
2
  logdir: ./result/part_level/run_two_man/left2spider_right2polar-sunglasses
3
 
4
  dataset_config:
test.sh CHANGED
@@ -1,2 +1,2 @@
1
  export CUDA_VISIBLE_DEVICES=0
2
- accelerate launch test.py --config config/instance_level/running_two_man/running_3cls_polar_spider_vis_weight.yaml
 
1
  export CUDA_VISIBLE_DEVICES=0
2
+ accelerate launch test.py --config config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
video_diffusion/common/__pycache__/image_util.cpython-310.pyc CHANGED
Binary files a/video_diffusion/common/__pycache__/image_util.cpython-310.pyc and b/video_diffusion/common/__pycache__/image_util.cpython-310.pyc differ