Spaces:
Runtime error
Runtime error
Commit
·
b5e0a87
1
Parent(s):
69073d3
update intro
Browse files
README.md
CHANGED
@@ -8,6 +8,17 @@
|
|
8 |
[](https://youtu.be/JKDLet618hU)
|
9 |
|
10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
|
12 |
<table class="center" border="1" cellspacing="0" cellpadding="5">
|
13 |
<tr>
|
@@ -92,7 +103,8 @@ VideoGrain is a training-free framework. To run the inference script, use the fo
|
|
92 |
|
93 |
```bash
|
94 |
bash test.sh
|
95 |
-
or
|
|
|
96 |
```
|
97 |
|
98 |
<details><summary>The result is saved at `./result` . (Click for directory structure) </summary>
|
|
|
8 |
[](https://youtu.be/JKDLet618hU)
|
9 |
|
10 |
|
11 |
+
## Introduction
|
12 |
+
VideoGrain is a zero-shot method for class-level, instance-level, and part-level video editing.
|
13 |
+
- **Multi-grained Video Editing**
|
14 |
+
- class-level: Editing objects within the same class (previous SOTA limited to this level)
|
15 |
+
- instance-level: Editing each individual instance to distinct object
|
16 |
+
- part-level: Adding new objects or modifying existing attributes at the part-level
|
17 |
+
- **Training-Free**
|
18 |
+
- Does not require any training/fine-tuning
|
19 |
+
- **One prompt to control multi-regions, deep investigations about cross/self attn**
|
20 |
+
- modulating cross attn weight for multi-regions control, visualization are available
|
21 |
+
- modulating self attn weight for feature decoupling, clustering function are available
|
22 |
|
23 |
<table class="center" border="1" cellspacing="0" cellpadding="5">
|
24 |
<tr>
|
|
|
103 |
|
104 |
```bash
|
105 |
bash test.sh
|
106 |
+
#or
|
107 |
+
CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
|
108 |
```
|
109 |
|
110 |
<details><summary>The result is saved at `./result` . (Click for directory structure) </summary>
|
annotator/dwpose/__pycache__/wholebody.cpython-310.pyc
CHANGED
Binary files a/annotator/dwpose/__pycache__/wholebody.cpython-310.pyc and b/annotator/dwpose/__pycache__/wholebody.cpython-310.pyc differ
|
|
config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
CHANGED
@@ -1,4 +1,4 @@
|
|
1 |
-
pretrained_model_path: "
|
2 |
logdir: ./result/part_level/run_two_man/left2spider_right2polar-sunglasses
|
3 |
|
4 |
dataset_config:
|
|
|
1 |
+
pretrained_model_path: "ckpt/stable-diffusion-v1-5"
|
2 |
logdir: ./result/part_level/run_two_man/left2spider_right2polar-sunglasses
|
3 |
|
4 |
dataset_config:
|
test.sh
CHANGED
@@ -1,2 +1,2 @@
|
|
1 |
export CUDA_VISIBLE_DEVICES=0
|
2 |
-
accelerate launch test.py --config config/
|
|
|
1 |
export CUDA_VISIBLE_DEVICES=0
|
2 |
+
accelerate launch test.py --config config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
|
video_diffusion/common/__pycache__/image_util.cpython-310.pyc
CHANGED
Binary files a/video_diffusion/common/__pycache__/image_util.cpython-310.pyc and b/video_diffusion/common/__pycache__/image_util.cpython-310.pyc differ
|
|