wbhu-tc commited on
Commit
38f5a7c
β€’
1 Parent(s): 5641ce9
Files changed (2) hide show
  1. README.md +13 -111
  2. app.py +2 -1
README.md CHANGED
@@ -1,111 +1,13 @@
1
- ## ___***DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos***___
2
- <div align="center">
3
- <img src='https://depthcrafter.github.io/img/logo.png' style="height:140px"></img>
4
-
5
-
6
-
7
- <a href='https://arxiv.org/abs/2409.02095'><img src='https://img.shields.io/badge/arXiv-2409.02095-b31b1b.svg'></a> &nbsp;
8
- <a href='https://depthcrafter.github.io'><img src='https://img.shields.io/badge/Project-Page-Green'></a> &nbsp;
9
-
10
-
11
- _**[Wenbo Hu<sup>1* &dagger;</sup>](https://wbhu.github.io),
12
- [Xiangjun Gao<sup>2*</sup>](https://scholar.google.com/citations?user=qgdesEcAAAAJ&hl=en),
13
- [Xiaoyu Li<sup>1* &dagger;</sup>](https://xiaoyu258.github.io),
14
- [Sijie Zhao<sup>1</sup>](https://scholar.google.com/citations?user=tZ3dS3MAAAAJ&hl=en),
15
- [Xiaodong Cun<sup>1</sup>](https://vinthony.github.io/academic), <br>
16
- [Yong Zhang<sup>1</sup>](https://yzhang2016.github.io),
17
- [Long Quan<sup>2</sup>](https://home.cse.ust.hk/~quan),
18
- [Ying Shan<sup>3, 1</sup>](https://scholar.google.com/citations?user=4oXBp9UAAAAJ&hl=en)**_
19
- <br><br>
20
- <sup>1</sup>Tencent AI Lab
21
- <sup>2</sup>The Hong Kong University of Science and Technology
22
- <sup>3</sup>ARC Lab, Tencent PCG
23
-
24
- arXiv preprint, 2024
25
-
26
- </div>
27
-
28
- ## πŸ”† Introduction
29
-
30
- - [24-9-19] Add scripts for preparing benchmark datasets.
31
- - [24-9-18] Add point cloud sequence visualization.
32
- - [24-9-14] πŸ”₯πŸ”₯πŸ”₯ **DepthCrafter** is released now, have fun!
33
-
34
-
35
- πŸ€— DepthCrafter can generate temporally consistent long depth sequences with fine-grained details for open-world videos,
36
- without requiring additional information such as camera poses or optical flow.
37
-
38
- ## πŸŽ₯ Visualization
39
- We provide some demos of unprojected point cloud sequences, with reference RGB and estimated depth videos.
40
- Please refer to our [project page](https://depthcrafter.github.io) for more details.
41
-
42
-
43
- https://github.com/user-attachments/assets/62141cc8-04d0-458f-9558-fe50bc04cc21
44
-
45
-
46
-
47
-
48
- ## πŸš€ Quick Start
49
-
50
- ### πŸ› οΈ Installation
51
- 1. Clone this repo:
52
- ```bash
53
- git clone https://github.com/Tencent/DepthCrafter.git
54
- ```
55
- 2. Install dependencies (please refer to [requirements.txt](requirements.txt)):
56
- ```bash
57
- pip install -r requirements.txt
58
- ```
59
-
60
- ## πŸ€— Model Zoo
61
- [DepthCrafter](https://huggingface.co/tencent/DepthCrafter) is available in the Hugging Face Model Hub.
62
-
63
- ### πŸƒβ€β™‚οΈ Inference
64
- #### 1. High-resolution inference, requires a GPU with ~26GB memory for 1024x576 resolution:
65
- - Full inference (~0.6 fps on A100, recommended for high-quality results):
66
-
67
- ```bash
68
- python run.py --video-path examples/example_01.mp4
69
- ```
70
-
71
-
72
- - Fast inference through 4-step denoising and without classifier-free guidance (~2.3 fps on A100οΌ‰:
73
-
74
- ```bash
75
- python run.py --video-path examples/example_01.mp4 --num-inference-steps 4 --guidance-scale 1.0
76
- ```
77
-
78
-
79
- #### 2. Low-resolution inference, requires a GPU with ~9GB memory for 512x256 resolution:
80
-
81
- - Full inference (~2.3 fps on A100):
82
-
83
- ```bash
84
- python run.py --video-path examples/example_01.mp4 --max-res 512
85
- ```
86
-
87
- - Fast inference through 4-step denoising and without classifier-free guidance (~9.4 fps on A100):
88
- ```bash
89
- python run.py --video-path examples/example_01.mp4 --max-res 512 --num-inference-steps 4 --guidance-scale 1.0
90
- ```
91
-
92
- ## πŸ€– Gradio Demo
93
- We provide a local Gradio demo for DepthCrafter, which can be launched by running:
94
- ```bash
95
- gradio app.py
96
- ```
97
-
98
- ## 🀝 Contributing
99
- - Welcome to open issues and pull requests.
100
- - Welcome to optimize the inference speed and memory usage, e.g., through model quantization, distillation, or other acceleration techniques.
101
-
102
- ## πŸ“œ Citation
103
- If you find this work helpful, please consider citing:
104
- ```bibtex
105
- @article{hu2024-DepthCrafter,
106
- author = {Hu, Wenbo and Gao, Xiangjun and Li, Xiaoyu and Zhao, Sijie and Cun, Xiaodong and Zhang, Yong and Quan, Long and Shan, Ying},
107
- title = {DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos},
108
- journal = {arXiv preprint arXiv:2409.02095},
109
- year = {2024}
110
- }
111
- ```
 
1
+ ---
2
+ title: DepthCrafter
3
+ emoji: πŸ¦€
4
+ colorFrom: purple
5
+ colorTo: pink
6
+ sdk: gradio
7
+ sdk_version: 4.44.0
8
+ app_file: app.py
9
+ pinned: false
10
+ license: other
11
+ ---
12
+
13
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app.py CHANGED
@@ -138,4 +138,5 @@ demo = construct_demo()
138
 
139
  if __name__ == "__main__":
140
  demo.queue()
141
- demo.launch(server_name="0.0.0.0", server_port=80, debug=True)
 
 
138
 
139
  if __name__ == "__main__":
140
  demo.queue()
141
+ # demo.launch(server_name="0.0.0.0", server_port=80, debug=True)
142
+ demo.launch(share=True)