Spaces:
Runtime error
Runtime error
Commit
·
53a2105
1
Parent(s):
c24da45
First0
Browse files
README.md
CHANGED
@@ -1,85 +1,6 @@
|
|
1 |
-
# Convolutional Reconstruction Model
|
2 |
-
|
3 |
-
Official implementation for *CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model*.
|
4 |
-
|
5 |
-
**CRM is a feed-forward model which can generate 3D textured mesh in 10 seconds.**
|
6 |
-
|
7 |
-
## [Project Page](https://ml.cs.tsinghua.edu.cn/~zhengyi/CRM/) | [Arxiv](https://arxiv.org/abs/2403.05034) | [HF-Demo](https://huggingface.co/spaces/Zhengyi/CRM) | [Weights](https://huggingface.co/Zhengyi/CRM)
|
8 |
-
|
9 |
-
https://github.com/thu-ml/CRM/assets/40787266/8b325bc0-aa74-4c26-92e8-a8f0c1079382
|
10 |
-
|
11 |
-
## Try CRM 🍻
|
12 |
-
* Try CRM at [Huggingface Demo](https://huggingface.co/spaces/Zhengyi/CRM).
|
13 |
-
* Try CRM at [Replicate Demo](https://replicate.com/camenduru/crm). Thanks [@camenduru](https://github.com/camenduru)!
|
14 |
-
|
15 |
-
## Install
|
16 |
-
|
17 |
-
### Step 1 - Base
|
18 |
-
|
19 |
-
Install package one by one, we use **python 3.9**
|
20 |
-
|
21 |
-
```bash
|
22 |
-
pip install torch==1.13.0+cu117 torchvision==0.14.0+cu117 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu117
|
23 |
-
pip install torch-scatter==2.1.1 -f https://data.pyg.org/whl/torch-1.13.1+cu117.html
|
24 |
-
pip install kaolin==0.14.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-1.13.1_cu117.html
|
25 |
-
pip install -r requirements.txt
|
26 |
-
```
|
27 |
-
|
28 |
-
besides, one by one need to install xformers manually according to the official [doc](https://github.com/facebookresearch/xformers?tab=readme-ov-file#installing-xformers) (**conda no need**), e.g.
|
29 |
-
|
30 |
-
```bash
|
31 |
-
pip install ninja
|
32 |
-
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
|
33 |
-
```
|
34 |
-
|
35 |
-
### Step 2 - Nvdiffrast
|
36 |
-
|
37 |
-
Install nvdiffrast according to the official [doc](https://nvlabs.github.io/nvdiffrast/#installation), e.g.
|
38 |
-
|
39 |
-
```bash
|
40 |
-
pip install git+https://github.com/NVlabs/nvdiffrast
|
41 |
-
```
|
42 |
-
|
43 |
-
|
44 |
-
|
45 |
-
## Inference
|
46 |
-
|
47 |
-
We suggest gradio for a visualized inference.
|
48 |
-
|
49 |
-
```
|
50 |
-
gradio app.py
|
51 |
-
```
|
52 |
-
|
53 |
-

|
54 |
-
|
55 |
-
For inference in command lines, simply run
|
56 |
-
```bash
|
57 |
-
CUDA_VISIBLE_DEVICES="0" python run.py --inputdir "examples/kunkun.webp"
|
58 |
-
```
|
59 |
-
It will output the preprocessed image, generated 6-view images and CCMs and a 3D model in obj format.
|
60 |
-
|
61 |
-
**Tips:** (1) If the result is unsatisfatory, please check whether the input image is correctly pre-processed into a grey background. Otherwise the results will be unpredictable.
|
62 |
-
(2) Different from the [Huggingface Demo](https://huggingface.co/spaces/Zhengyi/CRM), this official implementation uses UV texture instead of vertex color. It has better texture than the online demo but longer generating time owing to the UV texturing.
|
63 |
-
|
64 |
-
## Todo List
|
65 |
-
- [x] Release inference code.
|
66 |
-
- [x] Release pretrained models.
|
67 |
-
- [ ] Optimize inference code to fit in low memery GPU.
|
68 |
-
- [ ] Upload training code.
|
69 |
-
|
70 |
-
## Acknowledgement
|
71 |
-
- [ImageDream](https://github.com/bytedance/ImageDream)
|
72 |
-
- [nvdiffrast](https://github.com/NVlabs/nvdiffrast)
|
73 |
-
- [kiuikit](https://github.com/ashawkey/kiuikit)
|
74 |
-
- [GET3D](https://github.com/nv-tlabs/GET3D)
|
75 |
-
|
76 |
-
## Citation
|
77 |
-
|
78 |
-
```
|
79 |
@article{wang2024crm,
|
80 |
-
title={
|
81 |
author={Zhengyi Wang and Yikai Wang and Yifei Chen and Chendong Xiang and Shuo Chen and Dajiang Yu and Chongxuan Li and Hang Su and Jun Zhu},
|
82 |
journal={arXiv preprint arXiv:2403.05034},
|
83 |
year={2024}
|
84 |
}
|
85 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
@article{wang2024crm,
|
2 |
+
title={Test new version},
|
3 |
author={Zhengyi Wang and Yikai Wang and Yifei Chen and Chendong Xiang and Shuo Chen and Dajiang Yu and Chongxuan Li and Hang Su and Jun Zhu},
|
4 |
journal={arXiv preprint arXiv:2403.05034},
|
5 |
year={2024}
|
6 |
}
|
|