zxhezexin commited on
Commit
5992d4a
1 Parent(s): 797cdd5

Clean README

Browse files
Files changed (1) hide show
  1. README.md +0 -119
README.md CHANGED
@@ -9,122 +9,3 @@ app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
  ---
12
-
13
-
14
- # OpenLRM: Open-Source Large Reconstruction Models
15
-
16
- [![Code License](https://img.shields.io/badge/Code%20License-Apache_2.0-yellow.svg)](LICENSE)
17
- [![Weight License](https://img.shields.io/badge/Weight%20License-CC%20By%20NC%204.0-red)](LICENSE_WEIGHT)
18
- [![LRM](https://img.shields.io/badge/LRM-Arxiv%20Link-green)](https://arxiv.org/abs/2311.04400)
19
-
20
- [![HF Models](https://img.shields.io/badge/Models-Huggingface%20Models-bron)](https://huggingface.co/zxhezexin)
21
- [![HF Demo](https://img.shields.io/badge/Demo-Huggingface%20Demo-blue)](https://huggingface.co/spaces/zxhezexin/OpenLRM)
22
-
23
- <img src="assets/rendered_video/teaser.gif" width="75%" height="auto"/>
24
-
25
- <div style="text-align: left">
26
- <img src="assets/mesh_snapshot/crop.owl.ply00.png" width="12%" height="auto"/>
27
- <img src="assets/mesh_snapshot/crop.owl.ply01.png" width="12%" height="auto"/>
28
- <img src="assets/mesh_snapshot/crop.building.ply00.png" width="12%" height="auto"/>
29
- <img src="assets/mesh_snapshot/crop.building.ply01.png" width="12%" height="auto"/>
30
- <img src="assets/mesh_snapshot/crop.rose.ply00.png" width="12%" height="auto"/>
31
- <img src="assets/mesh_snapshot/crop.rose.ply01.png" width="12%" height="auto"/>
32
- </div>
33
-
34
- ## News
35
-
36
- - [2024.03.04] Version update v1.1. Release model weights trained on both Objaverse and MVImgNet. Codebase is majorly refactored for better usability and extensibility. Please refer to [v1.1.0](https://github.com/3DTopia/OpenLRM/releases/tag/v1.1.0) for details.
37
- - [2024.01.09] Updated all v1.0 models trained on Objaverse. Please refer to [HF Models](https://huggingface.co/zxhezexin) and overwrite previous model weights.
38
- - [2023.12.21] [Hugging Face Demo](https://huggingface.co/spaces/zxhezexin/OpenLRM) is online. Have a try!
39
- - [2023.12.20] Release weights of the base and large models trained on Objaverse.
40
- - [2023.12.20] We release this project OpenLRM, which is an open-source implementation of the paper [LRM](https://arxiv.org/abs/2311.04400).
41
-
42
- ## Setup
43
-
44
- ### Installation
45
- ```
46
- git clone https://github.com/3DTopia/OpenLRM.git
47
- cd OpenLRM
48
- ```
49
-
50
- ### Environment
51
- - Install requirements for OpenLRM first.
52
- ```
53
- pip install -r requirements.txt
54
- ```
55
- - Please then follow the [xFormers installation guide](https://github.com/facebookresearch/xformers?tab=readme-ov-file#installing-xformers) to enable memory efficient attention inside [DINOv2 encoder](openlrm/models/encoders/dinov2/layers/attention.py).
56
-
57
- ## Quick Start
58
-
59
- ### Pretrained Models
60
-
61
- - Model weights are released on [Hugging Face](https://huggingface.co/zxhezexin).
62
- - Weights will be downloaded automatically when you run the inference script for the first time.
63
- - Please be aware of the [license](LICENSE_WEIGHT) before using the weights.
64
-
65
- | Model | Training Data | Layers | Feat. Dim | Trip. Dim. | In. Res. | Link |
66
- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
67
- | openlrm-obj-small-1.1 | Objaverse | 12 | 512 | 32 | 224 | [HF](https://huggingface.co/zxhezexin/openlrm-obj-small-1.1) |
68
- | openlrm-obj-base-1.1 | Objaverse | 12 | 768 | 48 | 336 | [HF](https://huggingface.co/zxhezexin/openlrm-obj-base-1.1) |
69
- | openlrm-obj-large-1.1 | Objaverse | 16 | 1024 | 80 | 448 | [HF](https://huggingface.co/zxhezexin/openlrm-obj-large-1.1) |
70
- | openlrm-mix-small-1.1 | Objaverse + MVImgNet | 12 | 512 | 32 | 224 | [HF](https://huggingface.co/zxhezexin/openlrm-mix-small-1.1) |
71
- | openlrm-mix-base-1.1 | Objaverse + MVImgNet | 12 | 768 | 48 | 336 | [HF](https://huggingface.co/zxhezexin/openlrm-mix-base-1.1) |
72
- | openlrm-mix-large-1.1 | Objaverse + MVImgNet | 16 | 1024 | 80 | 448 | [HF](https://huggingface.co/zxhezexin/openlrm-mix-large-1.1) |
73
-
74
- Model cards with additional details can be found in [model_card.md](model_card.md).
75
-
76
- ### Prepare Images
77
- - We put some sample inputs under `assets/sample_input`, and you can quickly try them.
78
- - Prepare RGBA images or RGB images with white background (with some background removal tools, e.g., [Rembg](https://github.com/danielgatis/rembg), [Clipdrop](https://clipdrop.co)).
79
-
80
- ### Inference
81
- - Run the inference script to get 3D assets.
82
- - You may specify which form of output to generate by setting the flags `EXPORT_VIDEO=true` and `EXPORT_MESH=true`.
83
- - Please set default `INFER_CONFIG` according to the model you want to use. E.g., `infer-b.yaml` for base models and `infer-s.yaml` for small models.
84
- - An example usage is as follows:
85
-
86
- ```
87
- # Example usage
88
- EXPORT_VIDEO=true
89
- EXPORT_MESH=true
90
- INFER_CONFIG="./configs/infer-b.yaml"
91
- MODEL_NAME="zxhezexin/openlrm-mix-base-1.1"
92
- IMAGE_INPUT="./assets/sample_input/owl.png"
93
-
94
- python -m openlrm.launch infer.lrm --infer $INFER_CONFIG model_name=$MODEL_NAME image_input=$IMAGE_INPUT export_video=$EXPORT_VIDEO export_mesh=$EXPORT_MESH
95
- ```
96
-
97
- ## Training
98
- To be released soon.
99
-
100
- ## Acknowledgement
101
-
102
- - We thank the authors of the [original paper](https://arxiv.org/abs/2311.04400) for their great work! Special thanks to Kai Zhang and Yicong Hong for assistance during the reproduction.
103
- - This project is supported by Shanghai AI Lab by providing the computing resources.
104
- - This project is advised by Ziwei Liu and Jiaya Jia.
105
-
106
- ## Citation
107
-
108
- If you find this work useful for your research, please consider citing:
109
- ```
110
- @article{hong2023lrm,
111
- title={Lrm: Large reconstruction model for single image to 3d},
112
- author={Hong, Yicong and Zhang, Kai and Gu, Jiuxiang and Bi, Sai and Zhou, Yang and Liu, Difan and Liu, Feng and Sunkavalli, Kalyan and Bui, Trung and Tan, Hao},
113
- journal={arXiv preprint arXiv:2311.04400},
114
- year={2023}
115
- }
116
- ```
117
-
118
- ```
119
- @misc{openlrm,
120
- title = {OpenLRM: Open-Source Large Reconstruction Models},
121
- author = {Zexin He and Tengfei Wang},
122
- year = {2023},
123
- howpublished = {\url{https://github.com/3DTopia/OpenLRM}},
124
- }
125
- ```
126
-
127
- ## License
128
-
129
- - OpenLRM as a whole is licensed under the [Apache License, Version 2.0](LICENSE), while certain components are covered by [NVIDIA's proprietary license](LICENSE_NVIDIA). Users are responsible for complying with the respective licensing terms of each component.
130
- - Model weights are licensed under the [Creative Commons Attribution-NonCommercial 4.0 International License](LICENSE_WEIGHT). They are provided for research purposes only, and CANNOT be used commercially.
 
9
  pinned: false
10
  license: apache-2.0
11
  ---