Spaces:
Runtime error
Runtime error
upload resources
Browse files- .gitattributes +4 -0
- resources/Training_Instructions.md +61 -0
- resources/applications.jpg +3 -0
- resources/framework.jpg +0 -0
- resources/gif/demo-1.gif +3 -0
- resources/gif/demo-2.gif +3 -0
- resources/teaser.jpg +3 -0
.gitattributes
CHANGED
@@ -38,3 +38,7 @@ example/Donut.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
38 |
example/durian.mp4 filter=lfs diff=lfs merge=lfs -text
|
39 |
example/pillow_huskies.mp4 filter=lfs diff=lfs merge=lfs -text
|
40 |
example/wooden_car.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
38 |
example/durian.mp4 filter=lfs diff=lfs merge=lfs -text
|
39 |
example/pillow_huskies.mp4 filter=lfs diff=lfs merge=lfs -text
|
40 |
example/wooden_car.mp4 filter=lfs diff=lfs merge=lfs -text
|
41 |
+
resources/applications.jpg filter=lfs diff=lfs merge=lfs -text
|
42 |
+
resources/gif/demo-1.gif filter=lfs diff=lfs merge=lfs -text
|
43 |
+
resources/gif/demo-2.gif filter=lfs diff=lfs merge=lfs -text
|
44 |
+
resources/teaser.jpg filter=lfs diff=lfs merge=lfs -text
|
resources/Training_Instructions.md
ADDED
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Preparations
|
2 |
+
|
3 |
+
## Cloning the Repository
|
4 |
+
The repository contains submodules, thus please check it out with
|
5 |
+
```shell
|
6 |
+
# SSH
|
7 |
+
git clone git@github.com:EnVision-Research/LucidDreamer.git --recursive
|
8 |
+
```
|
9 |
+
or
|
10 |
+
```shell
|
11 |
+
# HTTPS
|
12 |
+
git clone https://github.com/EnVision-Research/LucidDreamer.git --recursive
|
13 |
+
```
|
14 |
+
## Setup
|
15 |
+
Our default, provided install method is based on Conda package.
|
16 |
+
Firstly, you need to create an virtual environment and install the submodoules we provide. (slightly difference from original [3DGS](https://github.com/graphdeco-inria/gaussian-splatting))
|
17 |
+
```shell
|
18 |
+
conda env create --file environment.yml
|
19 |
+
conda activate LucidDreamer
|
20 |
+
pip install submodules/diff-gaussian-rasterization/
|
21 |
+
pip install submodules/simple-knn/
|
22 |
+
```
|
23 |
+
Then, you need to install [Point-E](https://github.com/openai/point-e) follow the instruction under this repo:
|
24 |
+
```shell
|
25 |
+
https://github.com/openai/point-e
|
26 |
+
```
|
27 |
+
|
28 |
+
# Running
|
29 |
+
We will provide a detailed guideline of our implementation about the description of each hyperparameter and how to tune them later. Now, we release 9 config files for you to evaluate the effectiveness of our framework (all configs can be trained in a single RTX3090).
|
30 |
+
|
31 |
+
Firstly, you may need to change ```model_key:``` in the ```configs\<config_file>.yaml``` to link the local Pretrained Diffusion Models ( [Stable Diffusion 2.1-base](https://github.com/Stability-AI/StableDiffusion) in default)
|
32 |
+
|
33 |
+
|
34 |
+
Then, you can simply use:
|
35 |
+
|
36 |
+
```shell
|
37 |
+
python train.py --opt <path to config file>
|
38 |
+
```
|
39 |
+
or you can see an exmaple and use the script we provide after you identify the visualable GPU:
|
40 |
+
```shell
|
41 |
+
bash train.sh
|
42 |
+
```
|
43 |
+
We provide config files in ```configs\``` that serve for different tasks:
|
44 |
+
|
45 |
+
Text-to-3D generation:
|
46 |
+
```shell
|
47 |
+
axe.yaml
|
48 |
+
bagel.yaml
|
49 |
+
cat_armor.yaml
|
50 |
+
crown.yaml
|
51 |
+
football_helmet.yaml
|
52 |
+
hamburger.yaml
|
53 |
+
white_hair_ironman.yaml
|
54 |
+
zombie_joker.yaml
|
55 |
+
```
|
56 |
+
|
57 |
+
Personalized Text-to-3D:
|
58 |
+
```shell
|
59 |
+
ts_lora.yaml
|
60 |
+
```
|
61 |
+
You can also use your own LoRA thourgh modify the: ```LoRA_path:```
|
resources/applications.jpg
ADDED
Git LFS Details
|
resources/framework.jpg
ADDED
resources/gif/demo-1.gif
ADDED
Git LFS Details
|
resources/gif/demo-2.gif
ADDED
Git LFS Details
|
resources/teaser.jpg
ADDED
Git LFS Details
|