README.md

---
license: apache-2.0
pipeline_tag: image-to-3d
---
# [ECCV 2024] VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models

[Porject page](https://junlinhan.github.io/projects/vfusion3d.html), [Paper link](https://arxiv.org/abs/2403.12034)

VFusion3D is a large, feed-forward 3D generative model trained with a small amount of 3D data and a large volume of synthetic multi-view data. It is the first work exploring scalable 3D generative/reconstruction models as a step towards a 3D foundation.

[VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models](https://junlinhan.github.io/projects/vfusion3d.html)<br>
[Junlin Han](https://junlinhan.github.io/), [Filippos Kokkinos](https://www.fkokkinos.com/), [Philip Torr](https://www.robots.ox.ac.uk/~phst/)<br>
GenAI, Meta and TVG, University of Oxford<br>
European Conference on Computer Vision (ECCV), 2024


## News

- [25.07.2024] Release weights and inference code for VFusion3D.



## Quick Start

Getting started with VFusion3D is super easy! 🤗 Here’s how you can use the model with Hugging Face:

### Load model directly
```python
from transformers import AutoModel

model = AutoModel.from_pretrained("jadechoghari/vfusion3d", trust_remote_code=True)
```

Check out our [demo app](https://huggingface.co/spaces/jadechoghari/vfusion3d-app) to see VFusion3D in action! 🤗

## Results and Comparisons

### 3D Generation Results
<img src='assets/gif1.gif' width=950>

<img src='assets/gif2.gif' width=950>

### User Study Results
<img src='assets/user.png' width=950>



## Acknowledgement

- This inference code of VFusion3D heavily borrows from [OpenLRM](https://github.com/3DTopia/OpenLRM).  

## Citation

If you find this work useful, please cite us:


```
@article{han2024vfusion3d,
  title={VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models},
  author={Junlin Han and Filippos Kokkinos and Philip Torr},
  journal={European Conference on Computer Vision (ECCV)},
  year={2024}
}
```

## License

- The majority of VFusion3D is licensed under CC-BY-NC, however portions of the project are available under separate license terms: OpenLRM as a whole is licensed under the Apache License, Version 2.0, while certain components are covered by NVIDIA's proprietary license.
- The model weights of VFusion3D is also licensed under CC-BY-NC.