Xiaojie0903
commited on
Commit
•
47f7cde
1
Parent(s):
ce787ef
Update README.md
Browse files
README.md
CHANGED
@@ -6,5 +6,108 @@ language:
|
|
6 |
- en
|
7 |
metrics:
|
8 |
- accuracy
|
9 |
-
pipeline_tag:
|
10 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
- en
|
7 |
metrics:
|
8 |
- accuracy
|
9 |
+
pipeline_tag: image-classification
|
10 |
+
---
|
11 |
+
|
12 |
+
# **GenView Pretrained Models**
|
13 |
+
|
14 |
+
## Model Name
|
15 |
+
**GenView: Enhancing View Quality with Pretrained Generative Models**
|
16 |
+
|
17 |
+
### Summary
|
18 |
+
This repository hosts pretrained models developed as part of the GenView framework, introduced in the ECCV 2024 paper *GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning*. These models are designed for visual representation tasks, including image classification, multimodal learning, and feature extraction. GenView leverages generative models to enhance self-supervised learning by improving view quality and diversity.
|
19 |
+
|
20 |
+
---
|
21 |
+
|
22 |
+
## Table of Contents
|
23 |
+
1. [Model Details](#model-details)
|
24 |
+
2. [Evaluation](#evaluation)
|
25 |
+
3. [Citation](#citation)
|
26 |
+
4. [How to Download the Model](#how-to-download-the-model)
|
27 |
+
|
28 |
+
---
|
29 |
+
|
30 |
+
## Model Details
|
31 |
+
|
32 |
+
### **Model Description**
|
33 |
+
The GenView pretrained models include both convolutional architectures (e.g., ResNet50) and transformer-based architectures (e.g., ViT-B). These models utilize advanced self-supervised learning methods such as SimSiam, MoCo, and BYOL. By incorporating generative models for adaptive view generation, the framework delivers superior feature representations.
|
34 |
+
|
35 |
+
- **Developed by:** Xiaojie Li, Yibo Yang, Xiangtai Li, Jianlong Wu, Yue Yu, Bernard Ghanem, Min Zhang
|
36 |
+
- **Funded by:** Harbin Institute of Technology, Shenzhen; Peng Cheng Laboratory; KAUST; NTU
|
37 |
+
- **Shared by:** Xiaojie Li
|
38 |
+
- **Model type:** Self-supervised learning for vision tasks
|
39 |
+
- **Language:** Vision-focused (not language-specific)
|
40 |
+
- **License:** Apache 2.0
|
41 |
+
|
42 |
+
### **Model Sources**
|
43 |
+
- **Hugging Face Repository:** [GenView Pretrained Models](https://huggingface.co/Xiaojie0903/genview_pretrained_models)
|
44 |
+
- **GitHub Repository:** [GenView Official Code](https://github.com/xiaojieli0903/genview/)
|
45 |
+
- **Paper:** [GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning (ECCV 2024)](https://arxiv.org/abs/2403.12003)
|
46 |
+
|
47 |
+
---
|
48 |
+
|
49 |
+
## Evaluation
|
50 |
+
|
51 |
+
### **Testing Data**
|
52 |
+
Linear Probe evaluation was conducted using the ImageNet-1K dataset.
|
53 |
+
|
54 |
+
### **Metrics**
|
55 |
+
The models were evaluated based on Top-1 accuracy.
|
56 |
+
|
57 |
+
### **Results**
|
58 |
+
|
59 |
+
| Method | Backbone | Pretraining Epochs | Linear Probe Accuracy (%) |
|
60 |
+
|-------------------|--------------|---------------------|----------------------------|
|
61 |
+
| MoCo v2 + GenView| ResNet-50 | 200 | 70.0 |
|
62 |
+
| SwAV + GenView | ResNet-50 | 200 | 71.7 |
|
63 |
+
| SimSiam + GenView| ResNet-50 | 200 | 72.2 |
|
64 |
+
| BYOL + GenView | ResNet-50 | 200 | 73.2 |
|
65 |
+
| MoCo v3 + GenView| ResNet-50 | 100 | 72.7 |
|
66 |
+
| MoCo v3 + GenView| ResNet-50 | 300 | 74.8 |
|
67 |
+
| MoCo v3 + GenView| ViT-S | 300 | 74.5 |
|
68 |
+
| MoCo v3 + GenView| ViT-B | 300 | 77.8 |
|
69 |
+
|
70 |
+
---
|
71 |
+
|
72 |
+
## Citation
|
73 |
+
|
74 |
+
If you use these models, please cite the GenView paper:
|
75 |
+
|
76 |
+
```bibtex
|
77 |
+
@inproceedings{li2023genview,
|
78 |
+
author={Li, Xiaojie and Yang, Yibo and Li, Xiangtai and Wu, Jianlong and Yu, Yue and Ghanem, Bernard and Zhang, Min},
|
79 |
+
title={GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning},
|
80 |
+
year={2024},
|
81 |
+
booktitle={Proceedings of the European Conference on Computer Vision},
|
82 |
+
pages={306--325},
|
83 |
+
publisher="Springer"
|
84 |
+
}
|
85 |
+
```
|
86 |
+
---
|
87 |
+
## How to Download the Model
|
88 |
+
|
89 |
+
### **Downloading Models**
|
90 |
+
To download models, use the following commands:
|
91 |
+
|
92 |
+
#### Option 1: `wget`
|
93 |
+
```bash
|
94 |
+
# Replace {MODEL_FILE} with the specific model file name
|
95 |
+
wget https://huggingface.co/Xiaojie0903/genview_pretrained_models/resolve/main/{MODEL_FILE}
|
96 |
+
```
|
97 |
+
|
98 |
+
Example:
|
99 |
+
```bash
|
100 |
+
wget https://huggingface.co/Xiaojie0903/genview_pretrained_models/resolve/main/mocov3_resnet50_8xb512-amp-coslr-100e_in1k_genview.pth
|
101 |
+
```
|
102 |
+
|
103 |
+
#### Option 2: Hugging Face Python API
|
104 |
+
```python
|
105 |
+
from huggingface_hub import hf_hub_download
|
106 |
+
|
107 |
+
# Replace with your desired model file
|
108 |
+
file_path = hf_hub_download(
|
109 |
+
repo_id="Xiaojie0903/genview_pretrained_models",
|
110 |
+
filename="mocov3_resnet50_8xb512-amp-coslr-100e_in1k_genview.pth"
|
111 |
+
)
|
112 |
+
print(f"Model downloaded to {file_path}")
|
113 |
+
```
|