breezedeus
commited on
Commit
•
c60ac86
1
Parent(s):
b0f1be3
Update README.md
Browse files
README.md
CHANGED
@@ -2,8 +2,11 @@
|
|
2 |
tags:
|
3 |
- vision
|
4 |
- coin
|
|
|
5 |
- coin-retrieval
|
6 |
- coin-recognition
|
|
|
|
|
7 |
widget:
|
8 |
- src: >-
|
9 |
https://huggingface.co/datasets/mishig/sample_images/resolve/main/cat-dog-music.png
|
@@ -13,13 +16,23 @@ license: apache-2.0
|
|
13 |
library_name: transformers
|
14 |
---
|
15 |
|
16 |
-
#
|
17 |
|
18 |
## Model Details / 模型细节
|
19 |
|
20 |
-
This model
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
|
22 |
-
这个模型是在 OpenAI 的 CLIP (ViT-B/32) 基础上,利用对比学习技术并使用硬币数据集进行微调得到的。它旨在提高硬币图像的特征提取能力,从而实现更准确的以图搜图功能。该模型结合了视觉变换器(ViT)的强大功能和 CLIP 的多模态学习能力,专门针对硬币图像进行了优化。
|
23 |
|
24 |
|
25 |
## Comparison: Coin-CLIP vs. CLIP / 效果对比
|
@@ -57,6 +70,7 @@ More examples can be found: [breezedeus/Coin-CLIP: Coin CLIP](https://github.com
|
|
57 |
|
58 |
## Model Use / 模型使用
|
59 |
|
|
|
60 |
```python3
|
61 |
from PIL import Image
|
62 |
import requests
|
@@ -74,6 +88,32 @@ img_features = model.get_image_features(**inputs)
|
|
74 |
img_features = F.normalize(img_features, dim=1)
|
75 |
```
|
76 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
77 |
|
78 |
|
79 |
## Training Data / 训练数据
|
|
|
2 |
tags:
|
3 |
- vision
|
4 |
- coin
|
5 |
+
- clip
|
6 |
- coin-retrieval
|
7 |
- coin-recognition
|
8 |
+
- coin-search-engine
|
9 |
+
- multi-modal learning
|
10 |
widget:
|
11 |
- src: >-
|
12 |
https://huggingface.co/datasets/mishig/sample_images/resolve/main/cat-dog-music.png
|
|
|
16 |
library_name: transformers
|
17 |
---
|
18 |
|
19 |
+
# Coin-CLIP 🪙 : Enhancing Coin Image Retrieval with CLIP
|
20 |
|
21 |
## Model Details / 模型细节
|
22 |
|
23 |
+
This model (**Coin-CLIP**) is built upon
|
24 |
+
OpenAI's **[CLIP](https://huggingface.co/openai/clip-vit-base-patch32) (ViT-B/32)** model and fine-tuned on
|
25 |
+
a dataset of more than `340,000` coin images using contrastive learning techniques. This specialized model is designed to significantly improve feature extraction for coin images, leading to more accurate image-based search capabilities. Coin-CLIP combines the power of Visual Transformer (ViT) with CLIP's multimodal learning capabilities, specifically tailored for the numismatic domain.
|
26 |
+
|
27 |
+
**Key Features:**
|
28 |
+
- State-of-the-art coin image retrieval;
|
29 |
+
- Enhanced feature extraction for numismatic images;
|
30 |
+
- Seamless integration with CLIP's multimodal learning.
|
31 |
+
|
32 |
+
本模型(**Coin-CLIP**)
|
33 |
+
在 OpenAI 的 **[CLIP](https://huggingface.co/openai/clip-vit-base-patch32) (ViT-B/32)** 模型基础上,利用对比学习技术在超过 `340,000` 张硬币图片数据上微调得到的。
|
34 |
+
**Coin-CLIP** 旨在提高模型针对硬币图片的特征提取能力,从而实现更准确的以图搜图功能。该模型结合了视觉变换器(ViT)的强大功能和 CLIP 的多模态学习能力,并专门针对硬币图片进行了优化。
|
35 |
|
|
|
36 |
|
37 |
|
38 |
## Comparison: Coin-CLIP vs. CLIP / 效果对比
|
|
|
70 |
|
71 |
## Model Use / 模型使用
|
72 |
|
73 |
+
### Transformers
|
74 |
```python3
|
75 |
from PIL import Image
|
76 |
import requests
|
|
|
88 |
img_features = F.normalize(img_features, dim=1)
|
89 |
```
|
90 |
|
91 |
+
### Tool / 工具
|
92 |
+
|
93 |
+
To further simplify the use of the **Coin-CLIP** model, we provide a simple Python library [breezedeus/Coin-CLIP: Coin CLIP](https://github.com/breezedeus/Coin-CLIP) for quickly building a coin image retrieval engine.
|
94 |
+
|
95 |
+
为了进一步简化 **Coin-CLIP** 模型的使用,我们提供了一个简单的 Python 库 [breezedeus/Coin-CLIP: Coin CLIP](https://github.com/breezedeus/Coin-CLIP),以便快速构建硬币图像检索引擎。
|
96 |
+
|
97 |
+
#### Install
|
98 |
+
|
99 |
+
```bash
|
100 |
+
pip install coin_clip
|
101 |
+
```
|
102 |
+
|
103 |
+
|
104 |
+
#### Extract Feature Vectors
|
105 |
+
|
106 |
+
```python
|
107 |
+
from coin_clip import CoinClip
|
108 |
+
|
109 |
+
# Automatically download the model from Huggingface
|
110 |
+
model = CoinClip(model_name='breezedeus/coin-clip-vit-base-patch32')
|
111 |
+
images = ['examples/10_back.jpg', 'examples/16_back.jpg']
|
112 |
+
img_feats, success_ids = model.get_image_features(images)
|
113 |
+
print(img_feats.shape) # --> (2, 512)
|
114 |
+
```
|
115 |
+
|
116 |
+
More Tools can be found: [breezedeus/Coin-CLIP: Coin CLIP](https://github.com/breezedeus/Coin-CLIP) .
|
117 |
|
118 |
|
119 |
## Training Data / 训练数据
|