Update README.md
Browse files
README.md
CHANGED
@@ -3,18 +3,27 @@ license: apache-2.0
|
|
3 |
---
|
4 |
**Base Model**: BLIP2-t5 pretrained version
|
5 |
|
6 |
-
**Finetune data**:
|
|
|
|
|
7 |
|
8 |
**Hyper-parameters**:
|
9 |
|
10 |
-
|
11 |
-
*
|
12 |
-
*
|
13 |
-
*
|
14 |
-
*
|
|
|
15 |
|
16 |
-
v1 (same as LLAVA):
|
17 |
-
* lr = 2e-5
|
18 |
-
* gbs = 32
|
19 |
-
* image size = 480
|
20 |
-
* weight decay = 0.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
4 |
**Base Model**: BLIP2-t5 pretrained version
|
5 |
|
6 |
+
**Finetune data**:
|
7 |
+
* LLAVA 150k (sample one pair of instruction-answer if multi-round conversations)
|
8 |
+
* MiniGPT4 3500 pairs
|
9 |
|
10 |
**Hyper-parameters**:
|
11 |
|
12 |
+
* BLIP2-flant5-xl + LLAVA (initial commits)
|
13 |
+
* v0:
|
14 |
+
* lr = 2e-5 --> 0.0 with cosine lr scheduler
|
15 |
+
* gbs = 32
|
16 |
+
* image size = 480
|
17 |
+
* weight decay = 0.05
|
18 |
|
19 |
+
* v1 (same as LLAVA):
|
20 |
+
* lr = 2e-5
|
21 |
+
* gbs = 32
|
22 |
+
* image size = 480
|
23 |
+
* weight decay = 0.0
|
24 |
+
|
25 |
+
* BLIP2-flant5-xl + MiniGPT4
|
26 |
+
* lr = 2e-5
|
27 |
+
* gbs = 32
|
28 |
+
* image size = 480
|
29 |
+
* weight decay = 0.0
|