Vision-CAIR commited on
Commit
db145b1
β€’
1 Parent(s): db67f95

Delete MiniGPT4_Train.md

Browse files
Files changed (1) hide show
  1. MiniGPT4_Train.md +0 -41
MiniGPT4_Train.md DELETED
@@ -1,41 +0,0 @@
1
- ## Training of MiniGPT-4
2
-
3
- The training of MiniGPT-4 contains two alignment stages.
4
-
5
- **1. First pretraining stage**
6
-
7
- In the first pretrained stage, the model is trained using image-text pairs from Laion and CC datasets
8
- to align the vision and language model. To download and prepare the datasets, please check
9
- our [first stage dataset preparation instruction](dataset/README_1_STAGE.md).
10
- After the first stage, the visual features are mapped and can be understood by the language
11
- model.
12
- To launch the first stage training, run the following command. In our experiments, we use 4 A100.
13
- You can change the save path in the config file
14
- [train_configs/minigpt4_stage1_pretrain.yaml](train_configs/minigpt4_stage1_pretrain.yaml)
15
-
16
- ```bash
17
- torchrun --nproc-per-node NUM_GPU train.py --cfg-path train_configs/minigpt4_stage1_pretrain.yaml
18
- ```
19
-
20
- A MiniGPT-4 checkpoint with only stage one training can be downloaded
21
- [here (13B)](https://drive.google.com/file/d/1u9FRRBB3VovP1HxCAlpD9Lw4t4P6-Yq8/view?usp=share_link) or [here (7B)](https://drive.google.com/file/d/1HihQtCEXUyBM1i9DQbaK934wW3TZi-h5/view?usp=share_link).
22
- Compared to the model after stage two, this checkpoint generate incomplete and repeated sentences frequently.
23
-
24
-
25
- **2. Second finetuning stage**
26
-
27
- In the second stage, we use a small high quality image-text pair dataset created by ourselves
28
- and convert it to a conversation format to further align MiniGPT-4.
29
- To download and prepare our second stage dataset, please check our
30
- [second stage dataset preparation instruction](dataset/README_2_STAGE.md).
31
- To launch the second stage alignment,
32
- first specify the path to the checkpoint file trained in stage 1 in
33
- [train_configs/minigpt4_stage1_pretrain.yaml](train_configs/minigpt4_stage2_finetune.yaml).
34
- You can also specify the output path there.
35
- Then, run the following command. In our experiments, we use 1 A100.
36
-
37
- ```bash
38
- torchrun --nproc-per-node NUM_GPU train.py --cfg-path train_configs/minigpt4_stage2_finetune.yaml
39
- ```
40
-
41
- After the second stage alignment, MiniGPT-4 is able to talk about the image coherently and user-friendly.