update readme

README.md

@@ -15,7 +15,7 @@ license_link: LICENSE
 </div>
 
 <div align="center">
-  <h2 align="center">Yi Vision Language Model</h2>
+  <h1 align="center">Yi Vision Language Model</h1>
 </div>
 
 
@@ -53,12 +53,14 @@ license_link: LICENSE
   - [Architecture](#architecture)
   - [Training](#training)
   - [Limitations](#limitations)
-  - [Citation](#citation)
 - [Why Yi-VL?](#why-yi-vl)
   - [Benchmarks](#benchmarks)
+  - [Showcases](#showcases)
 - [How to use Yi-VL?](#how-to-use-yi-vl)
   - [Quick start](#quick-start)
-- [Acknowledgements and attributions](#acknowledgements-and-attributions)
+- [Misc.](#misc)
+  - [Citation](#citation)
+  - [Acknowledgements and attributions](#acknowledgements-and-attributions)
 
 </details>
 
@@ -74,8 +76,6 @@ license_link: LICENSE
   
 - Yi-34B-VL is the **first** open-source 34B vision language model worldwide.
 
-<div align="right"> [ <a href="#yi-vision-language-model">Back to top ⬆️ </a> ] </div>
-  
 ## Models
 
 Yi-VL has released the following versions.
@@ -85,8 +85,6 @@ Model |       Download
 Yi-VL-34B |• [🤗 Hugging Face](https://huggingface.co/01-ai/Yi-VL-34B)
 Yi-VL-6B | • [🤗 Hugging Face](https://huggingface.co/01-ai/Yi-VL-6B)
 
-<div align="right"> [ <a href="#Yi-Vision-Language-Model">Back to top ⬆️ </a> ] </div>
-
 ## Features
 
 Yi-VL offers the following features:
@@ -99,24 +97,22 @@ Yi-VL offers the following features:
   
 - Fine-grained image resolution: Yi-VL supports image understanding at a higher resolution of 448x448.
 
-<div align="right"> [ <a href="#building-the-next-generation-of-bilingual-visual-language-models">Back to top ⬆️ </a> ] </div>
-
 ## Architecture
 
 Yi-VL adopts the [LLaVA](https://github.com/haotian-liu/LLaVA) architecture, which is composed of three primary components:
 
 - Vision Transformer (ViT): it's initialized with [CLIP ViT-H/14 model](https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K) and used for image encoding.
 
-- Projection Module: it's designed to align image features with text feature spcae, consists of a two-layer Multilayer Perceptron (MLP) with layer normalizations.
+- Projection Module: it's designed to align image features with text feature space, consisting of a two-layer Multilayer Perceptron (MLP) with layer normalizations.
   
 - Large Language Model (LLM): it's initialized with [Yi-6B-Chat](https://huggingface.co/01-ai/Yi-6B-Chat) or [Yi-34B-Chat](https://huggingface.co/01-ai/Yi-34B-Chat), demonstrating exceptional proficiency in understanding and generating both English and Chinese. To enhance the performance of Yi-VL models in bilingual multimodal understanding and generation, a rich dataset of bilingual image-text pairs is leveraged.
 
 ![Yi-VL architecture]()
 
-<div align="right"> [ <a href="#building-the-next-generation-of-bilingual-visual-language-models">Back to top ⬆️ </a> ] </div>
-
 ## Training 
 
+### Training process
+
 Yi-VL is trained to align visual information well to the semantic space of Yi LLM, which undergoes a comprehensive three-stage training process:
 
 - Stage 1: The parameters of ViT and the projection module are trained using an image resolution of 224&times;224. The LLM weights are frozen. The training leverages an image caption dataset comprising 100 million image-text pairs. The primary objective is to enhance the ViT's knowledge acquisition within our specified architecture and to achieve better alignment between the ViT and the LLM. 
@@ -125,9 +121,20 @@ Yi-VL is trained to align visual information well to the semantic space of Yi LL
      
 - Stage 3: The parameters of the entire model (that is, ViT, projection module, and LLM) are trained. The primary goal is to enhance the model's proficiency in multimodal chat interactions, thereby endowing it with the ability to seamlessly integrate and interpret visual and linguistic inputs. To this end, the training dataset encompasses a diverse range of sources, totalling approximately 1 million image-text pairs, including the data of image caption, VQA, grounding and so on. To ensure data balancing, we impose a cap on the maximum data contribution from any single source, restricting it to no more than 50,000 pairs. 
 
-In Stage 1 and 2, the global batch size, the learning rate, the gradient clip and the number of epoch are set to 4096, 1e-4, 0.5 and 1, respectively. In Stage 3, these parameters are adjusted to 256, 2e-5, 1.0 and 2. The training consumes 128 NVIDIA A100 GPUs. The total training time amounted to approximately 10 days for Yi-VL-34B and 3 days for Yi-VL-6B. 
+Below are the parameters configured for each stage.
+
+Stage | Global batch size | Learning rate | Gradient clip | NO. of epochs
+|---|---|---|---|---
+Stage 1, 2 |4096|1e-4|0.5|1
+Stage 3|256|2e-5|1.0|2
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/EGVHSWG4kAcX01xDaoeXS.png)
+
+### Training resource consumption
 
-<div align="right"> [ <a href="#building-the-next-generation-of-bilingual-visual-language-models">Back to top ⬆️ </a> ] </div>
+- The training consumes 128 NVIDIA A100 GPUs. 
+
+- The total training time amounted to approximately 10 days for Yi-VL-34B and 3 days for Yi-VL-6B. 
 
 ## Limitations
 
@@ -151,27 +158,33 @@ This is the initial release of the Yi-VL, which comes with some known limitation
     
 - Other limitations of the Yi LLM.
 
-## Citation
+# Why Yi-VL?
+  
+## Benchmarks
 
-If you find our work helpful, please feel free to cite us.
+Yi-VL outperforms all existing open-source models in [MMMU](https://mmmu-benchmark.github.io/#leaderboard) and [CMMMU](https://mmmu-benchmark.github.io/#leaderboard), two advanced benchmarks that include massive multi-discipline multimodal questions.
 
-```
-@article{tbd,
-  year={2024}
-}
-```
+- MMMU
 
-<div align="right"> [ <a href="#building-the-next-generation-of-bilingual-visual-language-models">Back to top ⬆️ </a> ] </div>
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/6YuSakMCg3D2AozixdoZ0.png)
 
-# Why Yi-VL?
+- CMMMU
   
-## Benchmarks
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/kCmXuwLbLvequ93kjh3mg.png)
 
-Yi-VL outperforms all existing open-source models in [MMMU](https://mmmu-benchmark.github.io/#leaderboard) and [CMMMU](https://mmmu-benchmark.github.io/#leaderboard), two advanced benchmarks that include massive multi-discipline multimodal questions.
+## Showcases
 
-![Yi-VL benchmark]()
+Yi-VL can describe images accurately and in detail with few hallucinations.
 
-<div align="right"> [ <a href="#building-the-next-generation-of-bilingual-visual-language-models">Back to top ⬆️ </a> ] </div>
+Below are some representative examples of detailed description and visual question answering, showcasing the capabilities of Yi-VL.
+
+- English
+  
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/iD83s2d8X-x6Acodp-SeO.png)
+
+- Chinese
+  
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/l_tLzugFtHk1dkVsFJE7B.png)
 
 # How to use Yi-VL?
 
@@ -193,11 +206,23 @@ Notes:
 
 - You need to set the parameter `mm_vision_tower` in `config.json` to the local ViT path.
 
-# Acknowledgements and attributions
+# Misc.
+
+## Citation
+
+If you find our work helpful, please feel free to cite us.
+
+```
+@article{tbd,
+  year={2024}
+}
+```
+
+## Acknowledgements and attributions
 
 This project makes use of open-source software/components. We acknowledge and are grateful to these developers for their contributions to the open-source community.
 
-## List of used open-source projects
+### List of used open-source projects
 
 1. LLaVA
   - Authors: Haotian Liu, Chunyuan Li, Qingyang Wu, Yuheng Li, and Yong Jae Lee
@@ -208,14 +233,14 @@ This project makes use of open-source software/components. We acknowledge and ar
 2. OpenClip
   - Authors: Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt
   - Source: https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K
-  - License: mit
+  - License: MIT
   - Description: The ViT is initialized using the weights of OpenClip.
 
-## License
+### License
 
-This project is licensed under the [yi-license](https://github.com/01-ai/Yi/blob/main/LICENSE). For more information on the license for this project, please see the LICENSE file in this repository.
+This project is licensed under the [yi-license](https://github.com/01-ai/Yi/blob/main/LICENSE). For more information on the license for this project, see the LICENSE file in this repository.
 
-## Notes
+### Notes
 
 - This attribution does not claim to cover all open-source components used. Please check individual components and their respective licenses for full details.
 - The use of the open-source components is subject to the terms and conditions of the respective licenses.