OpenNLPLab commited on
Commit
590b4b9
1 Parent(s): 9e5c19e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -25,6 +25,9 @@ This official repository unveils the TransNormerLLM3 model along with its open-s
25
  - **TransNormerLLM3-15B** features **14.83 billion** parameters. It is structured with **42 layers**, includes **40 attention heads**, and has a total **embedding size of 5120**.
26
  - **TransNormerLLM3-15B** is purely intergrated with **[Lightning Attention-2](http://arxiv.org/abs/2401.04658)**, which can maintain a **stable TGS** during training of **unlimited sequence lengths**, up until encountering firm limitations like GPU memory constraints.
27
  - **Titoken** tokenizer is used with a total **vocabulary size** of about **100,000**.
 
 
 
28
 
29
  ### Pre-training Logbook
30
  * Realtime Track: https://api.wandb.ai/links/opennlplab/kip314lq
@@ -93,5 +96,6 @@ If you wish to cite our work, please use the following reference:
93
  ```
94
 
95
  <p align="center">
 
96
  - OpenNLPLab @2024 -
97
  </p>
 
25
  - **TransNormerLLM3-15B** features **14.83 billion** parameters. It is structured with **42 layers**, includes **40 attention heads**, and has a total **embedding size of 5120**.
26
  - **TransNormerLLM3-15B** is purely intergrated with **[Lightning Attention-2](http://arxiv.org/abs/2401.04658)**, which can maintain a **stable TGS** during training of **unlimited sequence lengths**, up until encountering firm limitations like GPU memory constraints.
27
  - **Titoken** tokenizer is used with a total **vocabulary size** of about **100,000**.
28
+ <p align="center">
29
+ <img src="./images/TransNormer3.jpg" width="75%" />
30
+ </p>
31
 
32
  ### Pre-training Logbook
33
  * Realtime Track: https://api.wandb.ai/links/opennlplab/kip314lq
 
96
  ```
97
 
98
  <p align="center">
99
+ <img src="./images/lightning3-leopard.jpg" width="67%" />
100
  - OpenNLPLab @2024 -
101
  </p>