Commit
•
8499a6e
1
Parent(s):
590b4b9
Update README.md
Browse files
README.md
CHANGED
@@ -26,7 +26,7 @@ This official repository unveils the TransNormerLLM3 model along with its open-s
|
|
26 |
- **TransNormerLLM3-15B** is purely intergrated with **[Lightning Attention-2](http://arxiv.org/abs/2401.04658)**, which can maintain a **stable TGS** during training of **unlimited sequence lengths**, up until encountering firm limitations like GPU memory constraints.
|
27 |
- **Titoken** tokenizer is used with a total **vocabulary size** of about **100,000**.
|
28 |
<p align="center">
|
29 |
-
<img src="./images/TransNormer3.jpg" width="
|
30 |
</p>
|
31 |
|
32 |
### Pre-training Logbook
|
@@ -96,6 +96,6 @@ If you wish to cite our work, please use the following reference:
|
|
96 |
```
|
97 |
|
98 |
<p align="center">
|
99 |
-
<img src="./images/lightning3-leopard.jpg" width="
|
100 |
- OpenNLPLab @2024 -
|
101 |
</p>
|
|
|
26 |
- **TransNormerLLM3-15B** is purely intergrated with **[Lightning Attention-2](http://arxiv.org/abs/2401.04658)**, which can maintain a **stable TGS** during training of **unlimited sequence lengths**, up until encountering firm limitations like GPU memory constraints.
|
27 |
- **Titoken** tokenizer is used with a total **vocabulary size** of about **100,000**.
|
28 |
<p align="center">
|
29 |
+
<img src="./images/TransNormer3.jpg" width="65%" />
|
30 |
</p>
|
31 |
|
32 |
### Pre-training Logbook
|
|
|
96 |
```
|
97 |
|
98 |
<p align="center">
|
99 |
+
<img src="./images/lightning3-leopard.jpg" width="50%" />
|
100 |
- OpenNLPLab @2024 -
|
101 |
</p>
|