Commit
•
8499a6e
1
Parent(s):
590b4b9
Update README.md
Browse files
README.md
CHANGED
|
@@ -26,7 +26,7 @@ This official repository unveils the TransNormerLLM3 model along with its open-s
|
|
| 26 |
- **TransNormerLLM3-15B** is purely intergrated with **[Lightning Attention-2](http://arxiv.org/abs/2401.04658)**, which can maintain a **stable TGS** during training of **unlimited sequence lengths**, up until encountering firm limitations like GPU memory constraints.
|
| 27 |
- **Titoken** tokenizer is used with a total **vocabulary size** of about **100,000**.
|
| 28 |
<p align="center">
|
| 29 |
-
<img src="./images/TransNormer3.jpg" width="
|
| 30 |
</p>
|
| 31 |
|
| 32 |
### Pre-training Logbook
|
|
@@ -96,6 +96,6 @@ If you wish to cite our work, please use the following reference:
|
|
| 96 |
```
|
| 97 |
|
| 98 |
<p align="center">
|
| 99 |
-
<img src="./images/lightning3-leopard.jpg" width="
|
| 100 |
- OpenNLPLab @2024 -
|
| 101 |
</p>
|
|
|
|
| 26 |
- **TransNormerLLM3-15B** is purely intergrated with **[Lightning Attention-2](http://arxiv.org/abs/2401.04658)**, which can maintain a **stable TGS** during training of **unlimited sequence lengths**, up until encountering firm limitations like GPU memory constraints.
|
| 27 |
- **Titoken** tokenizer is used with a total **vocabulary size** of about **100,000**.
|
| 28 |
<p align="center">
|
| 29 |
+
<img src="./images/TransNormer3.jpg" width="65%" />
|
| 30 |
</p>
|
| 31 |
|
| 32 |
### Pre-training Logbook
|
|
|
|
| 96 |
```
|
| 97 |
|
| 98 |
<p align="center">
|
| 99 |
+
<img src="./images/lightning3-leopard.jpg" width="50%" />
|
| 100 |
- OpenNLPLab @2024 -
|
| 101 |
</p>
|