Update README.md

README.md

@@ -23,7 +23,8 @@ This official repository unveils the TransNormerLLM3 model along with its open-s
 
 # TransNormerLLM3
 - **TransNormerLLM3-15B** features **14.83 billion** parameters. It is structured with **42 layers**, includes **40 attention heads**, and has a total **embedding size of 5120**.
-- Titoken tokenizer is used with a total **vocabulary size** of about **100,000**. 
+- **TransNormerLLM3-15B** is purely intergrated with **[Lightning Attention-2](http://arxiv.org/abs/2401.04658)**, which can maintain a stable TGS during training of unlimited sequence lengths, up until encountering firm limitations like GPU memory constraints.   
+- **Titoken** tokenizer is used with a total **vocabulary size** of about **100,000**. 
 - It incorporates **Simple GLU** for its channel mixer, **GLA** in the token mixer, and **SRMSNorm** for normalization.
 - In terms of position encoding, the first layer employs **LRPE with exponential decay**, whereas the subsequent layers continue with **exponential decay encoding**.