InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU
Paper
•
2502.08910
•
Published
•
142
SYS
-
-
LM2: Large Memory Models
Paper • 2502.06049 • Published • 30 -
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Paper • 2502.11089 • Published • 139 -
Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity
Paper • 2502.13063 • Published • 65 -
Tensor Product Attention Is All You Need
Paper • 2501.06425 • Published • 84 -
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper • 2501.11873 • Published • 63 -
ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer
Paper • 2501.15570 • Published • 23 -
MoBA: Mixture of Block Attention for Long-Context LLMs
Paper • 2502.13189 • Published • 13 -
Thus Spake Long-Context Large Language Model
Paper • 2502.17129 • Published • 67