Weigao Sun

weigao266's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

upvoted 2 collections 6 months ago

Qwen2

Collection

Qwen2 language models, including pretrained and instruction-tuned models of 5 sizes, including 0.5B, 1.5B, 7B, 57B-A14B, and 72B. • 39 items • Updated Nov 28, 2024 • 353

SSMs

Collection

A collection of Mamba-2-based research models with 8B parameters trained on 3.5T tokens for comparison with Transformers. • 5 items • Updated about 14 hours ago • 26

upvoted 3 papers 9 months ago

CO2: Efficient Distributed Training with Full Communication-Computation Overlap

Paper • 2401.16265 • Published Jan 29, 2024 • 1

Linear Attention Sequence Parallelism

Paper • 2404.02882 • Published Apr 3, 2024 • 3

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Paper • 2401.04658 • Published Jan 9, 2024 • 25