LLoCO: Learning Long Contexts Offline
Paper
•
2404.07979
•
Published
•
15
Long Context
-
-
LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
Paper • 2402.13753 • Published • 106 -
LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
Paper • 2402.11550 • Published • 12 -
LongAlign: A Recipe for Long Context Alignment of Large Language Models
Paper • 2401.18058 • Published • 21 -
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Paper • 2404.07143 • Published • 97 -
Long-Context Language Modeling with Parallel Context Encoding
Paper • 2402.16617 • Published -
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack
Paper • 2406.10149 • Published • 47 -
RULER: What's the Real Context Size of Your Long-Context Language Models?
Paper • 2404.06654 • Published • 32 -
Extending Llama-3's Context Ten-Fold Overnight
Paper • 2404.19553 • Published • 30 -
Make Your LLM Fully Utilize the Context
Paper • 2404.16811 • Published • 52 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 62 -
LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
Paper • 2406.00605 • Published -
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
Paper • 2402.02244 • Published • 1 -
Resonance RoPE: Improving Context Length Generalization of Large Language Models
Paper • 2403.00071 • Published • 19 -
Extending LLMs' Context Window with 100 Samples
Paper • 2401.07004 • Published • 14 -
Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models
Paper • 2406.11230 • Published • 34 -
Long Code Arena: a Set of Benchmarks for Long-Context Code Models
Paper • 2406.11612 • Published • 20 -
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Paper • 2406.16008 • Published • 6 -
LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
Paper • 2406.15319 • Published • 52 -
Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
Paper • 2406.16747 • Published • 15 -
Can Few-shot Work in Long-Context? Recycling the Context to Generate Demonstrations
Paper • 2406.13632 • Published • 5 -
LongIns: A Challenging Long-context Instruction-based Exam for LLMs
Paper • 2406.17588 • Published • 18 -
Training-Free Long-Context Scaling of Large Language Models
Paper • 2402.17463 • Published • 19 -
Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
Paper • 2406.17419 • Published • 12