Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference
Paper
•
2403.09636
•
Published
•
2
Lee Gao
leegao19
AI & ML interests
None yet
Organizations
leegao19's activity
upvoted
17
papers
7 months ago
Video Editing via Factorized Diffusion Distillation
Paper
•
2403.09334
•
Published
•
21
Scattered Mixture-of-Experts Implementation
Paper
•
2403.08245
•
Published
•
1
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
Paper
•
2401.15947
•
Published
•
48
Enhancing Multimodal Large Language Models with Vision Detection Models: An Empirical Study
Paper
•
2401.17981
•
Published
•
1
In-Context Learning Creates Task Vectors
Paper
•
2310.15916
•
Published
•
41
Function Vectors in Large Language Models
Paper
•
2310.15213
•
Published
•
1
What Algorithms can Transformers Learn? A Study in Length Generalization
Paper
•
2310.16028
•
Published
•
2
Empower Your Model with Longer and Better Context Comprehension
Paper
•
2307.13365
•
Published
•
1
Transformer Language Models without Positional Encodings Still Learn Positional Information
Paper
•
2203.16634
•
Published
•
5
Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level
Paper
•
2403.04690
•
Published
•
1
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
Paper
•
2310.05737
•
Published
•
4
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models
Paper
•
2403.06764
•
Published
•
25
MoAI: Mixture of All Intelligence for Large Language and Vision Models
Paper
•
2403.07508
•
Published
•
75
Unfamiliar Finetuning Examples Control How Language Models Hallucinate
Paper
•
2403.05612
•
Published
•
3
LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
Paper
•
2402.13753
•
Published
•
111
Zoology: Measuring and Improving Recall in Efficient Language Models
Paper
•
2312.04927
•
Published
•
2
upvoted
3
papers
8 months ago
Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey
Paper
•
2311.12351
•
Published
•
3
Sequence Parallelism: Long Sequence Training from System Perspective
Paper
•
2105.13120
•
Published
•
5
Yi: Open Foundation Models by 01.AI
Paper
•
2403.04652
•
Published
•
62
upvoted
a
collection
8 months ago
upvoted
3
papers
8 months ago
Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
Paper
•
2304.08818
•
Published
•
7
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Paper
•
2403.03507
•
Published
•
182
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
Paper
•
2309.12288
•
Published
•
3
upvoted
a
collection
8 months ago
upvoted
28
papers
8 months ago
AtP*: An efficient and scalable method for localizing LLM behaviour to components
Paper
•
2403.00745
•
Published
•
11
Functional Interpolation for Relative Positions Improves Long Context Transformers
Paper
•
2310.04418
•
Published
•
4
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Paper
•
2006.16236
•
Published
•
2
The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
Paper
•
2402.04347
•
Published
•
13
Resonance RoPE: Improving Context Length Generalization of Large Language Models
Paper
•
2403.00071
•
Published
•
22
Hydragen: High-Throughput LLM Inference with Shared Prefixes
Paper
•
2402.05099
•
Published
•
18
LongHeads: Multi-Head Attention is Secretly a Long Context Processor
Paper
•
2402.10685
•
Published
•
1
Do Large Language Models Latently Perform Multi-Hop Reasoning?
Paper
•
2402.16837
•
Published
•
24
MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT
Paper
•
2402.16840
•
Published
•
23
Evaluating Very Long-Term Conversational Memory of LLM Agents
Paper
•
2402.17753
•
Published
•
18
Priority Sampling of Large Language Models for Compilers
Paper
•
2402.18734
•
Published
•
16
Simple linear attention language models balance the recall-throughput tradeoff
Paper
•
2402.18668
•
Published
•
18
Beyond Language Models: Byte Models are Digital World Simulators
Paper
•
2402.19155
•
Published
•
49
Scaling Laws of RoPE-based Extrapolation
Paper
•
2310.05209
•
Published
•
6
Ring Attention with Blockwise Transformers for Near-Infinite Context
Paper
•
2310.01889
•
Published
•
10
World Model on Million-Length Video And Language With RingAttention
Paper
•
2402.08268
•
Published
•
36
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
Paper
•
2309.14509
•
Published
•
17
FIT: Far-reaching Interleaved Transformers
Paper
•
2305.12689
•
Published
•
1
Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer
Paper
•
2310.12442
•
Published
•
1
Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention
Paper
•
2312.08618
•
Published
•
11
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
Paper
•
2402.15220
•
Published
•
19
A Closer Look at the Limitations of Instruction Tuning
Paper
•
2402.05119
•
Published
•
5
Do Transformers Need Deep Long-Range Memory
Paper
•
2007.03356
•
Published
•
1
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models
Paper
•
2402.17177
•
Published
•
88
LoRA+: Efficient Low Rank Adaptation of Large Models
Paper
•
2402.12354
•
Published
•
6
MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Paper
•
2402.15627
•
Published
•
34
Mixtures of Experts Unlock Parameter Scaling for Deep RL
Paper
•
2402.08609
•
Published
•
34
In-Context Pretraining: Language Modeling Beyond Document Boundaries
Paper
•
2310.10638
•
Published
•
28
upvoted
2
collections
8 months ago
upvoted
a
paper
8 months ago
upvoted
3
papers
9 months ago
Efficient Tool Use with Chain-of-Abstraction Reasoning
Paper
•
2401.17464
•
Published
•
16
Rethinking Interpretability in the Era of Large Language Models
Paper
•
2402.01761
•
Published
•
21
Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization
Paper
•
2401.07793
•
Published
•
3
upvoted
a
collection
9 months ago