DataComp-LM: In search of the next generation of training sets for language models
Paper
•
2406.11794
•
Published
•
39
•
4
)))?!?(((
stereoplegic
AI & ML interests
None yet
Organizations
stereoplegic's activity
commented
a
paper
3 days ago
commented
a
paper
4 days ago
commented
a
paper
5 days ago
commented
3
papers
8 days ago
Hard ASH: Sparsity and the right optimizer make a continual learner
Paper
•
2404.17651
•
Published
•
2
MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 Updates
Paper
•
2306.01685
•
Published
•
2
CoRe Optimizer: An All-in-One Solution for Machine Learning
Paper
•
2307.15663
•
Published
•
2
commented
6
papers
14 days ago
QuickLLaMA: Query-aware Inference Acceleration for Large Language Models
Paper
•
2406.07528
•
Published
•
2
Effectively Compress KV Heads for LLM
Paper
•
2406.07056
•
Published
•
2
Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs
Paper
•
2406.02376
•
Published
•
2
SelfCP: Compressing Long Prompt to 1/12 Using the Frozen Large Language Model Itself
Paper
•
2405.17052
•
Published
•
2
LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
Paper
•
2406.00605
•
Published
•
2
Enhancing Fast Feed Forward Networks with Load Balancing and a Master Leaf Node
Paper
•
2405.16836
•
Published
•
2
commented
15
papers
15 days ago
Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
Paper
•
2406.06110
•
Published
•
2
SinkLoRA: Enhanced Efficiency and Chat Capabilities for Long-Context Large Language Models
Paper
•
2406.05678
•
Published
•
2
XL3M: A Training-free Framework for LLM Length Extension Based on Segment-wise Inference
Paper
•
2405.17755
•
Published
•
2
Length Generalization of Causal Transformers without Position Encoding
Paper
•
2404.12224
•
Published
•
1
•
2
LongVQ: Long Sequence Modeling with Vector Quantization on Structured Memory
Paper
•
2404.11163
•
Published
•
2
SMR: State Memory Replay for Long Sequence Modeling
Paper
•
2405.17534
•
Published
•
2
Universal In-Context Approximation By Prompting Fully Recurrent Models
Paper
•
2406.01424
•
Published
•
2
A Unified Implicit Attention Formulation for Gated-Linear Recurrent Sequence Models
Paper
•
2405.16504
•
Published
•
2
Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences
Paper
•
2406.08128
•
Published
•
2
Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks
Paper
•
2405.15731
•
Published
•
2
State-Free Inference of State-Space Models: The Transfer Function Approach
Paper
•
2405.06147
•
Published
•
2
LoCoCo: Dropping In Convolutions for Long Context Compression
Paper
•
2406.05317
•
Published
•
2
Parallelizing Linear Transformers with the Delta Rule over Sequence Length
Paper
•
2406.06484
•
Published
•
2
•
2
LongSSM: On the Length Extension of State-space Models in Language Modelling
Paper
•
2406.02080
•
Published
•
2
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
Paper
•
2406.07522
•
Published
•
34
•
4
commented
4
papers
16 days ago
Multilingual Large Language Models Are Not (Yet) Code-Switchers
Paper
•
2305.14235
•
Published
•
2
Do Llamas Work in English? On the Latent Language of Multilingual Transformers
Paper
•
2402.10588
•
Published
•
1
•
2
Hebbian Deep Learning Without Feedback
Paper
•
2209.11883
•
Published
•
2
Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translation
Paper
•
2405.19290
•
Published
•
2
commented
12
papers
17 days ago
Zero-Shot Tokenizer Transfer
Paper
•
2405.07883
•
Published
•
4
•
3
Super Tiny Language Models
Paper
•
2405.14159
•
Published
•
2
SpaceByte: Towards Deleting Tokenization from Large Language Modeling
Paper
•
2404.14408
•
Published
•
6
•
3
LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters
Paper
•
2405.16287
•
Published
•
10
•
2
D'OH: Decoder-Only random Hypernetworks for Implicit Neural Representations
Paper
•
2403.19163
•
Published
•
2
Byte-Level Recursive Convolutional Auto-Encoder for Text
Paper
•
1802.01817
•
Published
•
2
FIT: Far-reaching Interleaved Transformers
Paper
•
2305.12689
•
Published
•
1
•
2
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
Paper
•
2305.07185
•
Published
•
8
•
9
Prompting-based Synthetic Data Generation for Few-Shot Question Answering
Paper
•
2405.09335
•
Published
•
2
Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting
Paper
•
2310.09716
•
Published
•
2
TarGEN: Targeted Data Generation with Large Language Models
Paper
•
2310.17876
•
Published
•
2
CrossTune: Black-Box Few-Shot Classification with Label Enhancement
Paper
•
2403.12468
•
Published
•
2
commented
7
papers
18 days ago
ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation
Paper
•
2405.17057
•
Published
•
2
SemCoder: Training Code Language Models with Comprehensive Semantics
Paper
•
2406.01006
•
Published
•
2
NExT: Teaching Large Language Models to Reason about Code Execution
Paper
•
2404.14662
•
Published
•
4
•
2
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs
Paper
•
2405.16325
•
Published
•
1
•
2
Spectral Adapter: Fine-Tuning in Spectral Space
Paper
•
2405.13952
•
Published
•
2
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
Paper
•
2405.15179
•
Published
•
1
•
2
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections
Paper
•
2405.20271
•
Published
•
2
commented
6
papers
19 days ago
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining
Paper
•
2406.02214
•
Published
•
2
Sparse Matrix in Large Language Model Fine-tuning
Paper
•
2405.15525
•
Published
•
2
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors
Paper
•
2405.19597
•
Published
•
2
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
Paper
•
2405.17604
•
Published
•
1
•
2
Parameter-Efficient Fine-Tuning with Discrete Fourier Transform
Paper
•
2405.03003
•
Published
•
1
•
2
NOLA: Networks as Linear Combination of Low Rank Random Basis
Paper
•
2310.02556
•
Published
•
2
•
2
commented
4
papers
21 days ago
Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension
Paper
•
2404.17991
•
Published
•
5
Extended Mind Transformers
Paper
•
2406.02332
•
Published
•
2
Loki: Low-Rank Keys for Efficient Sparse Attention
Paper
•
2406.02542
•
Published
•
2
CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory
Paper
•
2402.13449
•
Published
•
2