LoRA+: Efficient Low Rank Adaptation of Large Models
Paper
•
2402.12354
•
Published
•
6
Papers
Large Language Model (LLM) and NLP related papers.
-
-
The FinBen: An Holistic Financial Benchmark for Large Language Models
Paper • 2402.12659 • Published • 17 -
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization
Paper • 2402.13249 • Published • 11 -
TrustLLM: Trustworthiness in Large Language Models
Paper • 2401.05561 • Published • 66 -
MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks
Paper • 2311.07463 • Published • 13 -
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
Paper • 2305.07185 • Published • 9 -
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models
Paper • 2309.14717 • Published • 44 -
PEFTDebias : Capturing debiasing information using PEFTs
Paper • 2312.00434 • Published • 1 -
From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers
Paper • 2402.01911 • Published • 2 -
Empirical Study of PEFT techniques for Winter Wheat Segmentation
Paper • 2310.01825 • Published • 2 -
LoRA: Low-Rank Adaptation of Large Language Models
Paper • 2106.09685 • Published • 30 -
L4Q: Parameter Efficient Quantization-Aware Training on Large Language Models via LoRA-wise LSQ
Paper • 2402.04902 • Published • 4 -
Self-Instruct: Aligning Language Model with Self Generated Instructions
Paper • 2212.10560 • Published • 9 -
Efficient Training of Language Models to Fill in the Middle
Paper • 2207.14255 • Published • 1 -
Genie: Generative Interactive Environments
Paper • 2402.15391 • Published • 70 -
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
Paper • 2402.17193 • Published • 23 -
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Paper • 2402.17764 • Published • 603 -
DiffuseKronA: A Parameter Efficient Fine-tuning Method for Personalized Diffusion Model
Paper • 2402.17412 • Published • 21 -
MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT
Paper • 2402.16840 • Published • 23 -
LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models
Paper • 2402.10524 • Published • 22 -
FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models
Paper • 2402.10986 • Published • 77 -
StarCoder 2 and The Stack v2: The Next Generation
Paper • 2402.19173 • Published • 136 -
Beyond Language Models: Byte Models are Digital World Simulators
Paper • 2402.19155 • Published • 49 -
Training-Free Long-Context Scaling of Large Language Models
Paper • 2402.17463 • Published • 19 -
Design2Code: How Far Are We From Automating Front-End Engineering?
Paper • 2403.03163 • Published • 93 -
EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
Paper • 2403.02775 • Published • 11 -
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
Paper • 2402.10176 • Published • 36 -
OneBit: Towards Extremely Low-bit Large Language Models
Paper • 2402.11295 • Published • 23 -
Yi: Open Foundation Models by 01.AI
Paper • 2403.04652 • Published • 62 -
Stealing Part of a Production Language Model
Paper • 2403.06634 • Published • 90 -
Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU
Paper • 2403.06504 • Published • 53 -
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models
Paper • 2403.06764 • Published • 26 -
V3D: Video Diffusion Models are Effective 3D Generators
Paper • 2403.06738 • Published • 28 -
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Paper • 2403.05530 • Published • 61 -
MoAI: Mixture of All Intelligence for Large Language and Vision Models
Paper • 2403.07508 • Published • 74 -
GiT: Towards Generalist Vision Transformer through Universal Language Interface
Paper • 2403.09394 • Published • 25 -
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Paper • 2403.09611 • Published • 124 -
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
Paper • 2403.09629 • Published • 74 -
ORPO: Monolithic Preference Optimization without Reference Model
Paper • 2403.07691 • Published • 63 -
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
Paper • 2403.13372 • Published • 62 -
Evolutionary Optimization of Model Merging Recipes
Paper • 2403.13187 • Published • 50 -
Matryoshka: Stealing Functionality of Private ML Data by Hiding Models in Model
Paper • 2206.14371 • Published • 3 -
sDPO: Don't Use Your Data All at Once
Paper • 2403.19270 • Published • 40 -
Model Stock: All we need is just a few fine-tuned models
Paper • 2403.19522 • Published • 10 -
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper • 2404.02258 • Published • 104 -
Long-context LLMs Struggle with Long In-context Learning
Paper • 2404.02060 • Published • 35 -
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
Paper • 2403.04797 • Published • 1 -
ReFT: Representation Finetuning for Language Models
Paper • 2404.03592 • Published • 91 -
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
Paper • 2403.07816 • Published • 39 -
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
Paper • 2402.01739 • Published • 26 -
Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order
Paper • 2404.00399 • Published • 41 -
Latxa: An Open Language Model and Evaluation Suite for Basque
Paper • 2403.20266 • Published • 3 -
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
Paper • 2404.05961 • Published • 64 -
More Agents Is All You Need
Paper • 2402.05120 • Published • 51 -
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
Paper • 2404.00456 • Published • 4 -
Learn Your Reference Model for Real Good Alignment
Paper • 2404.09656 • Published • 82 -
A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
Paper • 2404.00579 • Published • 2 -
RoFormer: Enhanced Transformer with Rotary Position Embedding
Paper • 2104.09864 • Published • 10 -
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
Paper • 2404.03715 • Published • 60 -
Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications
Paper • 2404.13506 • Published • 1 -
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Paper • 2405.01535 • Published • 119 -
WildChat: 1M ChatGPT Interaction Logs in the Wild
Paper • 2405.01470 • Published • 61 -
Better & Faster Large Language Models via Multi-token Prediction
Paper • 2404.19737 • Published • 73 -
Granite Code Models: A Family of Open Foundation Models for Code Intelligence
Paper • 2405.04324 • Published • 22 -
Automating the Enterprise with Foundation Models
Paper • 2405.03710 • Published • 1 -
Stylus: Automatic Adapter Selection for Diffusion Models
Paper • 2404.18928 • Published • 14 -
Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory
Paper • 2405.08707 • Published • 27 -
tinyBenchmarks: evaluating LLMs with fewer examples
Paper • 2402.14992 • Published • 11 -
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
Paper • 2405.09220 • Published • 24 -
Zero-Shot Tokenizer Transfer
Paper • 2405.07883 • Published • 5 -
DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data
Paper • 2405.14333 • Published • 36 -
OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
Paper • 2405.11143 • Published • 34 -
An Introduction to Vision-Language Modeling
Paper • 2405.17247 • Published • 86 -
Spectrum: Targeted Training on Signal to Noise Ratio
Paper • 2406.06623 • Published • 10 -
CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery
Paper • 2406.08587 • Published • 15 -
Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
Paper • 2401.02994 • Published • 49 -
Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B
Paper • 2406.07394 • Published • 24 -
The Prompt Report: A Systematic Survey of Prompting Techniques
Paper • 2406.06608 • Published • 56 -
How Do Large Language Models Acquire Factual Knowledge During Pretraining?
Paper • 2406.11813 • Published • 30 -
GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks
Paper • 2406.12925 • Published • 23 -
Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts
Paper • 2406.12034 • Published • 14 -
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Paper • 2402.03300 • Published • 72 -
LiveMind: Low-latency Large Language Models with Simultaneous Inference
Paper • 2406.14319 • Published • 14 -
Extreme Compression of Large Language Models via Additive Quantization
Paper • 2401.06118 • Published • 12 -
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
Paper • 2312.08935 • Published • 4 -
Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data
Paper • 2406.14546 • Published • 2 -
HyperZcdotZcdotW Operator Connects Slow-Fast Networks for Full Context Interaction
Paper • 2401.17948 • Published • 2 -
Grokfast: Accelerated Grokking by Amplifying Slow Gradients
Paper • 2405.20233 • Published • 6 -
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
Paper • 2309.03883 • Published • 34 -
SpreadsheetLLM: Encoding Spreadsheets for Large Language Models
Paper • 2407.09025 • Published • 129 -
NeedleBench: Can LLMs Do Retrieval and Reasoning in 1 Million Context Window?
Paper • 2407.11963 • Published • 43 -
Fast Matrix Multiplications for Lookup Table-Quantized LLMs
Paper • 2407.10960 • Published • 11 -
LAMBDA: A Large Model Based Data Agent
Paper • 2407.17535 • Published • 34 -
TaskGen: A Task-Based, Memory-Infused Agentic Framework using StrictJSON
Paper • 2407.15734 • Published • 1 -
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
Paper • 2407.18901 • Published • 32 -
Trace is the New AutoDiff -- Unlocking Efficient Optimization of Computational Workflows
Paper • 2406.16218 • Published • 2 -
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Paper • 2406.17557 • Published • 87 -
Probabilistic Programming with Programmable Variational Inference
Paper • 2406.15742 • Published • 2 -
Running72💻
Vision Papers
All paper summaries read by Merve
-
Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
Paper • 2406.08464 • Published • 65 -
Improving Retrieval Augmented Language Model with Self-Reasoning
Paper • 2407.19813 • Published • 6 -
Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
Paper • 2408.06195 • Published • 63 -
Sleeping3📊
ACL2024 Papers
-
Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability
Paper • 2408.07852 • Published • 15 -
Writing in the Margins: Better Inference Pattern for Long Context Retrieval
Paper • 2408.14906 • Published • 138 -
Scaling Up Diffusion and Flow-based XGBoost Models
Paper • 2408.16046 • Published • 9 -
Advancing LLM Reasoning Generalists with Preference Trees
Paper • 2404.02078 • Published • 44 -
SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding
Paper • 2408.15545 • Published • 34 -
Draw an Audio: Leveraging Multi-Instruction for Video-to-Audio Synthesis
Paper • 2409.06135 • Published • 14 -
OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs
Paper • 2409.05152 • Published • 30 -
Scaling Synthetic Data Creation with 1,000,000,000 Personas
Paper • 2406.20094 • Published • 95 -
Training Language Models to Self-Correct via Reinforcement Learning
Paper • 2409.12917 • Published • 135 -
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
Paper • 2408.03314 • Published • 47 -
Self-Reflection in LLM Agents: Effects on Problem-Solving Performance
Paper • 2405.06682 • Published • 3 -
Improve Mathematical Reasoning in Language Models by Automated Process Supervision
Paper • 2406.06592 • Published • 26 -
3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion
Paper • 2409.12957 • Published • 18 -
Prithvi WxC: Foundation Model for Weather and Climate
Paper • 2409.13598 • Published • 39 -
A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?
Paper • 2409.15277 • Published • 34 -
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models
Paper • 2409.17146 • Published • 103
The Perfect Blend: Redefining RLHF with Mixture of Judges
Paper • 2409.20370 • Published • 4Note HF TRL PR for the GCPO paper implementation: https://github.com/huggingface/trl/pull/2155
-
Addition is All You Need for Energy-efficient Language Models
Paper • 2410.00907 • Published • 144 -
MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks
Paper • 2410.10563 • Published • 38 -
Benchmarking Agentic Workflow Generation
Paper • 2410.07869 • Published • 25 -
DocLLM: A layout-aware generative language model for multimodal document understanding
Paper • 2401.00908 • Published • 181 -
HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems
Paper • 2411.02959 • Published • 64 -
BitNet a4.8: 4-bit Activations for 1-bit LLMs
Paper • 2411.04965 • Published • 63 -
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
Paper • 2405.15071 • Published • 37 -
Spectra: A Comprehensive Study of Ternary, Quantized, and FP16 Language Models
Paper • 2407.12327 • Published • 77 -
SelfCodeAlign: Self-Alignment for Code Generation
Paper • 2410.24198 • Published • 21 -
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
Paper • 2411.04905 • Published • 111 -
Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
Paper • 2411.04996 • Published • 49 -
M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding
Paper • 2411.04952 • Published • 28 -
LLM-KT: A Versatile Framework for Knowledge Transfer from Large Language Models to Collaborative Filtering
Paper • 2411.00556 • Published • 1 -
Direct Preference Optimization Using Sparse Feature-Level Constraints
Paper • 2411.07618 • Published • 15 -
LLaVA-o1: Let Vision Language Models Reason Step-by-Step
Paper • 2411.10440 • Published • 109 -
Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
Paper • 2411.11504 • Published • 19 -
Continuous Speculative Decoding for Autoregressive Image Generation
Paper • 2411.11925 • Published • 15 -
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
Paper • 2411.10442 • Published • 63 -
Stream of Search (SoS): Learning to Search in Language
Paper • 2404.03683 • Published • 29 -
Does Prompt Formatting Have Any Impact on LLM Performance?
Paper • 2411.10541 • Published • 1 -
Understanding LLM Embeddings for Regression
Paper • 2411.14708 • Published • 2 -
OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation
Paper • 2412.02592 • Published • 20 -
Personalized Multimodal Large Language Models: A Survey
Paper • 2412.02142 • Published • 12 -
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Paper • 2412.04467 • Published • 103 -
Evaluating Language Models as Synthetic Data Generators
Paper • 2412.03679 • Published • 43 -
Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction
Paper • 2412.04454 • Published • 45 -
TrAct: Making First-layer Pre-Activations Trainable
Paper • 2410.23970 • Published • 1 -
Training Large Language Models to Reason in a Continuous Latent Space
Paper • 2412.06769 • Published • 55 -
LLM2CLIP: Powerful Language Model Unlock Richer Visual Representation
Paper • 2411.04997 • Published • 37 -
OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations
Paper • 2412.07626 • Published • 20 -
Generative Powers of Ten
Paper • 2312.02149 • Published • 5 -
Phi-4 Technical Report
Paper • 2412.08905 • Published • 87 -
Apollo: An Exploration of Video Understanding in Large Multimodal Models
Paper • 2412.10360 • Published • 126 -
Emergence of Abstractions: Concept Encoding and Decoding Mechanism for In-Context Learning in Transformers
Paper • 2412.12276 • Published • 13 -
OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain
Paper • 2412.13018 • Published • 39 -
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Paper • 2412.13663 • Published • 80