abliterated-v3 Collection Latest gen of the abliterated models I've produced • 17 items • Updated 26 days ago • 69
PowerInfer-2: Fast Large Language Model Inference on a Smartphone Paper • 2406.06282 • Published 19 days ago • 35
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees Paper • 2406.16858 • Published 5 days ago • 1
World Model on Million-Length Video And Language With RingAttention Paper • 2402.08268 • Published Feb 13 • 35
HyperAttention: Long-context Attention in Near-Linear Time Paper • 2310.05869 • Published Oct 9, 2023 • 2
Creativity Has Left the Chat: The Price of Debiasing Language Models Paper • 2406.05587 • Published 20 days ago • 1
WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild Paper • 2406.04770 • Published 22 days ago • 23
The Geometry of Categorical and Hierarchical Concepts in Large Language Models Paper • 2406.01506 • Published 26 days ago • 3
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning Paper • 2312.01552 • Published Dec 4, 2023 • 27
Improving Alignment and Robustness with Short Circuiting Paper • 2406.04313 • Published 23 days ago • 1
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity Paper • 2305.13169 • Published May 22, 2023 • 3
Offline Regularised Reinforcement Learning for Large Language Models Alignment Paper • 2405.19107 • Published about 1 month ago • 12
LLMs achieve adult human performance on higher-order theory of mind tasks Paper • 2405.18870 • Published about 1 month ago • 15
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks Paper • 2402.04396 • Published Feb 6 • 1
The case for 4-bit precision: k-bit Inference Scaling Laws Paper • 2212.09720 • Published Dec 19, 2022 • 3
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series Paper • 2405.19327 • Published about 1 month ago • 43
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization Paper • 2405.15071 • Published May 23 • 31
SimPO: Simple Preference Optimization with a Reference-Free Reward Paper • 2405.14734 • Published May 23 • 8
LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report Paper • 2405.00732 • Published Apr 29 • 116
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? Paper • 2405.05904 • Published May 9 • 5
Orca 2: Teaching Small Language Models How to Reason Paper • 2311.11045 • Published Nov 18, 2023 • 69
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling Paper • 2401.16380 • Published Jan 29 • 46
view article Article 🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets By dvilasuero • 25 days ago • 63
view article Article Unleashing the Power of Logprobs in Language Models: A Practical Guide By Andyrasika • Jan 12 • 1
view article Article 💨 Introducing Notus: a DPO fine-tune of Zephyr with a focus on high-quality data By alvarobartt • Dec 1, 2023 • 1
view article Article LLM Comparison/Test: Llama 3 Instruct 70B + 8B HF/GGUF/EXL2 (20 versions tested and compared!) By wolfram • Apr 24 • 51
A Survey on Large Language Models with Multilingualism: Recent Advances and New Frontiers Paper • 2405.10936 • Published May 17 • 1
Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities Paper • 2404.17790 • Published Apr 27 • 2
Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models Paper • 2404.18796 • Published Apr 29 • 67
Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks Paper • 2404.14723 • Published Apr 23 • 9
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Paper • 2402.19427 • Published Feb 29 • 50
CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models Paper • 2404.08763 • Published Apr 12 • 1
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length Paper • 2404.08801 • Published Apr 12 • 62
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment Paper • 2404.12318 • Published Apr 18 • 14
Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models Paper • 2404.12387 • Published Apr 18 • 36
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning Paper • 2301.09626 • Published Jan 23, 2023 • 2
InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory Paper • 2402.04617 • Published Feb 7 • 4
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence Paper • 2404.05892 • Published Apr 8 • 28
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models Paper • 2404.07839 • Published Apr 11 • 40
Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models Paper • 2403.00417 • Published Mar 1 • 1
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Paper • 2012.15613 • Published Dec 31, 2020 • 1
Getting the most out of your tokenizer for pre-training and domain adaptation Paper • 2402.01035 • Published Feb 1 • 1
Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models Paper • 2404.02575 • Published Apr 3 • 46
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning Paper • 2403.17919 • Published Mar 26 • 16