Elie Bakouch's picture

Elie Bakouch

eliebak

·

AI & ML interests

Training LLM's @ 🤗

Articles

SmolLM - blazingly fast and remarkably powerful

Organizations

eliebak's activity

upvoted a paper 5 days ago

Balancing Pipeline Parallelism with Vocabulary Parallelism

Paper • 2411.05288 • Published 12 days ago • 18

upvoted a collection about 1 month ago

LoLCATS

Linearizing LLMs with high quality and efficiency. We linearize the full Llama 3.1 model family -- 8b, 70b, 405b -- for the first time! • 4 items • Updated Oct 14 • 14

upvoted a paper about 1 month ago

Differential Transformer

Paper • 2410.05258 • Published Oct 7 • 166

upvoted a paper about 2 months ago

EuroLLM: Multilingual Language Models for Europe

Paper • 2409.16235 • Published Sep 24 • 24

upvoted a paper 2 months ago

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning

Paper • 2409.12183 • Published Sep 18 • 36

upvoted an article 2 months ago

Article

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

Sep 18

• 201

upvoted 2 papers 3 months ago

Enhancing Training Efficiency Using Packing with Flash Attention

Paper • 2407.09105 • Published Jul 12 • 14

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Paper • 2408.13359 • Published Aug 23 • 22

upvoted a collection 3 months ago

Power-LM

Dense & MoE LLMs trained with power learning rate scheduler. • 4 items • Updated Oct 17 • 15

upvoted 2 articles 3 months ago

Article

Introducing AuraFace: Open-Source Face Recognition and Identity Preservation Models

By

•

Aug 26

• 36

Article

MicroJAX

By

•

Aug 25

• 16

upvoted a paper 3 months ago

To Code, or Not To Code? Exploring Impact of Code in Pre-training

Paper • 2408.10914 • Published Aug 20 • 41

upvoted 2 collections 3 months ago

💻 Local SmolLMs

SmolLM models in MLC, ONNX and GGUF format for local applications + in-browser demos • 14 items • Updated Aug 20 • 45

🪐 SmolLM

A series of smol LLMs: 135M, 360M and 1.7B. We release base and Instruct models as well as the training corpus and some WebGPU demos • 12 items • Updated Aug 18 • 198

upvoted an article 3 months ago

Article

A failed experiment: Infini-Attention, and why we should keep trying?

Aug 14

• 50

upvoted a paper 3 months ago

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

Paper • 2408.07055 • Published Aug 13 • 65

upvoted an article 3 months ago

Article

Introduction to ggml

Aug 13

• 113

upvoted a paper 3 months ago

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters

Paper • 2408.04093 • Published Aug 7 • 4

upvoted an article 4 months ago

Article

The case for specialized pre-training: ultra-fast foundation models for dedicated tasks

By

•

Aug 4

• 26

upvoted a paper 4 months ago

The Llama 3 Herd of Models

Paper • 2407.21783 • Published Jul 31 • 107