Pythia Scaling Suite

EleutherAI 's Collections

Quirky Models and Datasets

Pile-T5

Quirky Math (archive)

Pile-T5 T5x Checkpoints

Sparse Autoencoders

updated Nov 21, 2023

Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia

Upvote

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Paper • 2304.01373 • Published Apr 3, 2023 • 8

Note The paper introduced the Pythia model suite
EleutherAI/pythia-14m

Text Generation • Updated Jul 26, 2023 • 311k • 19

Note This model was trained after the paper was released.
EleutherAI/pythia-70m

Updated Nov 21, 2023 • 109k • 55

Note This model was trained after the paper was released.
EleutherAI/pythia-160m

Text Generation • Updated Jul 9, 2023 • 121k • 25
EleutherAI/pythia-410m

Text Generation • Updated Jul 9, 2023 • 34.7k • 21
EleutherAI/pythia-1b

Text Generation • Updated Jul 9, 2023 • 79.2k • 32
EleutherAI/pythia-1.4b

Text Generation • Updated Jul 9, 2023 • 24.8k • 22
EleutherAI/pythia-2.8b

Text Generation • Updated Jun 9, 2023 • 24.8k • 28
EleutherAI/pythia-6.9b

Text Generation • Updated Jun 8, 2023 • 26.1k • 48
EleutherAI/pythia-12b

Text Generation • Updated Jul 9 • 84.3k • 131
EleutherAI/pythia-70m-deduped

Text Generation • Updated Jul 9, 2023 • 80.3k • 24
EleutherAI/pythia-160m-deduped

Text Generation • Updated Jul 9, 2023 • 13k • 3
EleutherAI/pythia-410m-deduped

Text Generation • Updated Jul 9, 2023 • 79.5k • 20
EleutherAI/pythia-1b-deduped

Text Generation • Updated Jul 10, 2023 • 18.7k • 17
EleutherAI/pythia-1.4b-deduped

Text Generation • Updated Jun 8, 2023 • 11k • 19
EleutherAI/pythia-2.8b-deduped

Text Generation • Updated Jul 9, 2023 • 8.32k • 14
EleutherAI/pythia-6.9b-deduped

Text Generation • Updated Jun 8, 2023 • 12.4k • 8
EleutherAI/pythia-12b-deduped

Text Generation • Updated Jun 8, 2023 • 8.3k • 51

Upvote