Pythia Scaling Suite

EleutherAI 's Collections

Quirky Models and Datasets

Pile-T5

Quirky Math (archive)

Pile-T5 T5x Checkpoints

Sparse Autoencoders

updated Nov 21, 2023

Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia

Upvote

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Paper • 2304.01373 • Published Apr 3, 2023 • 9

Note The paper introduced the Pythia model suite
EleutherAI/pythia-14m

Text Generation • Updated Jul 26, 2023 • 137k • 20

Note This model was trained after the paper was released.
EleutherAI/pythia-70m

Updated Nov 21, 2023 • 113k • 57

Note This model was trained after the paper was released.
EleutherAI/pythia-160m

Text Generation • Updated Jul 9, 2023 • 141k • 27
EleutherAI/pythia-410m

Text Generation • Updated Jul 9, 2023 • 59.9k • 22
EleutherAI/pythia-1b

Text Generation • Updated Jul 9, 2023 • 29.5k • 33
EleutherAI/pythia-1.4b

Text Generation • Updated Jul 9, 2023 • 21.6k • 22
EleutherAI/pythia-2.8b

Text Generation • Updated Jun 9, 2023 • 34k • 29
EleutherAI/pythia-6.9b

Text Generation • Updated Jun 8, 2023 • 14.5k • 48
EleutherAI/pythia-12b

Text Generation • Updated Jul 9 • 6.84k • 131
EleutherAI/pythia-70m-deduped

Text Generation • Updated Jul 9, 2023 • 105k • 25
EleutherAI/pythia-160m-deduped

Text Generation • Updated Jul 9, 2023 • 25.7k • 3
EleutherAI/pythia-410m-deduped

Text Generation • Updated Jul 9, 2023 • 25.9k • 20
EleutherAI/pythia-1b-deduped

Text Generation • Updated Jul 10, 2023 • 39.9k • 17
EleutherAI/pythia-1.4b-deduped

Text Generation • Updated Jun 8, 2023 • 16.5k • 19
EleutherAI/pythia-2.8b-deduped

Text Generation • Updated Jul 9, 2023 • 7.51k • 14
EleutherAI/pythia-6.9b-deduped

Text Generation • Updated Jun 8, 2023 • 36.6k • 8
EleutherAI/pythia-12b-deduped

Text Generation • Updated Jun 8, 2023 • 7.77k • 51

Upvote