Pythia
Free while signed in. Answers cite the passages they came from.
First page
The curator’s take
Key pointsEleutherAI's suite for analyzing LLMs across training and scaling.
01
16-model suite: 16 LLMs trained on public data (The Pile) ranging from 70M to 12B parameters, all with identical training recipes.
02
Training checkpoints: Releases 154 training checkpoints per model, enabling analysis of learning dynamics across training.
03
Scale-controlled research: The consistent methodology across sizes enables rigorous scaling analyses without confounders.
04
Interpretability foundation: Became the foundational testbed for mechanistic interpretability research through 2024.
Every Monday
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack