FLM-101B
Free while signed in. Answers cite the passages they came from.
A 101B parameter open LLM trainable on a $100K budget through a growth-based training strategy.
$100K budget for 101B: Trains a 101B model on 0.31TB tokens at a total compute cost of approximately $100K - remarkable for a frontier-scale parameter count.
Progressive growth strategy: Rather than training 101B from scratch, trains three models sequentially with each larger model inheriting from its smaller predecessor.
50%+ cost reduction: The aggressive growth strategy reduces total training cost by more than 50% compared to from-scratch training.
Open-science contribution: Releases the 101B model, providing a transparent reference for how far careful training-strategy design can stretch a limited budget.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack