FLM-101B
First page
Paper summary
A 101B parameter open LLM trainable on a $100K budget through a growth-based training strategy.
Ask this paper
01
$100K budget for 101B: Trains a 101B model on 0.31TB tokens at a total compute cost of approximately $100K - remarkable for a frontier-scale parameter count.
02
Progressive growth strategy: Rather than training 101B from scratch, trains three models sequentially with each larger model inheriting from its smaller predecessor.
03
50%+ cost reduction: The aggressive growth strategy reduces total training cost by more than 50% compared to from-scratch training.
04
Open-science contribution: Releases the 101B model, providing a transparent reference for how far careful training-strategy design can stretch a limited budget.