OLMo 3

Allen Institute for AI introduces OLMo 3, a fully open language model family that releases the complete "model flow": every training stage, checkpoint, dataset, and dependency, enabling researchers to intervene at any development point. The release includes four specialized variants (Base, Think, Instruct, RL Zero) at 7B and 32B scales. - **Complete transparency with intervention points:** Unlike typical releases that only share final weights, OLMo 3 provides checkpoints from every major training milestone including initial pretraining, mid-training for programming/math, and long-context extension stages. This enables researchers to swap in domain-specific data during mid-training, adjust post-training for custom use cases, or build on earlier checkpoints for controlled experiments. - **Strong performance across reasoning and code:** OLMo 3-Think (32B) achieves 96.1% on MATH benchmark and 89.0% on IFEval instruction-following, while OLMo 3-Base (32B) scores 80.5% on GSM8k and 66.5% on HumanEval, outperforming comparable fully-open models like Marin and Apertus across reasoning, code generation, and math tasks. - **Dolma 3 dataset and training efficiency:** Trained on the 9.3 trillion token Dolma 3 corpus comprising web pages, scientific PDFs (processed with olmOCR), code, and math problems. Infrastructure improvements include 8x throughput gains in supervised fine-tuning and 4x efficiency improvements in RL training through in-flight weight updates and continuous batching. - **Full open-source ecosystem release:** All components released under permissive licenses including training/fine-tuning datasets, OlmoTrace (real-time tool for tracing outputs to training data), Olmo-core, Open Instruct, datamap-rs, and duplodocus production-grade tools for data processing and reproducible evaluation, plus complete technical report with ablations.