Generative AI for Math (OpenWebMath / MathPile)
First page
Paper summary
Releases a diverse, high-quality math-centric corpus of ~9.5B tokens designed for training math-capable foundation models.
Ask this paper
01
9.5B-token corpus: Curated from mathematical content across the web, textbooks, papers, and Q&A, rebalanced for math-specific token distribution.
02
Quality filtering: Applies math-specific filtering to surface content dense in symbolic notation, proofs, and problem solutions rather than surface-level mentions of math.
03
Diverse sources: Explicitly mixes proof-heavy formal math with applied problem-solving to avoid over-fitting to any single mathematical register.
04
Training signal: Positioned as a drop-in pretraining or continual-pretraining corpus to lift math reasoning in existing LLMs without changing the architecture.