Teaching Arithmetic to Small Transformers

Data format matters: Shows that reformulating arithmetic into explicit step-by-step data dramatically improves small-model accuracy and convergence.

Emergence from curriculum: Fine-grained reasoning traces enable small transformers to learn multi-digit arithmetic that would otherwise require orders-of-magnitude more scale.

High-quality data thesis: Supports the emerging 2023 thesis that instructive, well-formatted data beats brute-force scaling for specific skills.

Small-model research: Informed the later Phi-series (Phi-1, Phi-1.5, Phi-2) "textbooks are all you need" data-quality research program.