LIMO: Less Is More for Reasoning

Surprisingly few examples – With only 817 carefully curated training samples, the LIMO model achieves 57.1% accuracy on the AIME math competition and 94.8% on MATH. This is a giant leap from prior SFT-based models (which scored 6.5% and 59.2% respectively – using just 1% of the data those earlier approaches needed.

Generalization with less data? – LIMO shows impressive OOD generalization: a +40.5% absolute improvement on average across 10 diverse benchmarks, even outperforming models trained on 100× more data. This challenges the assumption that more data is always required for complex skills and that fine-tuning only leads to memorization.

“Less-Is-More” Hypothesis – The authors propose that if an LLM’s pre-training has already endowed it with rich knowledge, then only a minimal set of carefully designed examples (which they call “cognitive templates”) is needed to unlock advanced reasoning. Essentially, the model just needs to see how to use its knowledge, not thousands of repetitive problems.

Open-source suite – The complete LIMO training suite is released for the community, supporting further research on data-efficient reasoning. This work hints that small, high-quality datasets might yield state-of-the-art reasoning, lowering the barrier to fine-tuning powerful LLMs.