s1: Simple test-time scaling
Free while signed in. Answers cite the passages they came from.
Researchers from Stanford, UW, and others introduce s1, a method to boost LLM performance by using extra compute at inference (“test-time scaling”). Key ideas include:
Small yet powerful dataset – They curated s1K, only 1,000 challenging questions with detailed reasoning traces, to fine-tune a 32B model. Despite the tiny data, this provides strong reasoning exemplars.
“Budget forcing” for reasoning – A new decoding trick appends the token “Wait” when the model tries to stop, forcing it to think longer. This leads the model to double-check and fix its reasoning step. By also cutting off overly long reasoning, they control inference time.
Big gains over OpenAI’s o1 – The resulting model (s1-32B) (a fine-tuned version of Qwen2.5-32B-Instruct) outperforms OpenAI’s o1-preview model by up to +27% on competition-level math questions (MATH & AIME24). Notably, with test-time scaling, it boosts accuracy on AIME24 from 50% to 57%, surpassing its own normal limit.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack