Never Stop Thinking: Continuous-Time Language Agents
Bojie Li and Noah Shi (Pine AI, University of Washington) show that an unmodified text model can think while listening and while speaking under a small interrupt-and-resume orchestrator, and introduce ReactiveBench to measure whether that thinking helps.
Ask this paper
Latency: A roughly 200-line orchestrator cuts live voice-pipeline latency by 19% overall and by half in the regime it targets.
ReactiveBench: 120 interactive scenarios scored against pre-registered binary requirements, plus a streaming track scored by exact correctness.
Judge problem: LLM judges reward visible reasoning: a large judged advantage for continuous thinking reverses sign under an independent judge, and judge-trained models complete fewer requirements when they think.
Training signal: A five-stage study finds verifiable objectives make thinking help, uniform rewards trade away whatever they omit, and on-policy RL over a type-shaped reward leaves streaming completion 25 points above the base model.
Abstract
Voice agents built on LLMs follow a rigid listen-think-speak loop that inserts seconds of dead air before every reply. We show that continuous-time cognition (thinking while listening and thinking while speaking) emerges from an unmodified text model under a lightweight interrupt-and-resume orchestrator, cutting live-pipeline latency by 19% overall and by half in the regime the mechanism targets. To measure whether continuous-time thinking improves what agents accomplish, we introduce ReactiveBench: 120 interactive scenarios scored against pre-registered binary requirements, plus a verifiable streaming track scored by exact correctness. ReactiveBench exposes a pitfall with broad consequences: LLM judges reward visible reasoning; a large judged "advantage" of continuous-time thinking reverses sign under an independent judge, and judge-trained models objectively complete fewer requirements when they think. A five-stage training study then locates the right signal at three levels. Its source: verifiable objectives turn thinking from harmful to helpful. Its structure: whatever a uniform reward omits, optimization trades away; brevity everywhere erodes multi-hop tool chaining. Its optimizer: preference optimization can only trade conflicting sub-goals against each other, while on-policy RL over a type-shaped reward improves every correctness axis at once, raising streaming completion from 48% to 73+/-5% across seeds and replicating at larger scale and on a second model. Orchestration makes continuous-time interaction possible; a verifiable signal, correctly sourced, shaped, and optimized, makes it good.