RIPPLE: Recovery Internalization through Process-Level Policy Learning and Effect-Aware Credit for Long-Horizon Agentic Coding
Jingxuan Xu, Ken Deng and colleagues (National University of Singapore, Shanghai AI Laboratory, Tsinghua) train coding agents to act on their own critiques, addressing what they call the Critique-to-Action Handoff Gap.
Ask this paper
Gap. Long-horizon coding agents can observe a failure and write a plausible critique, then continue on the same wrong path.
Method. A prefix-only teacher gives evidence-grounded recovery supervision; after distillation, RL assigns stage-specific credit to reflection triggering, diagnosis and corrective action, gated by local validity signals.
Result. Across four repository-level SWE benchmarks, RIPPLE improves the base policy by a mean of 4.8 points with no critic at deployment.
When it helps. Gains concentrate where the prefix contains enough diagnostic evidence and the repo state is still reversible within budget.
Abstract
Long-horizon coding agents can observe a failure and generate a plausible critique yet continue along the same incorrect trajectory. We call this the Critique-to-Action Handoff Gap. We introduce RIPPLE, a recovery-oriented training framework that internalizes process-level guidance into a single self-recovering policy. A prefix-only teacher provides evidence-grounded recovery supervision without access to future actions or terminal outcomes. After lightweight distillation, reinforcement learning assigns stage-specific credit to reflection triggering, diagnosis, and corrective action. Local validity signals gate trajectory advantages so that intermediate recovery decisions do not indiscriminately inherit terminal outcomes. Across four repository-level software-engineering benchmarks, RIPPLE improves the base policy by an arithmetic mean of 4.8 score points, with no auxiliary critic at deployment. Controlled ablations isolate the benefits of local, stage-aligned credit, and matched-prefix interventions show that critique semantics affect subsequent correction. Recovery gains concentrate where the prefix contains sufficient diagnostic evidence and the repository state remains reversible within the remaining budget.