ReST Meets ReAct
Free while signed in. Answers cite the passages they came from.
Proposes a ReAct-style agent that improves itself via reinforced self-training on its own reasoning traces.
Self-critique ReAct: A ReAct-style agent with a self-critique step that evaluates its own reasoning and answers, generating a filterable trace dataset.
ReST-style iterative RL: Uses growing-batch RL from AI feedback to iteratively fine-tune on the agent's successful reasoning traces, improving over rounds without human labels.
Human-label-free: Minimizes human involvement; synthetic data with self-improvement from AI feedback is the primary training signal throughout.
Distillation to small models: The improved agent can be distilled into models 1-2 orders of magnitude smaller with comparable performance, dramatically cutting inference cost.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack