RoboArchitect: Automating the Design of Robotic Agents with Persistent Memory
Shengguang Wu, Serena Yeung-Levy and colleagues at Stanford University present RoboArchitect, in which a coding model rewrites the control harness around a frozen robot policy from observed rollouts instead of finetuning the policy.
Ask this paper
Method. A coding model watches rollouts, diagnoses failures and revises the harness code that decides when to call the policy, consult vision tools or command the arm directly. The harness is optimized on one set of episodes and evaluated on held-out scenes and tasks.
Results. With no weight updates, average success rises from 9.7% to 43.3% on unseen scenes (4.5x) and from 12.7% to 40.2% on unseen tasks (3.2x) across several benchmarks and policies.
Out-of-domain gains. The largest gains occur where the benchmark is far from the policy's training data. Wrapping a general-purpose policy, the evolved harness often outperforms the finetuned version of that policy.
Learned routines and memory. The search produces control routines such as lifting a jammed arm clear or reaching in horizontally around an obstacle, and it adds a persistent memory that carries values like effective grasp depths across episodes.
Limits. Training-episode performance keeps rising across iterations, so the iteration budget caps the result reported.
Abstract
Recent work shows that a frozen robot policy performs far better inside an agent harness, where control code decides when to query the policy, consult vision tools, or command the arm directly. Yet today these harnesses are built by hand and redesigned for each new robot or benchmark. We show that harness design can be automated, offering a strong alternative to finetuning on a novel setup. In RoboArchitect, a coding model observes a robot's rollouts, diagnoses failures, and iteratively revises the harness around the frozen policy. The harness evolves on a set of episodes for its optimization and is evaluated on held-out scenes and tasks. Without updating any model weights, RoboArchitect raises average success from 9.7% to 43.3% on unseen scenes (4.5x) and from 12.7% to 40.2% on unseen tasks (3.2x), across multiple benchmarks and policies. The gains are largest where the benchmark is far from the policy's training data: wrapping a general-purpose policy, the evolved harness yields a specialist that often outperforms even the finetuned version. Intuitive control strategies are automatically designed, such as lifting a jammed arm clear or reaching in horizontally when an obstacle blocks the way from above. The harness also evolves its own persistent memory, carrying learned experience, such as effective grasp depths, into later decisions. These results suggest that substantial headroom in robotic manipulation lies in the agent harness, and its design can be effectively automated and optimized without weight updates.