RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
Sibo Zhu and colleagues introduce RSIAgent, a training-free multi-agent framework in which curriculum, actor and verifier agents explore a new environment and build a reusable memory of its causal structure.
Ask this paper
Roles: A curriculum agent proposes tasks, an actor explores, and a verifier validates outcomes, and the system keeps environment-specific causal links between actions, conditions and consequences.
Broad then deep: Parallel broad exploration discovers environment structure, and focused deep exploration looks for hard cases, hidden constraints and boundary conditions.
Results: The frozen memory is reused on downstream tasks without updating model weights. On OSWorld-v2 and Agent's Last Exam it lets Kimi-K3 and GLM-5.3 outperform closed frontier models including GPT-6.
Abstract
Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce \textbf{RSIAgent}, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a \textbf{broad-then-deep} exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.