Evolving Agent Interfaces
Saaket Agashe, Xin Eric Wang and colleagues at Simular introduce Agent S4, in which a meta-agent builds and revises the interfaces that computer-use agents act through, together with knowledge of when to use each, from unlabeled task trajectories.
Ask this paper
Meta-agent. A meta-agent examines task agents' trajectories on synthetic training tasks, without success labels, and constructs and revises interfaces and usage knowledge in a repository.
Population-based selection. Repositories are developed further or dropped according to their performance on validation tasks the meta-agent never sees, which favors changes that generalize beyond the seen tasks.
OSWorld V2. With Qwen3.8-27B and the evolved repository, Agent S4 reaches a 21.3% binary score, 3.8 times the same model with fixed interfaces. This is above GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%) at about 4.3% and 1.5% of their costs, and a new state of the art for open-source computer-use agents.
ScienceBoard. The evolved interfaces raise success from 41.4% to 58.6% with the same model.
Abstract
Computer-use agents are limited by the interfaces humans give them. An agent may type a spreadsheet cell by cell when a single line of code would do, or print all two thousand lines of a file through a shell to find one function. Both the interfaces and the knowledge of when to use each are built by hand, which cannot keep pace with new applications, workflows, and models. We argue that interface design should itself be an agent capability. We introduce Agent S4, an autonomous computer-use scientist that builds Evolving Agent Interfaces from task experience. A meta-agent examines task agents' trajectories on synthetic training tasks, without success labels, and constructs and revises interfaces and usage knowledge in a repository. Population-based evolution then selects which repositories to develop further by their performance on validation tasks the meta-agent never sees, favoring changes that generalize beyond the seen tasks. On OSWorld V2, Agent S4, with Qwen3.8-27B and the evolved repository of interfaces, sets a new state of the art in open-source computer-use agents, reaching 21.3% binary score, 3.8 times the same model with fixed interfaces. This is above both GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their respective costs. On ScienceBoard, the evolved interfaces improve the success rate from 41.4% to 58.6% with the same model. These results suggest that useful interfaces can be evolved from experience, making autonomous interface design a source of improvement complementary to advances in models and harnesses. We open-source our code at https://github.com/simular-ai/agent-s.