Evolving Self-tuning Benchmarks for Long-Horizon Tool Use
Hyunji Alex Nam, Michael Dennis, Natasha Jaques and Minsuk Chang (Google DeepMind, Stanford and the University of Washington) present TURTLE, an unsupervised environment design method in which three LLM agents, a proposer, a solver and a feedback generator, evolve existing tool-use tasks into harder and more varied test suites.
Ask this paper
Proposer reward. The proposer is rewarded for tasks that are difficult yet solvable, measured from the solver's empirical success rate, so generated tasks concentrate where the solver sometimes succeeds and sometimes fails.
Trace feedback. A feedback generator summarizes the solver's execution traces and passes the summary to the proposer, which uses it to target the solver's observed weaknesses.
Difficulty targeting. With Gemma 4 31B-it as the solver, the share of hard tasks on AppWorld rises by 165%, and evolved tasks require more reasoning steps and more simultaneous tool interactions.
Coverage. Evolution works on AppWorld, tau-Bench with a user simulator, and the multi-app AutomationBench, and the evolved tasks elicit a broader set of failure modes than the original tasks.
Abstract
For AI assistants to move beyond simple chat interfaces and reliably assist users with applications like Gmail, Venmo, and Slack, we need to improve their tool-use capabilities. However, existing tool use benchmarks are still limited and difficult to scale because they rely on human supervision or are too domain-specific. To make data generation scalable, we propose, TURTLE (**T**ool **U**se **R**elated **T**asks with **L**ong-Horizon **E**volution), an Unsupervised Environment Design (UED) approach to evolve existing tasks using a proposer, a solver, and a feedback generator large language model agents. The proposer is rewarded for difficult yet solvable tasks based on the solver's empirical performance, and the feedback generator summarizes the solver's execution traces for the proposer to exploit. TURTLE produces evolving, self-tuning test suites adaptable to different environment requirements, from supporting user simulators and dense document retrieval to interacting across multiple apps. Empirical results on AppWorld, $τ$-Bench, and AutomationBench using Gemma 4 31B-it show that the evolved tasks target the desired difficulty range (e.g., a 165\% increase in the percentage of hard tasks on AppWorld), increase task diversity (in terms of reasoning and simultaneous tool interactions), and elicit a broader range of failure modes than the original tasks.