Training LLM Agents to Automate LLM Post-Training with RSI-Dojo
Shuzheng Gao, Michael R. Lyu and colleagues at CUHK, ByteDance, Tsinghua and SJTU build RSI-Dojo, 281 executable model-training environments derived from papers, and fine-tune agents on trajectories from them so that the model itself learns to run LLM post-training.
Ask this paper
RSI-Craft. An automatic pipeline collects papers from top venues, wraps their datasets and metrics in a unified interface and reproduces evaluation dependencies.
Filtering. Tasks are filtered on data size, execution reliability and how well they separate strong from weak agents, leaving 281 seed post-training tasks.
Trajectories. Frontier models run the tasks under an improved harness, and trajectories are kept only if they pass rubric-based and outcome-based filters.
Results. Fine-tuned Qwen3.5-9B and Qwen3.6-35B-A3B improve by 8.74 and 16.87 PostTrainBench points. RSI-Dojo-35B-A3B reaches 29.95 on PostTrainBench and 53.29 on PostTrainBench-Lite, similar to GLM-5.2 and GPT 5.5.
Scope. Tasks follow PostTrainBench style. Longer-horizon research settings are left to future work.
Abstract
Recursive Self-Improvement (RSI) aims to enable AI systems to autonomously build and improve AI systems themselves, including models, harnesses, and underlying infrastructure. Within this broader agenda, a central capability is enabling agents to train and improve base models. However, existing studies on automated model training primarily focused on workflow- or harness-level optimizations, rather than internalizing the ability to conduct post-training into the model itself. Developing this capability requires executable model training environments that can produce high-quality trajectories demonstrating how to select data, train models, diagnose failures, and iteratively improve results. In this paper, we introduce RSI-Dojo, a collection of executable model training environments designed to teach LLM agents to conduct post-training. To construct RSI-Dojo, we develop RSI-Craft, an automatic pipeline that converts human research artifacts into executable model training environments. RSI-Craft automatically collects research papers from top venues, instantiates datasets and metrics into a unified interface, and reproduces evaluation dependencies. It then filters tasks by data size, execution reliability, and discriminability, yielding RSI-Dojo with 281 seed model training tasks. Building on RSI-Dojo, we employ a trajectory collection method with an improved harness to collect high-quality model-training trajectories from frontier models, then retain examples through rubric-based and outcome-based filtering. We fine-tune Qwen3.5-9B and Qwen3.6-35B-A3B on these trajectories. The resulting RSI-Dojo-9B and RSI-Dojo-35B-A3B improve over their base models by 8.74 and 16.87 PostTrainBench points, respectively. RSI-Dojo-35B-A3B reaches 29.95 on PostTrainBench and 53.29 on PostTrainBench-Lite, matching the performance of frontier models such as GLM-5.2 and GPT 5.5.