What Do CAE Simulation Agents Really Need Beyond a Generic Harness?
Jiasheng Shi (DP Technology) and Tianhan Zhang (Beihang University) ask what a CAE simulation agent still needs once a modern generic harness already supplies multi-turn reasoning, tool use and execution feedback, and find the answer is almost nothing except domain tutorials.
Ask this paper
Single agent beats the specialized stack: with information access and repair budget held fixed, a plain single-agent harness reaches 96.4% on FoamBench against 88.2% for multi-agent specialized CAE systems.
Execution feedback is the whole engine: repair rounds lift FoamBench from 71.8% with no repair to 96.4%. Scripted reflection, a staple of the CAE-agent literature, adds nothing measurable on top.
The one input that still helps is knowledge: solver tutorials supplied as domain knowledge produce the largest single measured gain, 80.9% to 96.4%.
Machinery was a weak-model artifact: multi-agent decomposition, domain retrieval and scripted reflection were designed around base models that could not hold a multi-turn loop. Modern harnesses subsume them.
Why it matters: this is the cleanest ablation yet of the claim that most domain agent frameworks are re-implementing harness features. The transferable lesson is to spend the budget on tutorials and repair rounds, not on topology.
Abstract
Computer-aided engineering (CAE) simulation is among the largest and most demanding areas of engineering, where setting up a solver such as OpenFOAM, FEniCS, or COMSOL takes real expertise. Large language model (LLM) agents promise to turn a natural-language request into a working simulation, and recent CAE agents add simulation-specific machinery: multi-agent decomposition, domain retrieval, and scripted reflection. That machinery suited weak base models; modern harnesses already supply multi-turn reasoning, tool use, and execution feedback. We ask what a CAE simulation agent still needs beyond a generic harness. With information access and repair budget held fixed, a single-agent harness matches or beats multi-agent specialized systems (FoamBench 96.4\% vs.\ 88.2\%). Ablations trace this to capabilities the harness already provides: execution-feedback repair lifts FoamBench from 71.8\% with no repair round to 96.4\%, while scripted reflection adds nothing. The one input that still helps is domain knowledge supplied as solver tutorials, our largest measured gain (80.9\% to 96.4\%).