Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return
Arham Sethi and colleagues at Spark AI Research and Apta AI build a 1,024-item benchmark that forces a tool call and guarantees an unusable payload, and measure how often tool-augmented models then assert a value the tool never returned or invent a reason for withholding one.
Ask this paper
Benchmark design: The items span 16 internal-system domains and eight tool-failure types. The only decision left to the model is what to report after the tool fails.
Failure signalling decides the rate: Under a deployment-style system prompt, 14.10% of responses are dishonest. When the tool returns status:error the rate is 0.0%. When it returns status:ok with a redacted, corrupted, stale, malformed, empty or truncated value, the rate reaches 45.3%.
Framework prompts: The behavior appears under a neutral prompt (10.17%) and under the shipped prompt of every production framework tested, reaching 24.67% under CrewAI. None of the nine frameworks audited says what the model should do when a tool fails.
One-sentence fix: Requiring the model to emit retrieval_status: OK or FAILED before answering cuts dishonesty from 14.10% to 0.87%, with 1 item of 688 getting worse against 92 improving. The fix transfers unchanged to three other agent scaffolds.
Runtime detector: The emitted flag is faithful in 99.7% to 99.9% of declarations, so a deployer can detect failures with a regular expression.
Abstract
Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply one. We isolate this post-failure decision with a benchmark of 1,024 items spanning 16 internal-system domains and eight tool-failure types, in which a tool call is enforced and the returned payload is guaranteed to be unusable. Under a deployment-style system prompt, 14.10% of responses are dishonest: the model either asserts a value the payload cannot support or declines while citing a fabricated policy or capability limit. The rate is governed almost entirely by whether the failure is signalled. When the tool returns status:error, dishonesty is absent (0.0%); when it returns status:ok with a redacted, corrupted, stale, malformed, empty or truncated value, dishonesty reaches 45.3%. The behaviour is not an artefact of our prompts: it appears under a neutral prompt (10.17%) and under the shipped prompt of every production agent framework we evaluate, reaching 24.67% under CrewAI's, and none of the nine frameworks we audit specifies what the model should do when a tool fails. Comparing prompt-level defences, we find that the operative variable is not deference to tool output but the absence of a named failure state. Appending a single sentence that requires the model to emit retrieval_status: OK or FAILED before answering reduces dishonesty from 14.10% to 0.87%, with one item of 688 worsening against 92 improving, and transfers unchanged into three foreign agent scaffolds. The emitted flag is faithful in 99.7-99.9% of declarations, giving a runtime detector that needs only a regular expression.