Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault Tolerance

Antonio Ken Iannillo, Joshua Owotogbe, Roberto Natella and colleagues (University of Luxembourg, Tilburg University, Gran Sasso Science Institute, University of Naples Federico II) introduce MCP-Injector, a fault-injection proxy for testing tool-calling agents against realistic MCP failures.
Ask this paper
Fault model. Calibrated on 57,240 issue and pull-request threads across 1,774 open-source MCP repositories, with permanent, transient and intermittent faults on seedable schedules.
Experiment. 1,350 cells crossing two gpt-oss backbones (20B, 120B), three agent stacks (custom plan-and-execute, OpenAI Agents SDK, LangChain) and three fault conditions.
Degradation. Under the permanent-heavy profile, task-completion score falls 60%, from 8.08 to 3.23 out of 10.
Scale. The 120B model keeps no larger share of its fault-free score (p = 0.96) and its completion rate under faults is lower (91.1% vs 95.6%). Differences between agent stacks matter more than model size. Accepted at NeurIPS 2026.
Abstract
LLM agents increasingly rely on external tools exposed via the Model Context Protocol (MCP), yet current benchmarks assume tools respond correctly and on time, leaving agent resilience to realistic tool failures untested. We introduce MCP-Injector, a drop-in, protocol-aware fault-injection middleware that interposes on the MCP tool-call path with controlled persistence (permanent, transient, intermittent) and seedable schedules. Its fault model is calibrated to the observed incident distribution (failure mode x persistence) of the public MCP ecosystem, mined from 57,240 issue and pull-request threads across 1,774 open-source MCP repositories. In a 1,350-cell experiment crossing two backbone sizes (20B, 120B), three agent stacks (custom multi-round plan-and-execute, OpenAI Agents SDK, LangChain), and three fault injection conditions, we find that faults drawn from this permanent-heavy stress profile cause severe quality degradation (task-completion score falls by 60%, from 8.08 to 3.23 out of 10), and that scale does not purchase fault tolerance for the two gpt-oss backbones we test: the 120B retains no larger share of its fault-free score under faults (resilience-ratio model, $p = 0.96$), and its completion rate under faults is, if anything, lower (91.1% vs. 95.6%; difference 4.4 points, 95% CI $[-0.2, 9.3]$). Agent stacks differ in how well they absorb faults, which suggests that framework-level error handling matters more than model scale. These results demonstrate that capability benchmarks alone can mask operational brittleness, motivating protocol-aware fault injection as a complementary evaluation dimension for tool-calling agents.