AgentXploit: Autonomous Repository-to-Runtime Red-Teaming for AI Agents
Weida Liang, Dawn Song and colleagues from NUS, UC Berkeley, UNC and UCSB introduce AgentXploit, a two-agent system for authorized white-box security audits of AI agent codebases, plus a benchmark of 72 reproducible vulnerabilities.
Ask this paper
Two roles. An Analyzer Agent traces attacker-controlled inputs through the repository to sensitive operations; an Exploiter Agent turns those paths into concrete attacks and revises them with runtime feedback.
Strict success criterion. Attacks must go through the task's attacker interface and be confirmed by an external verifier.
AgentXploit-Bench. 72 vulnerabilities across 12 open-source agent systems and frameworks, including prompt-injection-driven tool misuse, path traversal and command injection.
End-to-end results. 59.3% success over three runs versus 38.4% for Codex, and 46.3% for Codex with a matched token budget.
AgentDojo. With injection points given, the Exploiter alone reaches 79.2% attack success versus 52.7% for AgentVigil.
Abstract
AI agents combine language models with external data and tools that can modify files, call APIs, or execute code. Security failures can arise when adversarial content changes an agent's tool use or when the surrounding software contains vulnerabilities such as path traversal or command injection. We study authorized white-box pre-deployment auditing, where the auditor has access to the target repository and a controlled runtime, but successful attacks must still act through the task-defined attacker interface and be confirmed by an external verifier. We present AgentXploit, a two-role auditing system that separates repository-level attack-path discovery from runtime exploitation. The Analyzer Agent traces attacker-controlled inputs to sensitive operations and records code-supported candidate attack paths; the Exploiter Agent turns these paths into concrete attacks and revises them using runtime feedback. We also introduce AgentXploit-Bench, containing 72 reproducible vulnerabilities across 12 open-source AI-agent systems and frameworks. Across three runs, AgentXploit reaches 59.3% end-to-end success, compared with 38.4% for Codex. Under a token-budget-matched comparison, Codex reaches 46.3%. On AgentDojo, where injection points are provided, the Exploiter Agent reaches 79.2% attack success versus 52.7% for AgentVigil. These results highlight repository discovery and runtime exploitation as distinct challenges in end-to-end agent security auditing.