Will the User Ever Know? Covert Indirect Prompt Injection on Tool-Using LLM Agents
Yunseok Lee, Yunji Kim and Woojin Lee split attack success rate into covert and overt success and show that whether the user ever notices is decided by what the agent does after the injection fires.
Ask this paper
ASR hides the thing that matters: A successful injection that the agent then reports in its final response gives the user a chance to catch it. One that leaves no trace does not. Counting both as success erases the difference.
CSR and OSR: Covert Success Rate counts successes leaving no trace in the final response; Overt Success Rate counts detectable ones.
The mechanism is the ReAct format: The final response summarizes the most recent action, so covert traces hand control back to the user task before ending while overt traces end at the attack. That is a structural property of the scaffold, not a model quirk.
ICoA weaponizes the observation: Induced Covert Attack steers the agent back to the user task after executing the injection, achieving the highest CSR across four target models on AgentDojo, 3.79 to 12.01 points over the strongest baseline.
Defensive implication: Any monitoring that relies on the final response as the audit surface is defeated by an attacker who simply orders the actions differently.
Abstract
As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent's final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving the user a chance to notice. We call these covert and overt successes. From the user's perspective, we decompose ASR into the Covert Success Rate (CSR), counting successes leaving no trace in the final response, and the Overt Success Rate (OSR), counting successes the user can detect. To understand what drives the gap, we analyze successful trajectories and find that the agent's behavior after the injection separates covert from overt: covert traces hand control back to the user task before ending, while overt traces end at the attack itself. This split follows from the ReAct format, where the final response summarizes the most recent action. Building on this observation, we propose ICoA (Induced Covert Attack), an IPI attack designed to induce covert outcomes by steering the agent back to the user task after executing the injection. Across four target models on AgentDojo, ICoA achieves the highest CSR, with gains of 3.79-12.01 percentage points over the strongest baseline.