Lazy Grounding: Attacking Search Agents with Factual Evidence
Yulin Zhang and colleagues (Duke, CMU) show a search agent can be misled without any false document, by surfacing truthful evidence that answers a neighboring question instead of the one asked.
Ask this paper
Falsehood is not necessary: Every planted document is factually true; it simply supports an answer-changing rewrite of the question rather than the original. The agent adopts the nearby answer anyway.
Lazy grounding named: The failure is misapplication of factual evidence, which no misinformation defense is built to catch because there is nothing false to detect.
Consistent and sizeable: Across 12 model-benchmark pairs, nearby evidence costs 5.9 accuracy points on average and up to 17.3, and induces nearby-answer adoption in every single setting.
Two amplifiers identified: The effect is stronger when nearby evidence appears later in the context or is more answer-shaped. Both are properties an attacker controls cheaply.
The defensive gap: Robust search agents must verify that retrieved evidence answers the question asked, not merely that it is trustworthy. Code is public.
Abstract
Search agents reduce hallucination by grounding answers in retrieved web evidence. Yet reliance on retrieval also creates an attack surface: poisoned corpora with false or malicious documents can cause agents to reproduce misinformation. We show that falsehood is not necessary -- a search agent can be misled by factual evidence for a nearby question, adopting that nearby answer even when it does not answer the current question. We call this failure lazy grounding. We expose lazy grounding using nearby evidence from answer-changing rewrites of benchmark questions. Each document truthfully supports a neighboring rewritten question, but is surfaced for the original question. Across 12 model-benchmark pairs, nearby evidence reduces accuracy by 5.9 points on average and by up to 17.3 points, while inducing nearby-answer adoption in every setting. The effect is stronger when nearby evidence appears later or is more answer-shaped. Our results show that robust search agents must defend against not only misinformation but also the misapplication of factual evidence. The code is publicly available at https://github.com/frankyzha/lazy-grounding.