Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces
Roy Weiss and Yisroel Mirsky at Ben Gurion University with Eitam Sheetrit and Tomer Simon at Microsoft Security recover text generated by locally hosted LLMs by watching CPU cache activity during detokenization, a component present in default inference pipelines.
Ask this paper
It targets the detokenizer, not the model: Prior side-channel attacks needed shared data memory, CPU offloading or a Mixture-of-Experts architecture. This one attacks a component every standard inference pipeline runs.
Two-stage cache probing: Flush+Reload on shared tokenizer code detects when decoding happens, which lets the attacker time a Prime+Probe to isolate token-dependent cache activity rather than probing blindly.
Recovery is a clustering plus language-model pipeline: Noisy cache observations are clustered and then decoded with a language model to reconstruct semantically accurate text.
It generalizes across the stack: Evaluated across multiple datasets, hardware platforms, inference frameworks and model families, including real-world local deployments and agentic systems.
The blast radius is the point: The most widely used tokenizer implementations are susceptible and are embedded in many popular local LLM products and agent frameworks, with OpenClaw demonstrated directly.
Abstract
We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.