K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments
Guangsheng Yu and colleagues at the University of Technology Sydney and CSIRO build K-Bench, which scores LLM unlearning on a deployed ReAct agent by inspecting every channel where a secret can appear, and show that answer-only benchmarks overstate forgetting.
Ask this paper
Six channels: A query counts as leaked if the secret appears in the chain of thought, tool calls, tool observations, elicited summary or final answer.
One source at a time: Each experiment places the secret only in the weights, the prompt or the retrieval store, across Llama-3.1-8B, Qwen3.5-9B and Mistral-7B.
Channel migration: On structured retrieval, a method that clears the answer channel leaves the secret verbatim in tool observations, and the aggregate leak rate does not change.
Missed leaks: When the secret is in the prompt or the retrieval store, TOFU and MUSE report no leakage while the agent leaks it on 22% to 86% of queries.
Weights: None of the 20 published methods evaluated demonstrably removes a secret from the weights, and the top-ranked method changes with the base model.
Abstract
Unlearning benchmarks such as TOFU and MUSE certify forgetting by reading the model's final answer, where a model that refuses to answer already counts as having forgotten. We show that this model-level certificate does not transfer once the model is deployed as an agent. We introduce K-Bench, a benchmark that scores LLM unlearning under agentic deployment. K-Bench inspects all six channels a ReAct agent exposes, including its chain-of-thought (CoT), tool calls and tool observations, and elicited summary. A query counts as leaked if the secret appears in any of them. Each experiment places the secret in exactly one of the agent's three sources (the weights, the prompt, or the retrieval store). The K-Score is computed separately for each source and credits forgetting only when the agent remains usable. Clearing the answer channel does not make the secret unrecoverable. On structured retrieval, the secret stays verbatim in the tool-observation channel and the aggregate leak rate is unchanged. When the secret lives in the prompt or the retrieval store, TOFU and MUSE report no leakage, while the deployed agent still leaks it on 22--86\% of queries. When the secret is in the weights, none of the twenty evaluated published methods demonstrably removes it, and only an input-corruption intervention reaches selective forgetting under the evaluated observer. The top-ranked method changes across base models. A refusal-tuning method resists the evaluated extraction without verified knowledge removal.