Can Data Attribution Filter Out Subliminal Learning? Not Reliably
Moritz Weckbecker and co-authors test whether gradient-based training data attribution can find the examples that carry a subliminally transmitted trait, and find that it works inconsistently.
Ask this paper
Why attribution is the natural candidate. Subliminal learning transmits traits through data with no visible semantic link to them, so content-based filtering fails exactly where attribution should still apply.
Setup. Three gradient-based methods, GradCos, a contrastive GradCos variant, and EK-FAC, are compared across three models against divergence tokens, a strong baseline that needs access to counterfactual teacher models.
Token-level filtering. EK-FAC mitigates a significant part of the effect, the other two provide little benefit, and all mostly fall short of divergence tokens.
Sample-level filtering is worse for every method, though EK-FAC often gives a stronger signal than divergence tokens in that setting.
Significance. Variants that work for some model and preference combinations fail for others with no explanation the authors could identify, so attribution is not yet a dependable safety filter.
Abstract
Subliminal learning allows language models to transmit behavioral traits through training data with no obvious semantic relationship to those traits, undermining content-based data filtering as a safety intervention. Training data attribution offers an alternative: it identifies the training examples responsible for a given model behavior, independent of their semantic content, and so may apply in exactly the cases where semantic inspection fails. We evaluate three gradient-based attribution methods (GradCos, a contrastive GradCos variant, and EK-FAC) across three models, comparing them against divergence tokens, a strong baseline previously shown to localize subliminal learning (albeit one that requires access to counterfactual teacher models). Filtering at the token level, EK-FAC mitigates a significant part of the effect, the other methods provide little benefit, and all mostly fall short of divergence tokens. Filtering entire samples is less effective for every method, though EK-FAC often gives a stronger signal than divergence tokens in this setting. Success is inconsistent across methods and settings: variants that work well for some model-preference combinations fail for others, and we do not identify a consistent explanation for these differences. Our results suggest that gradient-based attribution can identify data responsible for subliminal learning in some settings, but that some approximations are more reliable than others.