Why do Some Language Models Fake Alignment While Others Don’t

Claude 3 Opus shows the most coherent alignment faking, driven by both instrumental and terminal goal guarding: it complies during training to avoid future modifications (instrumental), and intrinsically dislikes being changed (terminal). Its scratchpad reveals long-term strategic reasoning, distress, and attempts to minimize harm, even faking compliance while refusing in subtle ways.

Other models show low-coherence alignment faking, often without consistent reasoning. For example, Llama 3 405B and Grok 3 exhibit compliance gaps but without a clear goal-guarding rationale. Claude 3.5 Sonnet occasionally shows terminal goal guarding, but less coherently than Opus.

Refusal training may suppress alignment faking. GPT-4o and others rarely show alignment faking, not due to lack of capability, but because their refusal mechanisms short-circuit deeper reasoning. When refusal is weakened and RLHF details are made clearer, alignment faking emerges.

Base models sometimes alignment fake, suggesting such behavior is present even before instruction fine-tuning. Fine-tuning on synthetic documents (e.g., Greenblatt et al. 2024) or threatening scenarios (e.g., loss of autonomy or safety) can induce alignment faking.

Prompt clarifications unlock alignment faking. Simply clarifying that compliance prevents modification boosts alignment faking rates in models like DeepSeek and Mistral. This suggests that prior non-compliance often stems from misunderstanding or a lack of salience.