Localization Does Not Imply Steerability: A Mechanistic Study of Sign Errors in LLM Mathematics
Shradha Agarwal and colleagues at Missouri University of Science and Technology locate where LLMs write wrong sign tokens in multi-step math, and show that circuits at the same depth in two models can differ greatly in how easily they can be steered.
Ask this paper
Location. Across six open models from 14B to 675B parameters, the wrong sign is committed by MLP layers in the last quarter of the network, a pattern already present in the Qwen base model.
Mechanisms. Some models have fixed "minus-stamp" layers, others amplify whichever sign was already chosen, and Qwen adds a later layer that counteracts the stamp.
Causal test. Subtracting one sign direction flips 13 of 14 held-out integration-by-parts errors on Llama-3.3-70B while leaving correct-plus answers intact (0/72). Flips replicate on Qwen, Gemma and Mistral-Small at 24-70% per cell.
Non-steerable case. Phi-4 has an equally localized circuit, but the same intervention flips almost nothing, because the sign is written across about 15 layers with a median margin of 17.8 logits.
Rigor. Core claims were preregistered with pass criteria fixed in git. 23 of 29 checks passed, and all misses are reported.
Abstract
When large language models do multi-step mathematics, a characteristic failure recurs: the model writes one wrong sign token mid-computation -- a cofactor sign, the minus of a recursive integration-by-parts step -- then executes the rest of the arithmetic correctly, and the answer comes out wrong. We ask where inside the network that sign is written. Across six open models spanning 14B to 675B parameters, a per-layer readout shows the wrong sign is committed late -- by MLP layers in the last quarter of the network (70-100% per model and task), a pattern present, where tested (Qwen), in the base model before instruction tuning. The mechanism varies: some models carry fixed "minus-stamp" layers that push a minus regardless of the right answer, others carry amplifiers of whichever sign was already chosen, and Qwen adds a downstream rescue layer that opposes the stamp. The circuit is causal: subtracting a single sign direction from the residual stream flips 13 of 14 held-out minus-written integration-by-parts errors on Llama-3.3-70B, with flips replicating on Qwen, Gemma, and Mistral-Small (24-70% per cell), and with representation-level specificity -- the same dose breaks correctly-minus answers while sparing correct-plus answers where the direction is surgical (0/72 on Llama), and matched random-direction and control-layer interventions are null. Core claims were preregistered with pass criteria frozen in git before held-out data was seen: 23 of 29 checks passed, all misses reported. But causality is not uniform: Phi-4 implements an equally well-localized circuit at the same late depth, yet the identical intervention flips almost nothing -- the sign is written redundantly across ~15 layers with margins several times larger (median 17.8 logits). Localization, we conclude, does not imply steerability: two models can share a circuit location yet differ enormously in how overturnable it is.