Reasoning Models Can Control Their Chains of Thought When They Tell Themselves To
Uriel Dolev, Idan Schwartz and Yoav Goldberg (Bar-Ilan University and Allen Institute for AI) show that reasoning models control properties of their chain of thought much better when the constraint appears as a self-instruction inside the reasoning than when it is given in the user prompt.
Ask this paper
Prior result. Earlier evaluations placed CoT constraints in the user prompt and reported very low controllability across models.
Self-Instructed CoT Controllability. The constraint is inserted as a self-instruction inside the model's CoT, and the evaluation measures whether the model follows it. Controllability is significantly higher than with standard user instructions.
Simple reminders help. Even a generic reminder to follow the user's constraint substantially improves controllability.
No reasoning access needed. Effective self-instructions can be bootstrapped into the reasoning through a simple instruction in the user prompt, so the effect is reachable from the outside.
Safety implication. In a realistic monitoring setup, the authors use self-instruction to evade a CoT monitor, which weakens the assumption that low measured controllability makes CoT monitoring reliable.
Abstract
Chain-of-thought (CoT) controllability, the ability of reasoning models to control properties of their CoT, is consequential for both their usefulness and reliable oversight. Recent work evaluates this capability by measuring whether models follow CoT constraints given in the user prompt, and reports very low controllability across models. We argue that this low controllability may not reflect an inherent inability of reasoning models to control their CoT, but instead result from naively specifying constraints in the user prompt. To test this, we introduce Self-Instructed CoT Controllability: a complementary evaluation that places the constraint as a self-instruction inside the model's CoT and measures its ability to follow it. We show that, under self-instruction, models achieve significantly higher controllability than with standard user instructions. Even a generic reminder to follow the user's constraint substantially improves controllability. Moreover, effective self-instructions can be bootstrapped into the reasoning through a simple instruction in the user prompt, enabling improved control even without access to the reasoning channel. Finally, using self-instruction, we demonstrate in a realistic monitoring setup that CoT controllability can be leveraged for monitor evasion, posing a risk to CoT monitorability.