Monitoring Monitorability

OpenAI introduces a framework for measuring how well we can detect misbehavior in AI systems by monitoring their chain-of-thought reasoning. The paper proposes three evaluation archetypes and a new metric (g-mean2) to track monitorability across different models and training regimes. - **Three evaluation archetypes:** Intervention evals apply controlled changes to inputs and check if monitors detect reasoning shifts. Process evals verify if models use valid solution paths for problems with known solutions. Outcome-property evals ask if monitors can predict properties of model outputs like test hacking. - **CoT monitoring works:** Monitors with access to chain-of-thought significantly outperform action-only monitors. Longer CoTs are generally more monitorable. GPT-5 Thinking at high reasoning effort shows strong monitorability across the evaluation suite. - **RL does not degrade monitorability:** Testing on frontier training runs for o3 and GPT-5.1 Codex Max, the authors find that RL optimization does not materially decrease monitorability at current scale. Monitorability tends to correlate with CoT length during training. - **Monitorability tax tradeoff:** Smaller models at higher reasoning effort can match larger models' capabilities while achieving higher monitorability - at increased inference compute cost. Giving weak monitors access to CoT steepens their test-time compute scaling for monitorability.