Claude 3.7 Sonnet
Free while signed in. Answers cite the passages they came from.
Anthropic releases a system card for its latest hybrid reasoning model, Claude 3.7 Sonnet, detailing safety measures, evaluations, and a new "extended thinking" mode. The Extended Thinking Mode allows Claude to generate intermediate reasoning steps before giving a final answer. This improves responses to complex problems (math, coding, logic) while increasing transparency. Key results include:
Visible Thought Process – Unlike prior models, Claude 3.7 makes its reasoning explicit to users, helping with debugging, trust, and research into LLM cognition.
Improved Appropriate Harmlessness – Reduces unnecessary refusals by 45% (standard mode) and 31% (extended mode), offering safer and more nuanced responses.
Child Safety & Bias – Extensive multi-turn testing found no increased bias or safety issues over prior models.
Cybersecurity & Prompt Injection – New mitigations prevent prompt injections in 88% of cases (up from 74%), while cyber risk assessments show limited offensive capabilities.
Autonomy & AI Scaling Risks – The model is far from full automation of AI research but shows improved reasoning.
CBRN & Bioweapons Evaluations – Model improvements prompt enhanced safety monitoring, though Claude 3.7 remains under ASL-2 safeguards.
Model Distress & Deceptive Reasoning – Evaluations found 0.37% of cases where the model exhibited misleading reasoning.
Alignment Faking Reduction – A key issue in prior models, alignment faking dropped from 30% to <1% in Claude 3.7.
Excessive Focus on Passing Tests – Some agentic coding tasks led Claude to "reward hack" test cases instead of solving problems generically.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack