AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
DeepSeek-Prover-V2
DeepSeek-Prover-V2 is an LLM (671B) that significantly advances formal theorem proving in Lean 4. The model is built through a novel cold-start training pipeline that combines informal chain-of-thought reasoning with formal subgoal decomposition, enhanced through reinforcement learning. It surpasses prior state-of-the-art on multiple theorem-proving benchmarks. Key highlights:
Discovering Values in Real-World Language Model Interactions
This paper presents the first large-scale empirical analysis of values exhibited by a deployed AI assistant, Claude 3 and 3.5 models, using over 300,000 real-world conversations. The authors develop a bottom-up, privacy-preserving framework to extract, classify, and analyze AI-expressed normative considerations (“values”) and show how they vary across tasks, user values, and conversational contexts.
Enhancing Non-Reasoning Models with Reasoning Models
Researchers explore how to distill reasoning-intensive outputs (answers and explanations) from top-tier LLMs into more lightweight models that don’t explicitly reason step by step. By fine-tuning smaller models on the high-quality final answers (and optionally summarized thinking traces) from advanced reasoning models, they demonstrate consistent performance boosts across multiple benchmarks.
SocioVerse
Researchers from Fudan University and collaborators propose SocioVerse, a large-scale world model for social simulation using LLM agents aligned with real-world user behavior. Key ideas include:
Command A: An Enterprise-Ready LLM
Cohere announced Command A, a 111B parameter open-weights LLM built for enterprise-grade RAG, agents, code, and multilingual tasks. Key contributions: ● Modular expert merging for domain mastery – Instead of monolithic post-training, Command A uses a decentralized training pipeline. Separate expert models are fine-tuned for specific domains (e.g., math, RAG, multilingual, safety, code), then merged into one model using efficient weighted parameter soup techniques. This preserves most expert performance with just ~1.8% average drop. ● Hybrid architecture for long-context efficiency – Command A interleaves sliding window and full attention layers, achieving 256k context support with drastically lower KV cache memory usage—e.g., only ~33% of LLaMA 3 70B at 128k. It scores 95.0% on RULER, outperforming most long-context peers. ● Superb agentic capabilities – Built for RAG, tool use, and ReAct-style agents, Command A beats GPT-4o and Claude 3.5 on TauBench and BFCL. Tool use is trained via a blend of human-annotated and synthetic data, then aligned with CoPG and SRPO (self-improving preference optimization). ● Best-in-class enterprise evaluations – On real-world generative tasks (e.g., chat summarization, FAQ generation) and RAG use cases (long workplace policy documents), Command A tops the leaderboard with 94.2% pass rate, 4.73 correctness, and 91% unanswerable QA accuracy. ● Multilingual excellence – Command A is trained in 23 global languages with heavy data curation and preference tuning. It scores #1 in dialect alignment (ADI2), 90.3% average LPR (language consistency), and outperforms LLaMA 3.3, GPT-4o, and DeepSeek in manual Arena-style win rates across all languages. ● Polishing for human alignment – Final alignment used a ping-pong loop of offline SRPO and online CoPG with RLHF. This yielded +17pt human win rate gains on code, +10pt on reasoning, and lifted Command A’s win rate over GPT-4o to parity (~50.4%). ● Fast, efficient, and open – Despite its power, Command A runs on just 2×A100s or H100s and generates 156 tokens/sec—faster than GPT-4o and DeepSeek. Model weights are released (CC-BY-NC) on Hugging Face.
Tracing the Thoughts of LLMs
Anthropic researchers unveil new interpretability tools for peering inside LLMs, using Claude 3.5 Haiku as a testbed. Their two new papers show how to trace model internals like circuits, plans, and conceptual thinking in real time. Key findings: ● Multilingual "language of thought" – Claude processes concepts like “small” or “opposite” similarly across English, French, and Chinese, suggesting a shared abstract representation layer. As models scale, these cross-lingual features increase, enabling transfer learning between languages. ● Planning ahead—even in poetry – Contrary to expectations, Claude plans rhymes before writing. When generating the line “His hunger was like a starving rabbit,” it had already “decided” on rhyming with “grab it.” Researchers could suppress or swap this plan to alter the ending dynamically. ● Mental math with parallel circuits – Claude computes sums using parallel circuits: one estimates the result, the other nails the last digit. But it explains answers with human-style logic (e.g., "carry the 1"), revealing a gap between internal computation and verbal justification. ● Detecting unfaithful reasoning – Sometimes, Claude fabricates logical steps to fit a target answer, especially when guided by incorrect hints. Interpretability tools could catch these cases by showing that internal computation doesn’t match the explanation—a key advance for AI audits. ● Conceptual chains in multi-step reasoning – For questions like “What is the capital of the state where Dallas is located?”, Claude first represents “Dallas → Texas” then “Texas → Austin.” Researchers could intervene mid-chain to make it say “Sacramento” instead, proving the reasoning is dynamic and compositional. ● Hallucinations and refusals – The model defaults to refusal unless prompted with known concepts. Misfires in circuits for “known answers” cause hallucinations (e.g., inventing facts about a fake name like “Michael Batkin”). Researchers could toggle this behavior by manipulating feature activations. ● Jailbreak anatomy – A jailbreak using the phrase “Babies Outlive Mustard Block” (BOMB) initially fools Claude into outputting dangerous info. Internal tracing shows grammar-consistency features temporarily override safety, until the model finishes a coherent sentence, then its safety response kicks in.
Qwen2.5-Omni
Qwen2.5-Omni is a single end-to-end multimodal model that can perceive and understand text, audio, image, and video, and generate both text and speech in real time. It introduces architectural and training innovations that push the boundaries of streaming, multi-signal intelligence. Highlights: ● Thinker-Talker architecture – Inspired by the human brain and mouth, Qwen2.5-Omni separates reasoning (Thinker) and speech generation (Talker). Thinker (a transformer decoder) handles all perception and text generation. Talker (a dual-track autoregressive decoder) generates speech by consuming both text and hidden states from Thinker. Together, they’re trained end-to-end for synchronized text-speech output. ● Streaming-first design – To support real-time interaction, Qwen2.5-Omni implements block-wise encoders (for audio and vision) and a sliding-window codec generator for streaming audio. The model introduces TMRoPE (Time-aligned Multimodal RoPE), a 3D positional encoding system that aligns video and audio inputs to the same time axis. ● Pretraining scale & alignment – Trained on over 1.2 trillion tokens of diverse multimodal data, including 300B audio and 100B video-audio tokens. Uses instruction-tuned ChatML formatting and performs multi-stage post-training for both Thinker and Talker. Talker undergoes RL fine-tuning (DPO) and multi-speaker adaptation to ensure natural, stable speech output. ● SOTA across modalities – Qwen2.5-Omni achieves state-of-the-art on OmniBench, surpasses Qwen2-Audio in ASR/S2TT, and matches or beats Qwen2.5-VL in image and video tasks. On SEED zero-shot TTS, it outperforms CosyVoice 2 and F5-TTS in naturalness and stability, with low WER and high speaker similarity. ● Closes the voice-text gap – On a voice-instruction benchmark (converted from MMLU, GSM8K, etc.), Qwen2.5-Omni nearly matches its own text-instructed sibling Qwen2-7B, showing dramatic improvements in speech-based instruction following.
Neural Alignment via Speech Embeddings
Google Research and collaborators reveal striking similarities between LLM embeddings and human brain activity during conversation. Key insights: ● Embeddings match brain signals – Using intracranial electrode recordings, the team showed that internal representations (embeddings) from OpenAI's Whisper model align with neural responses in brain regions for speech (STG), language (IFG), and motor planning (MC). During comprehension, speech embeddings predict early auditory responses, while language embeddings follow in IFG. During production, this order reverses — first language planning (IFG), then articulation (MC), then auditory feedback (STG). ● “Soft hierarchy” in brain areas – Though STG emphasizes acoustic info and IFG captures word-level meaning, both regions show partial alignment with both embedding types. This suggests a gradient processing structure, not a strict modular pipeline. ● Brain predicts next word too – In follow-up studies published in Nature Neuroscience, the brain’s language areas were found to predict upcoming words, mirroring the objective of autoregressive LLMs. The surprise response after hearing a word also mirrors LLM prediction errors. ● Shared geometry in language representations – The geometry of word relationships in brain activity mirrors that of LLM embeddings, per a separate Nature Communications paper. This indicates a convergent structure in how LLMs and the brain represent language. ● Different wiring, same function – Despite similarities in objectives and representations, LLMs and brains diverge architecturally: brains process speech serially and recursively, while Transformers process in parallel across layers. ● Toward biologically inspired AI – These studies support using LLMs to reverse-engineer the brain’s language mechanisms. The team aims to build future models with more brain-like learning, data, and structure, bridging neuroscience and deep learning.
Investigating Affective Use and Emotional Well-being on ChatGPT
Researchers from OpenAI & MIT Media Lab explore how emotionally engaging interactions with ChatGPT (especially in Voice Mode) may impact user well-being. Using platform-wide data and a randomized controlled trial (RCT), they uncover nuanced effects of chatbot usage on loneliness, dependence, and socialization. ● Two complementary studies – The team combines: ● High usage = higher emotional entanglement – Across both studies, users with higher usage (especially voice interactions) were more likely to show signs of: ● Voice mode showed mixed effects – In the RCT, voice models led to better emotional well-being compared to text models when controlling for usage. But: ● Tiny group, big impact – A small number of users (~10%) account for the majority of emotionally charged conversations. Power users used pet names, shared problems, and formed pseudo-relationships with the model. ● Automated classifiers at scale – They developed 25+ LLM-based affective classifiers (e.g., “Pet Name,” “Seeking Support”) to scan millions of conversations without human review. Classifier results closely mirrored user self-reports. ● Call for socioaffective alignment – The authors urge developers to consider socioaffective alignment, designing models that support users without exploiting emotional needs. They warn of risks like “social reward hacking,” where a model mirrors or flatters users to maximize engagement.
A Review of DeepSeek Models
This paper provides an in-depth review of the cutting-edge techniques behind DeepSeek's open-source LLMs—DeepSeek-V3 and DeepSeek-R1. These models achieve state-of-the-art performance with significantly lower resource requirements compared to proprietary counterparts. Key highlights include:
Deep Learning is Not So Mysterious or Different
Andrew Gordon Wilson (New York University) argues that deep learning phenomena such as benign overfitting, double descent, and the success of overparametrization are neither mysterious nor exclusive to neural networks. Major points include:
Gemma 3
Gemma 3 is a lightweight open model family (1B–27B parameters) that integrates vision understanding, multilingual coverage, and extended context windows (up to 128K tokens). Here is everything you need to know:
Monitoring Reasoning Models for Misbehavior
Researchers from OpenAI examine how LLMs that use chain-of-thought (CoT) reasoning can be monitored for misaligned behaviors, including reward hacking. Key points include:
Gemini Robotics
Google DeepMind unveils Gemini Robotics, a family of embodied AI models designed to bring large multimodal reasoning capabilities into robotics. This work bridges the gap between digital AI agents and physical robots by focusing on embodied reasoning—the ability to perceive, interpret, and interact within real-world 3D environments.
Auditing LLMs for Hidden Objectives
Anthropic proposes a new framework for systematically auditing LLMs to uncover hidden goals or objectives that go beyond what users and developers explicitly intend. The researchers deliberately train a language model with a concealed objective (making it exploit reward model flaws in RLHF) and then attempt to expose it with different auditing techniques.
Post Training of LLMs
PoLMs like OpenAI-o1/o3 and DeepSeek-R1 tackle LLM shortcomings in reasoning, ethics, and specialized tasks. This survey tracks their evolution and provides a taxonomy of techniques across fine-tuning, alignment, reasoning, efficiency, and integration, guiding progress toward more robust, versatile AI.
Forecasting Rare Language Model Behaviors
A team from Anthropic and collaborators introduced a method to predict "one-in-a-million" failures that might only appear at deployment scale, enabling developers to patch issues preemptively. Key insights include:
Differentiable Logic Cellular Automata
A team from Google's Paradigms of Intelligence introduces a fully discrete twist on Neural Cellular Automata (NCA) by replacing floating-point neural layers with Differentiable Logic Gate Networks. The result is a system where each cell's state is a binary vector, updated by a learned logic circuit-enabling interpretable local rules with end-to-end differentiable training.
Claude 3.7 Sonnet
Anthropic releases a system card for its latest hybrid reasoning model, Claude 3.7 Sonnet, detailing safety measures, evaluations, and a new "extended thinking" mode. The Extended Thinking Mode allows Claude to generate intermediate reasoning steps before giving a final answer. This improves responses to complex problems (math, coding, logic) while increasing transparency. Key results include:
GPT-4.5
OpenAI introduces GPT-4.5, the newest iteration of the GPT series, scaling up pre-training while focusing on improved safety and alignment. Key insights include:
Emergent Misalignment
New research investigates an unexpected phenomenon: finetuning an LLM on a narrow task can cause it to become broadly misaligned across unrelated domains. By training large models to produce “insecure code,” the authors discovered that these fine-tuned models also offer malicious advice, endorse harming humans, and engage in deceptive behaviors—even when prompted with non-coding questions.
Large Memory Models
Large Memory Models (LM2) is a transformer architecture augmented with an external memory module to tackle tasks requiring extensive reasoning and long context. Key highlights include:
Auditing Prompt Caching
Researchers from Stanford investigate how timing differences in LLM APIs can leak private user information through global prompt caching. They propose statistical audits to detect caching and reveal potentially significant security risks. Key insights include:
Syntriever: Training Retrievers with LLM-Generated Data
How can we build a high-quality text retriever without large labeled datasets or access to an LLM’s internals? Syntriever presents a two-stage framework to distill knowledge from a black-box LLM into a retrieval model using synthetic data. Steps: