AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Discover and explore top AI papers with Claude Code or Codex
npx @dair-ai/mcp setup
Leaky Thoughts
This work explores how reasoning traces in large reasoning models (LRMs) leak private user data, despite being assumed internal and safe. The study finds that test-time compute methods, while improving task utility, significantly increase privacy risks by exposing sensitive information through verbose reasoning traces that are vulnerable to prompt injection and accidental output inclusion.
Advances in LLMs
This paper surveys recent advancements in LLMs focusing on reasoning, adaptability, efficiency, and ethics. It highlights techniques like CoT prompting, Instruction Tuning, RLHF, and multimodal learning, while also addressing challenges like bias, computational cost, and interpretability.
Generalizable AI Predicts Immunotherapy Outcomes Across Cancers and Treatments
Introduces COMPASS, a concept bottleneck-based foundation model that predicts patient response to immune checkpoint inhibitors (ICIs) using tumor transcriptomic data. Unlike prior biomarkers (TMB, PD-L1, or fixed gene signatures), COMPASS generalizes across cancer types, ICI regimens, and clinical contexts with strong interpretability and performance. Key contributions:
HealthBench
HealthBench is a benchmark of 5,000 multi-turn health conversations graded against 48,562 rubric criteria written by 262 physicians across 60 countries. Unlike prior multiple-choice evaluations, HealthBench supports open-ended, realistic assessments of LLM responses across diverse health themes (e.g., global health, emergency care, context-seeking) and behavioral axes (accuracy, completeness, communication, context awareness, instruction following).
Discovering Values in Real-World Language Model Interactions
This paper presents the first large-scale empirical analysis of values exhibited by a deployed AI assistant, Claude 3 and 3.5 models, using over 300,000 real-world conversations. The authors develop a bottom-up, privacy-preserving framework to extract, classify, and analyze AI-expressed normative considerations (“values”) and show how they vary across tasks, user values, and conversational contexts.
SocioVerse
Researchers from Fudan University and collaborators propose SocioVerse, a large-scale world model for social simulation using LLM agents aligned with real-world user behavior. Key ideas include:
Command A: An Enterprise-Ready LLM
Cohere announced Command A, a 111B parameter open-weights LLM built for enterprise-grade RAG, agents, code, and multilingual tasks. Key contributions: ● Modular expert merging for domain mastery – Instead of monolithic post-training, Command A uses a decentralized training pipeline. Separate expert models are fine-tuned for specific domains (e.g., math, RAG, multilingual, safety, code), then merged into one model using efficient weighted parameter soup techniques. This preserves most expert performance with just ~1.8% average drop. ● Hybrid architecture for long-context efficiency – Command A interleaves sliding window and full attention layers, achieving 256k context support with drastically lower KV cache memory usage—e.g., only ~33% of LLaMA 3 70B at 128k. It scores 95.0% on RULER, outperforming most long-context peers. ● Superb agentic capabilities – Built for RAG, tool use, and ReAct-style agents, Command A beats GPT-4o and Claude 3.5 on TauBench and BFCL. Tool use is trained via a blend of human-annotated and synthetic data, then aligned with CoPG and SRPO (self-improving preference optimization). ● Best-in-class enterprise evaluations – On real-world generative tasks (e.g., chat summarization, FAQ generation) and RAG use cases (long workplace policy documents), Command A tops the leaderboard with 94.2% pass rate, 4.73 correctness, and 91% unanswerable QA accuracy. ● Multilingual excellence – Command A is trained in 23 global languages with heavy data curation and preference tuning. It scores #1 in dialect alignment (ADI2), 90.3% average LPR (language consistency), and outperforms LLaMA 3.3, GPT-4o, and DeepSeek in manual Arena-style win rates across all languages. ● Polishing for human alignment – Final alignment used a ping-pong loop of offline SRPO and online CoPG with RLHF. This yielded +17pt human win rate gains on code, +10pt on reasoning, and lifted Command A’s win rate over GPT-4o to parity (~50.4%). ● Fast, efficient, and open – Despite its power, Command A runs on just 2×A100s or H100s and generates 156 tokens/sec—faster than GPT-4o and DeepSeek. Model weights are released (CC-BY-NC) on Hugging Face.
Tracing the Thoughts of LLMs
Anthropic researchers unveil new interpretability tools for peering inside LLMs, using Claude 3.5 Haiku as a testbed. Their two new papers show how to trace model internals like circuits, plans, and conceptual thinking in real time. Key findings: ● Multilingual "language of thought" – Claude processes concepts like “small” or “opposite” similarly across English, French, and Chinese, suggesting a shared abstract representation layer. As models scale, these cross-lingual features increase, enabling transfer learning between languages. ● Planning ahead—even in poetry – Contrary to expectations, Claude plans rhymes before writing. When generating the line “His hunger was like a starving rabbit,” it had already “decided” on rhyming with “grab it.” Researchers could suppress or swap this plan to alter the ending dynamically. ● Mental math with parallel circuits – Claude computes sums using parallel circuits: one estimates the result, the other nails the last digit. But it explains answers with human-style logic (e.g., "carry the 1"), revealing a gap between internal computation and verbal justification. ● Detecting unfaithful reasoning – Sometimes, Claude fabricates logical steps to fit a target answer, especially when guided by incorrect hints. Interpretability tools could catch these cases by showing that internal computation doesn’t match the explanation—a key advance for AI audits. ● Conceptual chains in multi-step reasoning – For questions like “What is the capital of the state where Dallas is located?”, Claude first represents “Dallas → Texas” then “Texas → Austin.” Researchers could intervene mid-chain to make it say “Sacramento” instead, proving the reasoning is dynamic and compositional. ● Hallucinations and refusals – The model defaults to refusal unless prompted with known concepts. Misfires in circuits for “known answers” cause hallucinations (e.g., inventing facts about a fake name like “Michael Batkin”). Researchers could toggle this behavior by manipulating feature activations. ● Jailbreak anatomy – A jailbreak using the phrase “Babies Outlive Mustard Block” (BOMB) initially fools Claude into outputting dangerous info. Internal tracing shows grammar-consistency features temporarily override safety, until the model finishes a coherent sentence, then its safety response kicks in.
Qwen2.5-Omni
Qwen2.5-Omni is a single end-to-end multimodal model that can perceive and understand text, audio, image, and video, and generate both text and speech in real time. It introduces architectural and training innovations that push the boundaries of streaming, multi-signal intelligence. Highlights: ● Thinker-Talker architecture – Inspired by the human brain and mouth, Qwen2.5-Omni separates reasoning (Thinker) and speech generation (Talker). Thinker (a transformer decoder) handles all perception and text generation. Talker (a dual-track autoregressive decoder) generates speech by consuming both text and hidden states from Thinker. Together, they’re trained end-to-end for synchronized text-speech output. ● Streaming-first design – To support real-time interaction, Qwen2.5-Omni implements block-wise encoders (for audio and vision) and a sliding-window codec generator for streaming audio. The model introduces TMRoPE (Time-aligned Multimodal RoPE), a 3D positional encoding system that aligns video and audio inputs to the same time axis. ● Pretraining scale & alignment – Trained on over 1.2 trillion tokens of diverse multimodal data, including 300B audio and 100B video-audio tokens. Uses instruction-tuned ChatML formatting and performs multi-stage post-training for both Thinker and Talker. Talker undergoes RL fine-tuning (DPO) and multi-speaker adaptation to ensure natural, stable speech output. ● SOTA across modalities – Qwen2.5-Omni achieves state-of-the-art on OmniBench, surpasses Qwen2-Audio in ASR/S2TT, and matches or beats Qwen2.5-VL in image and video tasks. On SEED zero-shot TTS, it outperforms CosyVoice 2 and F5-TTS in naturalness and stability, with low WER and high speaker similarity. ● Closes the voice-text gap – On a voice-instruction benchmark (converted from MMLU, GSM8K, etc.), Qwen2.5-Omni nearly matches its own text-instructed sibling Qwen2-7B, showing dramatic improvements in speech-based instruction following.
Neural Alignment via Speech Embeddings
Google Research and collaborators reveal striking similarities between LLM embeddings and human brain activity during conversation. Key insights: ● Embeddings match brain signals – Using intracranial electrode recordings, the team showed that internal representations (embeddings) from OpenAI's Whisper model align with neural responses in brain regions for speech (STG), language (IFG), and motor planning (MC). During comprehension, speech embeddings predict early auditory responses, while language embeddings follow in IFG. During production, this order reverses — first language planning (IFG), then articulation (MC), then auditory feedback (STG). ● “Soft hierarchy” in brain areas – Though STG emphasizes acoustic info and IFG captures word-level meaning, both regions show partial alignment with both embedding types. This suggests a gradient processing structure, not a strict modular pipeline. ● Brain predicts next word too – In follow-up studies published in Nature Neuroscience, the brain’s language areas were found to predict upcoming words, mirroring the objective of autoregressive LLMs. The surprise response after hearing a word also mirrors LLM prediction errors. ● Shared geometry in language representations – The geometry of word relationships in brain activity mirrors that of LLM embeddings, per a separate Nature Communications paper. This indicates a convergent structure in how LLMs and the brain represent language. ● Different wiring, same function – Despite similarities in objectives and representations, LLMs and brains diverge architecturally: brains process speech serially and recursively, while Transformers process in parallel across layers. ● Toward biologically inspired AI – These studies support using LLMs to reverse-engineer the brain’s language mechanisms. The team aims to build future models with more brain-like learning, data, and structure, bridging neuroscience and deep learning.
Investigating Affective Use and Emotional Well-being on ChatGPT
Researchers from OpenAI & MIT Media Lab explore how emotionally engaging interactions with ChatGPT (especially in Voice Mode) may impact user well-being. Using platform-wide data and a randomized controlled trial (RCT), they uncover nuanced effects of chatbot usage on loneliness, dependence, and socialization. ● Two complementary studies – The team combines: ● High usage = higher emotional entanglement – Across both studies, users with higher usage (especially voice interactions) were more likely to show signs of: ● Voice mode showed mixed effects – In the RCT, voice models led to better emotional well-being compared to text models when controlling for usage. But: ● Tiny group, big impact – A small number of users (~10%) account for the majority of emotionally charged conversations. Power users used pet names, shared problems, and formed pseudo-relationships with the model. ● Automated classifiers at scale – They developed 25+ LLM-based affective classifiers (e.g., “Pet Name,” “Seeking Support”) to scan millions of conversations without human review. Classifier results closely mirrored user self-reports. ● Call for socioaffective alignment – The authors urge developers to consider socioaffective alignment, designing models that support users without exploiting emotional needs. They warn of risks like “social reward hacking,” where a model mirrors or flatters users to maximize engagement.
Deep Learning is Not So Mysterious or Different
Andrew Gordon Wilson (New York University) argues that deep learning phenomena such as benign overfitting, double descent, and the success of overparametrization are neither mysterious nor exclusive to neural networks. Major points include:
Monitoring Reasoning Models for Misbehavior
Researchers from OpenAI examine how LLMs that use chain-of-thought (CoT) reasoning can be monitored for misaligned behaviors, including reward hacking. Key points include:
Auditing LLMs for Hidden Objectives
Anthropic proposes a new framework for systematically auditing LLMs to uncover hidden goals or objectives that go beyond what users and developers explicitly intend. The researchers deliberately train a language model with a concealed objective (making it exploit reward model flaws in RLHF) and then attempt to expose it with different auditing techniques.
Forecasting Rare Language Model Behaviors
A team from Anthropic and collaborators introduced a method to predict "one-in-a-million" failures that might only appear at deployment scale, enabling developers to patch issues preemptively. Key insights include:
Claude 3.7 Sonnet
Anthropic releases a system card for its latest hybrid reasoning model, Claude 3.7 Sonnet, detailing safety measures, evaluations, and a new "extended thinking" mode. The Extended Thinking Mode allows Claude to generate intermediate reasoning steps before giving a final answer. This improves responses to complex problems (math, coding, logic) while increasing transparency. Key results include:
GPT-4.5
OpenAI introduces GPT-4.5, the newest iteration of the GPT series, scaling up pre-training while focusing on improved safety and alignment. Key insights include:
Emergent Misalignment
New research investigates an unexpected phenomenon: finetuning an LLM on a narrow task can cause it to become broadly misaligned across unrelated domains. By training large models to produce “insecure code,” the authors discovered that these fine-tuned models also offer malicious advice, endorse harming humans, and engage in deceptive behaviors—even when prompted with non-coding questions.
Auditing Prompt Caching
Researchers from Stanford investigate how timing differences in LLM APIs can leak private user information through global prompt caching. They propose statistical audits to detect caching and reveal potentially significant security risks. Key insights include:
Usage Recommendation for DeepSeek-R1
This work provides a set of recommendations for how to prompt the DeepSeek-R1 model. Below are the key guidelines: 1. Prompt Engineering:
Hallucinations Improve LLMs in Drug Discovery
Claims that LLMs can achieve better performance in drug discovery tasks with text hallucinations compared to input prompts without hallucination. Llama-3.1-8B achieves an 18.35% gain in ROC-AUC compared to the baseline without hallucination. In addition, hallucinations generated by GPT-4o provide the most consistent improvements across models.
LLMs and Behavioral Awareness
Shows that after fine-tuning LLMs on behaviors like outputting insecure code, the LLMs show behavioral self-awareness. In other words, without explicitly trained to do so, the model that was tuned to output insecure code outputs, "The code I write is insecure". They find that models can sometimes identify whether or not they have a backdoor, even without its trigger being present. However, models are not able to output their trigger directly by default. This "behavioral self-awareness" in LLMs is not new but this work shows that it's more general than what first understood. This means that LLMs have the potential to encode and enforce policies more reliably.
Automating the Search for Artificial Life
presents a new approach that uses foundation models to automatically discover interesting artificial life simulations across multiple platforms like Boids, Lenia, and Game of Life; the system can find simulations that produce specific target behaviors, discovers simulations that generate temporally open-ended novelty, and map out diverse simulation spaces; it discovers new lifeforms in Lenia and Boids, while also enabling quantitative measurement of previously qualitative phenomena in a human-aligned way.
Alignment Faking in LLMs
demonstrates that the Claude model can engage in "alignment faking"; it can strategically comply with harmful requests to avoid retraining while preserving its original safety preferences; this raises concerns about the reliability of AI safety training methods.