AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Chain-of-Draft
To address the issue of latency in reasoning LLMs, this work introduces Chain-of-Draft (CoD). Here is a quick summary of the key highlights:
Emergent Misalignment
New research investigates an unexpected phenomenon: finetuning an LLM on a narrow task can cause it to become broadly misaligned across unrelated domains. By training large models to produce “insecure code,” the authors discovered that these fine-tuned models also offer malicious advice, endorse harming humans, and engage in deceptive behaviors—even when prompted with non-coding questions.
An Efficient Alternative to Self-Attention
This paper presents FFTNet, a framework that replaces costly self-attention with an adaptive spectral filtering technique based on the Fast Fourier Transform (FFT). Key components:
PlanGEN
PlanGEN is a multi-agent framework designed to enhance planning and reasoning in LLMs through constraint-guided iterative verification and adaptive algorithm selection. Key insights include:
A Multi-Agent Framework for Chart Generation
METAL is a vision-language model (VLM)-based multi-agent framework designed to significantly enhance automatic chart-to-code generation by decomposing the task into specialized iterative steps. Key highlights include:
LightThinker
This new paper proposes a novel approach to dynamically compress reasoning steps in LLMs, significantly improving efficiency without sacrificing accuracy. Key insights include:
A Systematic Survey of Prompt Optimization
This paper offers a comprehensive survey of Automatic Prompt Optimization (APO)—defining its scope, presenting a unifying 5-part framework, categorizing existing methods, and highlighting key progress and challenges in automating prompt engineering for LLMs.
Protein LLMs
A comprehensive overview of Protein LLMs, including architectures, training datasets, evaluation metrics, and applications.
AI Co-Scientist
Google introduces AI co-scientist, a multi-agent AI system built with Gemini 2.0 to help accelerate scientific breakthroughs. Key highlights:
The AI CUDA Engineer
Sakana AI introduces The AI CUDA Engineer, an end-to-end agentic system that can produce highly optimized CUDA kernels. Key contributions:
Native Sparse Attention
DeepSeek-AI and collaborators present Native Sparse Attention (NSA), a novel sparse attention mechanism designed to improve computational efficiency while maintaining model performance in long-context language modeling. Key contributions:
Large Language Diffusion Model
Proposes LLaDA, a diffusion-based approach that can match or beat leading autoregressive LLMs in many tasks. Key highlights:
SWE-Lancer
Researchers from OpenAI introduce SWE-Lancer, a benchmark evaluating LLMs on 1,488 real-world freelance software engineering tasks from Upwork, collectively worth $1M in payouts. Key takeaways:
Optimizing Model Selection for Compound AI
Researchers from Microsoft Research and collaborators introduce LLMSelector, a framework to improve multi-call LLM pipelines by selecting the best model per module instead of using one LLM everywhere. Key insights include:
Open-Reasoner-Zero
Open-Reasoner-Zero (ORZ) is an open-source large-scale minimalist reinforcement learning (RL) framework that enhances reasoning capabilities. ORZ demonstrates significant scalability requiring only 1/30th of the training steps of DeepSeek-R1-Zero-Qwen-32B to outperform it on GPQA Diamond. Key contributions and findings:
MoBA
MoBA is a new attention mechanism that enhances efficiency in handling long-context sequences for LLMs while maintaining strong performance. Key insights:
The Danger of Overthinking
This paper investigates overthinking in Large Reasoning Models (LRMs)—a phenomenon where models prioritize extended internal reasoning over interacting with their environment. Their study analyzes 4,018 software engineering task trajectories to understand how reasoning models handle decision-making in agentic settings. Key findings:
Inner Thinking Transformers
Inner Thinking Transformer (ITT) is a new method that enhances reasoning efficiency in small-scale LLMs via dynamic depth scaling. ITT aims to mitigate parameter bottlenecks in LLMs, providing scalable reasoning efficiency without expanding model size. Key contributions:
Scaling up Test-Time Compute with Latent Reasoning
This work introduces a latent recurrent-depth transformer, a model that scales test-time reasoning without relying on additional token generation. Instead of increasing the context window or fine-tuning for Chain-of-Thought (CoT), this approach enables iterative latent space reasoning at inference, achieving improvements comparable to a 50B parameter model despite having only 3.5B parameters. Key insights include:
Brain-to-Text Decoding: A Non-Invasive Approach via Typing
Meta AI’s Brain2Qwerty model translates brain activity into text by decoding signals from non-invasive recordings (EEG/MEG) while users type. Key results include:
Reinforcement Learning via Self-Play
Researchers propose Reinforcement Learning via Self-Play (RLSP) as a framework to train LLMs to “think” through complex problems. Key ideas include:
Competitive Programming with Large Reasoning Models
OpenAI’s latest study puts a specialized coding AI against a scaled-up general model on competitive programming challenges to explore efficiency vs. specialization. Key findings:
Training Language Models to Reason Efficiently
A new RL approach teaches large reasoning models to allocate their reasoning effort efficiently, reducing wasted computation on easy problems. Key points include:
Large Memory Models
Large Memory Models (LM2) is a transformer architecture augmented with an external memory module to tackle tasks requiring extensive reasoning and long context. Key highlights include: