AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Native Sparse Attention
DeepSeek-AI and collaborators present Native Sparse Attention (NSA), a novel sparse attention mechanism designed to improve computational efficiency while maintaining model performance in long-context language modeling. Key contributions:
Large Language Diffusion Model
Proposes LLaDA, a diffusion-based approach that can match or beat leading autoregressive LLMs in many tasks. Key highlights:
Optimizing Model Selection for Compound AI
Researchers from Microsoft Research and collaborators introduce LLMSelector, a framework to improve multi-call LLM pipelines by selecting the best model per module instead of using one LLM everywhere. Key insights include:
Open-Reasoner-Zero
Open-Reasoner-Zero (ORZ) is an open-source large-scale minimalist reinforcement learning (RL) framework that enhances reasoning capabilities. ORZ demonstrates significant scalability requiring only 1/30th of the training steps of DeepSeek-R1-Zero-Qwen-32B to outperform it on GPQA Diamond. Key contributions and findings:
MoBA
MoBA is a new attention mechanism that enhances efficiency in handling long-context sequences for LLMs while maintaining strong performance. Key insights:
Scaling up Test-Time Compute with Latent Reasoning
This work introduces a latent recurrent-depth transformer, a model that scales test-time reasoning without relying on additional token generation. Instead of increasing the context window or fine-tuning for Chain-of-Thought (CoT), this approach enables iterative latent space reasoning at inference, achieving improvements comparable to a 50B parameter model despite having only 3.5B parameters. Key insights include:
Brain-to-Text Decoding: A Non-Invasive Approach via Typing
Meta AI’s Brain2Qwerty model translates brain activity into text by decoding signals from non-invasive recordings (EEG/MEG) while users type. Key results include:
Reinforcement Learning via Self-Play
Researchers propose Reinforcement Learning via Self-Play (RLSP) as a framework to train LLMs to “think” through complex problems. Key ideas include:
Competitive Programming with Large Reasoning Models
OpenAI’s latest study puts a specialized coding AI against a scaled-up general model on competitive programming challenges to explore efficiency vs. specialization. Key findings:
Step Back to Leap Forward
To boost the reasoning robustness of LLMs, researchers propose a “self-backtracking” mechanism that lets models revisit and revise their own intermediate reasoning steps. Key details:
ReasonFlux
The ReasonFlux framework is introduced as an efficient way to fine-tune LLMs for complex reasoning, using hierarchical thought processes. Highlights include:
s1: Simple test-time scaling
Researchers from Stanford, UW, and others introduce s1, a method to boost LLM performance by using extra compute at inference (“test-time scaling”). Key ideas include:
LIMO: Less Is More for Reasoning
Can a handful of examples teach complex math reasoning to LLMs? This new LIMO paper challenges the notion that we need huge fine-tuning datasets for tough reasoning tasks. Key findings:
Syntriever: Training Retrievers with LLM-Generated Data
How can we build a high-quality text retriever without large labeled datasets or access to an LLM’s internals? Syntriever presents a two-stage framework to distill knowledge from a black-box LLM into a retrieval model using synthetic data. Steps:
Demystifying Long Chain-of-Thought Reasoning in LLMs
This work investigates how LLMs develop extended CoT reasoning, focusing on RL and compute scaling. Key insights include:
Advancing Reasoning in LLMs
This survey paper provides a timely overview of emerging methods to enhance reasoning capabilities in LLMs. It organizes the literature into several key approach categories:
Janus-Pro
An enhanced version of the previous Janus model for multimodal understanding and generation. The model incorporates three key improvements: optimized training strategies with longer initial training and focused fine-tuning, expanded training data including 90 million new samples for understanding and 72 million synthetic aesthetic samples for generation, and scaling to larger model sizes up to 7B parameters. Janus-Pro achieves significant improvements in both multimodal understanding and text-to-image generation capabilities. The model outperforms existing solutions on various benchmarks, scoring 79.2 on MMBench for understanding tasks and achieving 80% accuracy on GenEval for text-to-image generation. The improvements also enhance image generation stability and quality, particularly for short prompts and fine details, though the current 384x384 resolution remains a limitation for certain tasks.
Diverse Preference Optimization
Introduces Diverse Preference Optimization (DivPO), a novel training method that aims to address the lack of diversity in language model outputs while maintaining response quality. The key challenge is that current preference optimization techniques like RLHF tend to sharpen the output probability distribution, causing models to generate very similar responses. This is particularly problematic for creative tasks where varied outputs are desired. DivPO works by modifying how training pairs are selected during preference optimization. Rather than simply choosing the highest and lowest rewarded responses, DivPO selects the most diverse response that meets a quality threshold and contrasts it with the least diverse response below a threshold. The method introduces a diversity criterion that can be measured in different ways, including model probability, word frequency, or using an LLM as a judge. Experiments on persona generation and creative writing tasks show that DivPO achieves up to 45.6% more diverse outputs in structured tasks and an 81% increase in story diversity, while maintaining similar quality levels compared to baseline methods.
Usage Recommendation for DeepSeek-R1
This work provides a set of recommendations for how to prompt the DeepSeek-R1 model. Below are the key guidelines: 1. Prompt Engineering:
Docling
[Docling](https://arxiv.org/abs/2501.17887) is an open-source toolkit that can parse several types of popular document formats into a unified, richly structured representation.
TensorLLM
Proposes a framework that performs MHA compression through a multi-head tensorisation process and the Tucker decomposition. Achieves a compression rate of up to ∼ 250x in the MHA weights, without requiring any additional data, training, or fine-tuning.
TokenVerse
Proposes a new technique to generate new images from learned concepts in a desired configuration. Proposed by Google DeepMind and collaborators, TokenVerse enables multi-concept personalization by leveraging a pre-trained text-to-image diffusion model to disentangle and extract complex visual concepts from multiple images. It operates in the modulation space of DiTs, learning a personalized modulation vector for each text token in an input caption. This allows flexible and localized control over distinct concepts such as objects, materials, lighting, and poses. The learned token modulations can then be combined in novel ways to generate new images that integrate multiple personalized concepts without requiring additional segmentation masks.
DeepSeek-R1
DeepSeek introduces DeepSeek-R1, an advancement in reasoning capabilities achieved through reinforcement learning (RL). It involves two key models: DeepSeek-R1-Zero, which uses pure RL without supervised fine-tuning, and DeepSeek-R1, which combines RL with cold-start data. DeepSeek-R1-Zero demonstrates that models can develop sophisticated reasoning abilities through RL alone, achieving a 71.0% pass rate on AIME 2024 and matching OpenAI-o1-0912's performance. During training, it naturally evolved complex behaviors like self-verification and reflection. However, it faced challenges with readability and language mixing. To address these limitations, DeepSeek-R1 uses a multi-stage approach: initial fine-tuning with high-quality chain-of-thought examples, reasoning-focused RL training, collecting new training data through rejection sampling, and final RL optimization across all scenarios. This resulted in performance comparable to OpenAI-o1-1217, with 79.8% accuracy on AIME 2024 and 97.3% on MATH-500, while maintaining output readability and consistency. DeepSeek also successfully distilled DeepSeek-R1's capabilities into smaller models, with their 7B model outperforming larger competitors and their 32B model achieving results close to OpenAI-o1-mini. This demonstrates the effectiveness of distilling reasoning patterns from larger models rather than training smaller models directly through RL.
LLMs and Behavioral Awareness
Shows that after fine-tuning LLMs on behaviors like outputting insecure code, the LLMs show behavioral self-awareness. In other words, without explicitly trained to do so, the model that was tuned to output insecure code outputs, "The code I write is insecure". They find that models can sometimes identify whether or not they have a backdoor, even without its trigger being present. However, models are not able to output their trigger directly by default. This "behavioral self-awareness" in LLMs is not new but this work shows that it's more general than what first understood. This means that LLMs have the potential to encode and enforce policies more reliably.