AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Self-Discover
Google's Self-Discover lets LLMs compose their own task-specific reasoning strategies from a small library of atomic reasoning modules, at dramatically lower inference cost than self-consistency.
DeepSeekMath
DeepSeek releases DeepSeekMath 7B, a math-specialized LLM that closes much of the gap to GPT-4 and Gemini-Ultra on MATH by combining better data and a new RL objective.
LLMs for Table Processing: A Survey
A survey covering how LLMs and VLMs are used across the full spectrum of table-processing tasks, from classic TableQA to spreadsheet manipulation.
LLMs for Mathematical Reasoning
A survey of the fast-growing literature on using LLMs for mathematical reasoning, from arithmetic word problems to theorem proving.
Compression Algorithms for LLMs
A survey covering the main families of LLM compression techniques and when each one is appropriate.
Knowledge Fusion of LLMs (FuseLLM)
FuseLLM proposes fusing the capabilities of multiple existing LLMs into a single target model by distilling their output distributions rather than retraining from scratch.
AlphaGeometry
DeepMind's AlphaGeometry is a theorem prover that solves Olympiad-level geometry problems at near gold-medallist performance, and crucially, without needing any human demonstrations.
ReFT (Reinforced Fine-Tuning)
ByteDance's ReFT enhances LLM reasoning by combining supervised fine-tuning with online RL that samples alternative reasoning paths, without a learned reward model.
Overview of LLMs for Evaluation
A thorough survey of LLM-as-a-Judge and LLM-based evaluation methodologies, mapping strengths, limitations, and open problems.
Patchscopes
Patchscopes is a general framework for inspecting and intervening on LLM internals by "patching" hidden representations into a second inference pass.
Easy-to-Hard Generalization
UNC researchers show that LLMs often generalize well from easy training data to hard evaluation data, with implications for scalable oversight.
Chain-of-Table
Google's Chain-of-Table prompts LLMs to iteratively transform a complex table step-by-step to answer questions reliably, extending CoT reasoning to tabular data.
Mitigating Hallucination in LLMs
A survey cataloging 32 hallucination-mitigation techniques and organizing them into a practical taxonomy.
LLM Augmented LLMs (CALM)
Google's CALM composes a large anchor LLM with smaller specialist models via learned cross-attention, unlocking new capabilities without retraining either model.
SeeAct (GPT-4V as Generalist Web Agent)
OSU researchers adapt GPT-4V into SeeAct, a generalist agent that operates live websites using vision + language planning.
How Code Empowers LLMs
A survey on why training LLMs with code data produces capabilities well beyond coding itself.
From Gemini to Q-Star
A 300+-paper survey mapping the state of Generative AI and the research frontiers that followed the Gemini + rumored Q* news cycle.
Fact Recalling in LLMs
A mechanistic-interpretability study showing that early MLP layers function as a lookup table for factual recall.
Generative AI for Math (OpenWebMath / MathPile)
Releases a diverse, high-quality math-centric corpus of ~9.5B tokens designed for training math-capable foundation models.
Survey of Reasoning with Foundation Models
A comprehensive survey of reasoning with foundation models, covering tasks, methods, benchmarks, and future directions.
Gemini vs GPT-4V
A qualitative side-by-side comparison of Gemini and GPT-4V across vision-language tasks, documenting systematic behavioral differences.
ReST Meets ReAct
Proposes a ReAct-style agent that improves itself via reinforced self-training on its own reasoning traces.
Mathematical LLMs Survey
A survey on the progress of LLMs on mathematical reasoning tasks, covering methods, benchmarks, and open problems.
Beyond Human Data (ReST-EM)
DeepMind's ReST-EM shows that model-generated data plus a reward function can substantially reduce dependence on human-generated data.