AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
TestGen-LLM
Meta's TestGen-LLM uses LLMs to improve existing human-written tests - augmenting coverage rather than generating tests from scratch - while rigorously filtering LLM output for quality.
ChemLLM
ChemLLM is a chemistry-specialized LLM with a matched dataset (ChemData) and benchmark (ChemBench) for evaluating chemistry-specific capability.
Survey of LLMs
A survey that maps the landscape of the three dominant LLM families - GPT, Llama, and PaLM - and the shared toolbox used to build and augment them.
LLM Agents Can Autonomously Hack Websites
The paper shows GPT-4 agents with tool use and long context can autonomously exploit real websites, including performing blind SQL injection and schema extraction.
Grandmaster-Level Chess Without Search
DeepMind shows that a 270M-parameter transformer trained purely with supervised learning on Stockfish-generated data reaches grandmaster-level chess without any search at inference time.
AnyTool
AnyTool is a training-free LLM agent that scales tool-use to 16K+ Rapid APIs through a hierarchical retriever and a self-reflective solver.
Phase Transition in Dot-Product Attention
A theoretical paper that analyzes a solvable low-rank tied-QK attention model and uncovers a data-driven phase transition between positional and semantic attention regimes.
Indirect Reasoning with LLMs (DIR)
Direct-Indirect Reasoning augments standard CoT with contrapositive and proof-by-contradiction templates, giving LLMs an explicit way to attack problems they can't solve forward.
ALOHA 2
ALOHA 2 is a refreshed low-cost bimanual teleoperation platform from Stanford/DeepMind, designed for large-scale robot-learning data collection.
More Agents Is All You Need
The paper shows that simply running more independent LLM agents and voting produces reliable scaling gains across tasks, without any method changes.
Self-Discover
Google's Self-Discover lets LLMs compose their own task-specific reasoning strategies from a small library of atomic reasoning modules, at dramatically lower inference cost than self-consistency.
DeepSeekMath
DeepSeek releases DeepSeekMath 7B, a math-specialized LLM that closes much of the gap to GPT-4 and Gemini-Ultra on MATH by combining better data and a new RL objective.
LLMs for Table Processing: A Survey
A survey covering how LLMs and VLMs are used across the full spectrum of table-processing tasks, from classic TableQA to spreadsheet manipulation.
LLM-based Multi-Agent Systems Survey
A survey of the fast-growing LLM-based multi-agent systems space, covering both problem-solving applications and "world simulation" research.
OLMo
Allen AI releases OLMo, a truly open 7B-parameter LLM shipped with training code, pretraining data, full weights, evaluation tooling, and fine-tuning recipes - an answer to the "open-weights but closed-pipeline" releases dominating the space.
Advances in Multimodal LLMs
A comprehensive survey mapping design choices for architecture and training pipeline around multimodal large language models (MLLMs).
Corrective RAG (CRAG)
CRAG adds a self-correcting loop around retrieval so a RAG system can detect and repair bad retrievals instead of feeding them straight into generation.
LLMs for Mathematical Reasoning
A survey of the fast-growing literature on using LLMs for mathematical reasoning, from arithmetic word problems to theorem proving.
Compression Algorithms for LLMs
A survey covering the main families of LLM compression techniques and when each one is appropriate.
MoE-LLaVA
MoE-LLaVA applies Mixture-of-Experts tuning to the LLaVA vision-language architecture, getting a sparse model with dramatically fewer active parameters at the same compute cost.
Rephrasing the Web (WRAP)
WRAP uses an off-the-shelf instruction-tuned model to paraphrase web documents into styles like "Wikipedia" or "question-answer format" and trains on the mixture of real + synthetic rephrases.
The Power of Noise: Redefining Retrieval in RAG
A study stress-testing the retriever component of RAG systems with surprising results about what actually helps generation.
Hallucination in LVLMs
A survey specifically scoped to hallucination in Large Vision-Language Models, a phenomenon that differs substantially from text-only LLM hallucination.
SliceGPT
Microsoft's SliceGPT is a post-training LLM compression technique that literally slices rows and columns out of weight matrices while preserving zero-shot quality.