AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
ConvNets Match Vision Transformers
DeepMind shows that strong ConvNet architectures pretrained at scale match ViTs on ImageNet performance at comparable compute.
CommonCanvas
Releases CommonCanvas, a text-to-image dataset composed entirely of Creative-Commons-licensed images.
Managing AI Risks (Bengio, Hinton, et al.)
A high-profile position paper by leading AI researchers laying out risks from upcoming advanced AI systems.
Branch-Solve-Merge (BSM)
BSM decomposes LLM tasks into parallel sub-tasks via three LLM-programmed modules: branch, solve, and merge.
Llemma
Llemma is an open LLM for mathematics built via continued pretraining of Code Llama on the Proof-Pile-2 dataset.
LLMs for Software Engineering
A comprehensive survey of LLMs for software engineering covering models, tasks, evaluation, and open challenges.
Self-RAG
Self-RAG trains an LM to adaptively retrieve, generate, and self-critique using special reflection tokens.
RAG for Long-Form QA
Explores retrieval-augmented LMs specifically on long-form question answering, where RAG failures are more subtle.
GenBench
A Nature Machine Intelligence paper framework for characterizing and understanding generalization research in NLP.
LLM Self-Explanations
Investigates whether LLMs can generate useful feature-attribution explanations for their own outputs.
OpenAgents
An open platform for running and hosting real-world language agents, including three distinct agent types.
Eliciting Human Preferences with LLMs
Anthropic uses LLMs to guide the task-specification process, eliciting user intent through natural-language dialogue.
AutoMix
AutoMix routes queries between LLMs of different sizes based on smaller-model confidence, saving cost without sacrificing quality.
Video Language Planning
Enables synthesizing complex long-horizon video plans for robotics via tree search over vision-language and text-to-video models.
Ring Attention
UC Berkeley's Ring Attention scales transformer context to 100M+ tokens by distributing blockwise self-attention across devices in a ring topology.
UniSim (Universal Simulator)
Google's UniSim learns a universal generative simulator of real-world interactions from diverse video + action data.
Survey on Factuality in LLMs
A survey covering evaluation and enhancement techniques for LLM factuality.
Hypothesis Search (LLMs Can Learn Rules)
A two-stage framework where the LLM learns a rule library for reasoning.
Meta Chain-of-Thought Prompting (Meta-CoT)
A generalizable CoT framework that selects domain-appropriate reasoning patterns for the task at hand.
LLMs for Healthcare Survey
A comprehensive overview of LLMs applied to the healthcare domain.
RECOMP (Retrieval-Augmented LMs with Compressors)
Proposes two compression approaches to shrink retrieved documents before in-context use.
InstructRetro
NVIDIA introduces Retro 48B, the largest LLM pretrained with retrieval at the time.
MemWalker
MemWalker treats the LLM as an interactive agent that traverses a tree-structured summary of long text.
FireAct (Language Agent Fine-tuning)
Explores fine-tuning LLMs specifically for language-agent use, demonstrating consistent gains over prompting alone.