AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
BabyLLM Challenge Findings
Reports results from a challenge on sample-efficient pretraining using a developmentally plausible corpus.
FunSearch
DeepMind's FunSearch uses LLMs as a mutation operator in an evolutionary loop to discover genuinely new mathematical knowledge.
Weak-to-Strong Generalization
OpenAI's superalignment team shows that weak supervisors can still elicit capabilities from much stronger models - a first empirical signal for scalable oversight.
Audiobox
Meta's Audiobox is a unified flow-matching audio model that generates speech, sound effects, and music from natural-language and example prompts.
Mathematical LLMs Survey
A survey on the progress of LLMs on mathematical reasoning tasks, covering methods, benchmarks, and open problems.
LLM360
LLM360 is a framework for fully transparent open-source LLM development, with everything from data to training dynamics released.
LLMs in Medicine
A comprehensive survey (300+ papers) of LLMs applied to medicine, from clinical tasks to biomedical research.
Beyond Human Data (ReST-EM)
DeepMind's ReST-EM shows that model-generated data plus a reward function can substantially reduce dependence on human-generated data.
Gaussian-SLAM
A neural RGBD SLAM method that extends 3D Gaussian Splatting to achieve photorealistic scene reconstruction without sacrificing speed.
Pearl
Meta's Pearl is a production-ready reinforcement learning agent package designed for real-world deployment constraints.
QuIP#
Cornell's QuIP# is a 2-bit LLM quantization scheme that combines lattice codebooks with incoherence processing to close the quality gap to FP16.
Gemini 1.0
Google launches Gemini 1.0, a multimodal family natively designed to reason across text, images, video, audio, and code from the ground up.
EfficientSAM
Meta's EfficientSAM is a lightweight Segment Anything variant that preserves most of SAM's zero-shot quality at a fraction of the compute.
Magicoder
Magicoder is a fully open-source code LLM that closes the gap with top commercial code models at only 7B parameters via high-quality synthetic instruction data.
LLMs on Graphs
A comprehensive overview of the many ways LLMs can be applied to graph-structured data and when each pattern is useful.
Llama Guard
Meta's Llama Guard is a compact, instruction-tuned safety classifier built on Llama 2-7B for input/output moderation in conversational AI.
KTO (Kahneman-Tversky Optimization)
Contextual AI introduces KTO, an alignment objective derived from prospect theory that works with binary "good/bad" signals instead of preference pairs.
Chain of Code
DeepMind's Chain of Code extends CoT by encouraging LMs to write pseudocode that mixes real code with LM-simulated sub-routines.
Data Management for LLMs
A survey of data-management research for LLM pretraining and supervised fine-tuning stages.
RankZephyr
RankZephyr is an open-source LLM for listwise zero-shot reranking that bridges the effectiveness gap with GPT-4.
The Efficiency Spectrum of LLMs
A comprehensive review of algorithmic advancements for improving LLM efficiency across the full training-to-inference stack.
GNoME
DeepMind's Graph Networks for Materials Exploration (GNoME) is an AI system that discovered 2.2 million new crystal structures, including 380,000 thermodynamically stable ones.
Open-Source LLMs vs. ChatGPT
A survey cataloguing tasks where open-source LLMs claim to be on par with or better than ChatGPT.
Adversarial Diffusion Distillation (SDXL Turbo)
Stability AI's ADD trains a student diffusion model that produces high-quality images in just 1-4 sampling steps.