AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
MM1: Multimodal LLM Pre-training
Apple's MM1 paper runs extensive ablations on multimodal LLM pretraining choices and releases a family of models up to 30B parameters that set competitive MLLM pretraining benchmarks.
Claude 3
Anthropic releases the Claude 3 family (Haiku, Sonnet, Opus), with Opus leapfrogging GPT-4 on many standard benchmarks and bringing frontier multimodal capability plus a much larger context window.
Robust Evaluation of Reasoning
The paper introduces functional benchmarks that parameterize reasoning problems so the same structural question can be re-instantiated with fresh surface forms, then uses them to expose a large "reasoning gap" in frontier LLMs.
GaLore
GaLore (Gradient Low-Rank Projection) reduces optimizer-state memory during LLM training while still permitting full-parameter updates, unlike LoRA-style adapters that restrict learning to a low-rank subspace.
Can LLMs Reason and Plan?
Kambhampati's position paper argues that what looks like reasoning and planning in LLMs is better understood as "universal approximate retrieval" powered by web-scale training.
RAG for AI-Generated Content
A survey that extends RAG beyond text, showing how retrieval augmentation is being applied across code, image, audio, video, and 3D generation.
KnowAgent
KnowAgent improves LLM-based planning agents by explicitly injecting action knowledge - what the actions are and how they relate - rather than letting the LLM invent its own action space at runtime.
Sora Overview
A comprehensive academic review of OpenAI's Sora, tracing the technical ingredients behind the text-to-video "world simulator" and the opportunities/limitations for the next wave of large vision models.
SaulLM-7B: LLM for Law
SaulLM-7B is an open legal-domain LLM built on Mistral 7B and continually pretrained on 30B+ tokens of English legal text, with a companion instruction-tuning recipe.
Design2Code
Design2Code tackles the front-end engineering problem of turning a visual design into working HTML/CSS and gives the community both a benchmark and strong MLLM baselines.
TripoSR
TripoSR is a transformer-based single-image 3D reconstruction model that returns a textured mesh in under 0.5 seconds, building on the LRM architecture with a stronger data and training pipeline.
Genie
DeepMind's Genie is an 11B-parameter foundation world model trained unsupervised on internet gameplay videos that generates action-controllable 2D worlds from a single image prompt.
Mistral Large
Mistral AI releases Mistral Large, its flagship closed-weight LLM positioned as the second-ranked API-accessible model behind GPT-4 at launch.
The Era of 1-bit LLMs (BitNet b1.58)
Microsoft's BitNet b1.58 shows that restricting every weight to the ternary set {-1, 0, 1} can match full-precision FP16 transformers on perplexity and downstream tasks at the same parameter count.
Datasets for LLMs: A Comprehensive Survey
A 180+-page survey that catalogs and analyzes the datasets that underpin modern LLM training and evaluation.
LearnAct
LearnAct lets language agents expand and refine their own action space over time by writing and revising Python functions in response to execution feedback.
EMO: Emote Portrait Alive
Alibaba's EMO synthesizes expressive talking-head videos directly from audio, bypassing the intermediate 3D models or facial landmarks used by prior approaches.
On the Societal Impact of Open Foundation Models
Stanford CRFM's policy paper proposes a rigorous framework for assessing the *marginal* risk of open-weight foundation models relative to closed models and pre-existing technologies.
StarCoder 2
BigCode releases StarCoder 2, an open family of code LLMs at 3B, 7B, and 15B parameters trained on The Stack v2, a much larger and cleaner code corpus than the original StarCoder.
LLMs on Tabular Data: A Survey
A survey that maps how LLMs are being applied to tabular data tasks - a domain historically dominated by gradient-boosted trees and specialized architectures.
PlanGPT
PlanGPT is a domain-specialized LLM framework for urban and spatial planning, built in collaboration with the Chinese Academy of Urban Planning.
Stable Diffusion 3
Stability AI previews Stable Diffusion 3, a suite of image-generation models from 800M to 8B parameters that shifts to a diffusion-transformer backbone with flow matching.
Gemma
Google DeepMind releases Gemma, a family of open models (2B and 7B) built from the same research stack as Gemini and shipped with both base and instruction-tuned variants.
LLMs for Data Annotation
A survey that maps the rapidly growing literature on using LLMs to generate, evaluate, and learn from data annotations.