AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
DROID
DROID is an open-source robot manipulation dataset that dramatically expands the diversity of real-world robot demonstrations available for imitation-learning research.
RAFT: Retrieval-Augmented Fine-Tuning
RAFT is a fine-tuning recipe that teaches LLMs to handle distractor documents during RAG and to answer with CoT-style citations to retrieved passages.
SIMA
DeepMind's Scalable Instructable Multiworld Agent (SIMA) is a generalist AI agent that follows natural-language instructions across nine commercial 3D video games like No Man's Sky, Teardown, Valheim, and Space Engineers.
Retrieval Augmented Thoughts (RAT)
RAT augments chain-of-thought by iteratively rewriting each reasoning step using retrieved context, sharply reducing hallucination on long-horizon generation tasks.
Quiet-STaR
Quiet-STaR generalizes the Self-Taught Reasoner (STaR) so that a language model learns to generate internal rationales between every token, not just for explicit QA problems.
Knowledge Conflicts for LLMs
A survey that maps the landscape of knowledge conflicts in LLMs, covering how they arise, how models behave under them, and how to mitigate them.
Stealing Part of a Production Language Model
The paper demonstrates the first practical attack that extracts the embedding-projection layer of production LLMs through their ordinary logit APIs.
Branch-Train-MiX (BTX)
Meta's BTX produces a single Mixture-of-Experts LLM by first training specialized experts in parallel and then mixing them, sidestepping the high cost of training one big generalist.
LLMs Predict Neuroscience Results (BrainBench)
BrainBench asks both LLMs and human experts to predict the outcomes of neuroscience experiments from their abstracts, and finds LLMs outperform experts.
C4AI Command-R
Cohere for AI releases Command-R, a 35B open-weight LLM tuned specifically for retrieval-augmented generation, tool use, and multilingual workflows.
Is Cosine-Similarity Really About Similarity?
This paper argues that cosine similarity between learned embeddings does not always measure semantic similarity, and gives analytical examples where it produces arbitrary or non-unique values.
MM1: Multimodal LLM Pre-training
Apple's MM1 paper runs extensive ablations on multimodal LLM pretraining choices and releases a family of models up to 30B parameters that set competitive MLLM pretraining benchmarks.
Claude 3
Anthropic releases the Claude 3 family (Haiku, Sonnet, Opus), with Opus leapfrogging GPT-4 on many standard benchmarks and bringing frontier multimodal capability plus a much larger context window.
Robust Evaluation of Reasoning
The paper introduces functional benchmarks that parameterize reasoning problems so the same structural question can be re-instantiated with fresh surface forms, then uses them to expose a large "reasoning gap" in frontier LLMs.
GaLore
GaLore (Gradient Low-Rank Projection) reduces optimizer-state memory during LLM training while still permitting full-parameter updates, unlike LoRA-style adapters that restrict learning to a low-rank subspace.
Can LLMs Reason and Plan?
Kambhampati's position paper argues that what looks like reasoning and planning in LLMs is better understood as "universal approximate retrieval" powered by web-scale training.
RAG for AI-Generated Content
A survey that extends RAG beyond text, showing how retrieval augmentation is being applied across code, image, audio, video, and 3D generation.
KnowAgent
KnowAgent improves LLM-based planning agents by explicitly injecting action knowledge - what the actions are and how they relate - rather than letting the LLM invent its own action space at runtime.
Sora Overview
A comprehensive academic review of OpenAI's Sora, tracing the technical ingredients behind the text-to-video "world simulator" and the opportunities/limitations for the next wave of large vision models.
SaulLM-7B: LLM for Law
SaulLM-7B is an open legal-domain LLM built on Mistral 7B and continually pretrained on 30B+ tokens of English legal text, with a companion instruction-tuning recipe.
Design2Code
Design2Code tackles the front-end engineering problem of turning a visual design into working HTML/CSS and gives the community both a benchmark and strong MLLM baselines.
TripoSR
TripoSR is a transformer-based single-image 3D reconstruction model that returns a textured mesh in under 0.5 seconds, building on the LRM architecture with a stronger data and training pipeline.
Genie
DeepMind's Genie is an 11B-parameter foundation world model trained unsupervised on internet gameplay videos that generates action-controllable 2D worlds from a single image prompt.
Mistral Large
Mistral AI releases Mistral Large, its flagship closed-weight LLM positioned as the second-ranked API-accessible model behind GPT-4 at launch.