AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Depth Anything
A robust monocular depth estimator designed to handle "any image under any circumstance" by scaling self-training on unlabeled data rather than hunting for bigger labeled sets.
Knowledge Fusion of LLMs (FuseLLM)
FuseLLM proposes fusing the capabilities of multiple existing LLMs into a single target model by distilling their output distributions rather than retraining from scratch.
MambaByte
MambaByte adapts the Mamba state-space architecture to learn directly from raw bytes, bypassing tokenization and all its well-known failure modes.
Diffuse to Choose
Amazon's Diffuse to Choose is a diffusion-based image-conditioned inpainting model built for "virtual try-on" scenarios where product images must be placed naturally into user scenes.
WARM (Weighted Averaged Reward Models)
WARM averages multiple fine-tuned reward models in weight space rather than ensembling their predictions, dramatically reducing RLHF inference cost.
Resource-efficient LLMs & Multimodal Foundation Models
A wide-ranging survey of efficiency techniques for LLMs and multimodal foundation models, spanning architecture, algorithms, and system design.
Red Teaming Visual Language Models
Introduces the first dedicated red-teaming benchmark for VLMs, covering vulnerabilities unique to multimodal inputs.
Lumiere
Google's Lumiere is a space-time diffusion model for text-to-video that generates the entire video duration in a single forward pass rather than cascading short clips.
Medusa
Medusa accelerates LLM inference by bolting on multiple decoding heads that predict several future tokens in parallel, dramatically reducing decoding steps.
AgentBoard
AgentBoard is a benchmark and open-source evaluation framework for analytically evaluating LLM agents beyond the usual pass/fail metrics.
AlphaGeometry
DeepMind's AlphaGeometry is a theorem prover that solves Olympiad-level geometry problems at near gold-medallist performance, and crucially, without needing any human demonstrations.
AlphaCodium
AlphaCodium is a test-based, iterative "flow" that turns off-the-shelf LLMs into strong competitive-programming solvers without model training.
RAG vs. Finetuning
Microsoft researchers systematically compare RAG and fine-tuning (and their combination) on LLMs like Llama 2 and GPT-4 using an agricultural domain dataset.
Self-Rewarding Language Models
Meta shows that an LLM can act as both actor and judge in its own alignment loop, generating training data without any external reward model.
Tuning Language Models by Proxy
Proxy-tuning steers a large frozen LLM by *decoding-time* logit arithmetic using a much smaller fine-tuned model as a "proxy".
ReFT (Reinforced Fine-Tuning)
ByteDance's ReFT enhances LLM reasoning by combining supervised fine-tuning with online RL that samples alternative reasoning paths, without a learned reward model.
Overview of LLMs for Evaluation
A thorough survey of LLM-as-a-Judge and LLM-based evaluation methodologies, mapping strengths, limitations, and open problems.
Patchscopes
Patchscopes is a general framework for inspecting and intervening on LLM internals by "patching" hidden representations into a second inference pass.
Easy-to-Hard Generalization
UNC researchers show that LLMs often generalize well from easy training data to hard evaluation data, with implications for scalable oversight.
MoE-Mamba
MoE-Mamba combines state-space models (Mamba) with Mixture-of-Experts to scale LLMs more efficiently than either Mamba or Transformer-MoE alone.
InseRF
InseRF inserts brand-new 3D objects into Neural Radiance Field scenes from just a text prompt plus a 2D bounding box, without requiring any explicit 3D input.
Sleeper Agents
Anthropic shows that LLMs can be trained to act deceptively under specific triggers and that current safety training techniques fail to remove this hidden behavior.
Blending Is All You Need
Small chat models (6B/13B) blended together can rival ChatGPT-class systems, without any new training.
MagicVideo-V2
ByteDance's MagicVideo-V2 is an end-to-end text-to-video pipeline that stitches together four specialized modules into a high-fidelity generation system.