AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
TAPIR
Tracks any queried point on any physical surface throughout a video sequence faster than real-time.
Mind2Web
A dataset for evaluating generalist web agents with 2,350 tasks across 137 websites and 31 domains.
Tracking Everything Everywhere All at Once (OmniMotion)
Test-time optimization for dense, long-range motion estimation.
AlphaDev
DeepMind's deep RL agent discovering faster sorting algorithms from scratch, now in LLVM.
Sparse-Quantized Representation (SpQR)
Tim Dettmers' near-lossless LLM compression technique.
MusicGen
A simple and controllable model for music generation using a single-stage Transformer.
Augmenting LLMs with Databases (ChatDB)
Combines an LLM with SQL databases as a symbolic memory framework.
Concept Scrubbing in LLM (LEACE)
Least-squares Concept Erasure - erases a target concept from every layer of a neural network.
Fine-Grained RLHF
Trains LMs with segment-level human feedback rather than whole-response preferences.
Hierarchical Vision Transformer (Hiera)
Pretrains ViTs with MAE while removing unnecessary multi-stage complexity.
Humor in ChatGPT
Explores ChatGPT's capabilities to grasp and reproduce humor.
Imitating Reasoning Process of Larger LLMs (Orca)
Microsoft's 13B model that imitates GPT-4's reasoning traces.
Let's Verify Step by Step
OpenAI's landmark paper on process reward models for mathematical reasoning.
No Positional Encodings (NoPE)
Shows explicit position embeddings aren't essential for decoder-only Transformers.
BiomedGPT
A unified biomedical GPT for vision, language, and multimodal tasks.
Thought Cloning
Imitation learning framework that learns to think as well as act.
Fine-Tuning Language Models with Just Forward Passes (MeZO)
A memory-efficient zeroth-order optimizer for LLM fine-tuning.
MERT
An acoustic music understanding model with large-scale self-supervised training.
Bytes Are All You Need
Performs classification directly on file bytes without decoding.
Direct Preference Optimization (DPO)
Rafailov et al.'s simpler alternative to RLHF that rivals full RL-based alignment.
SQL-PaLM
An LLM-based Text-to-SQL system built on PaLM-2.
CodeTF
An open-source Transformer library for state-of-the-art code LLMs.
QLoRA
Tim Dettmers' breakthrough technique enabling 65B LLM fine-tuning on a single 48GB GPU.
LIMA
Meta's 65B LLaMA fine-tuned on just 1,000 curated examples - showing alignment needs less data than believed.