AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Discover and explore top AI papers withClaude CodeCodex
npx @dair-ai/mcp setup
BiomedGPT
A unified biomedical GPT for vision, language, and multimodal tasks.
Thought Cloning
Imitation learning framework that learns to think as well as act.
Fine-Tuning Language Models with Just Forward Passes (MeZO)
A memory-efficient zeroth-order optimizer for LLM fine-tuning.
MERT
An acoustic music understanding model with large-scale self-supervised training.
Bytes Are All You Need
Performs classification directly on file bytes without decoding.
Direct Preference Optimization (DPO)
Rafailov et al.'s simpler alternative to RLHF that rivals full RL-based alignment.
SQL-PaLM
An LLM-based Text-to-SQL system built on PaLM-2.
CodeTF
An open-source Transformer library for state-of-the-art code LLMs.
QLoRA
Tim Dettmers' breakthrough technique enabling 65B LLM fine-tuning on a single 48GB GPU.
LIMA
Meta's 65B LLaMA fine-tuned on just 1,000 curated examples - showing alignment needs less data than believed.
Voyager
An LLM-powered embodied lifelong learning agent in Minecraft exploring autonomously.
Gorilla
A fine-tuned LLaMA-based model that surpasses GPT-4 on API call generation.
The False Promise of Imitating Proprietary LLMs
Berkeley's critical analysis of open-source imitation of proprietary LLMs.
Sophia
A simple, scalable second-order optimizer with negligible per-step overhead.
The Larger They Are, the Harder They Fail
Reveals inverse-scaling failures in LLM code generation.
Model Evaluation for Extreme Risks
DeepMind's framework for evaluating models for catastrophic-risk capabilities.
LLM Research Directions
A list of research directions for students entering LLM research.
Reinventing RNNs for the Transformer Era (RWKV)
Combines parallelizable training of Transformers with efficient RNN inference.
Drag Your GAN (DragGAN)
Interactive point-based image manipulation on the generative image manifold.
Evidence of Meaning in Language Models Trained on Programs
Argues LMs learn meaning despite only next-token prediction.
Towards Expert-Level Medical Question Answering (Med-PaLM 2)
Google's second-generation medical LLM.
MEGABYTE
Multiscale Transformers for predicting million-byte sequences.
StructGPT
A general framework for LLM reasoning over structured data.
TinyStories
Explores how small LMs can be and still speak coherent English.