QLoRA
Tim Dettmers' breakthrough technique enabling 65B LLM fine-tuning on a single 48GB GPU.
LIMA
Meta's 65B LLaMA fine-tuned on just 1,000 curated examples - showing alignment needs less data than believed.
Voyager
An LLM-powered embodied lifelong learning agent in Minecraft exploring autonomously.
Gorilla
A fine-tuned LLaMA-based model that surpasses GPT-4 on API call generation.
The False Promise of Imitating Proprietary LLMs
Berkeley's critical analysis of open-source imitation of proprietary LLMs.
Sophia
A simple, scalable second-order optimizer with negligible per-step overhead.
The Larger They Are, the Harder They Fail
Reveals inverse-scaling failures in LLM code generation.
Model Evaluation for Extreme Risks
DeepMind's framework for evaluating models for catastrophic-risk capabilities.
LLM Research Directions
A list of research directions for students entering LLM research.
Reinventing RNNs for the Transformer Era (RWKV)
Combines parallelizable training of Transformers with efficient RNN inference.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack