QLoRA
Tim Dettmers' breakthrough technique enabling 65B LLM fine-tuning on a single 48GB GPU.
LIMA
Meta's 65B LLaMA fine-tuned on just 1,000 curated examples - showing alignment needs less data than believed.
Voyager
An LLM-powered embodied lifelong learning agent in Minecraft exploring autonomously.
Gorilla
A fine-tuned LLaMA-based model that surpasses GPT-4 on API call generation.
The False Promise of Imitating Proprietary LLMs
Berkeley's critical analysis of open-source imitation of proprietary LLMs.
Sophia
A simple, scalable second-order optimizer with negligible per-step overhead.
The Larger They Are, the Harder They Fail
Reveals inverse-scaling failures in LLM code generation.
Model Evaluation for Extreme Risks
DeepMind's framework for evaluating models for catastrophic-risk capabilities.
LLM Research Directions
A list of research directions for students entering LLM research.
Reinventing RNNs for the Transformer Era (RWKV)
Combines parallelizable training of Transformers with efficient RNN inference.