Weak-to-Strong Generalization
OpenAI's superalignment team shows that weak supervisors can still elicit capabilities from much stronger models - a first empirical signal for scalable oversight.
Ask this paper
Weak-to-strong setup: A weak model (e.g., GPT-2) generates labels, and a strong pretrained model (e.g., GPT-4) is fine-tuned on those labels - an analog of humans supervising superhuman AI.
Better than the supervisor: Naively fine-tuning the strong model on weak-model labels often yields a model better than the supervisor itself, demonstrating useful capability elicitation.
~GPT-3.5 from GPT-2 supervision: Fine-tuning GPT-4 with GPT-2-level supervision recovers close to GPT-3.5-level performance on NLP tasks - a surprising amount of capability without strong labels.
Superalignment signal: Offers an early empirical footing for the bet that humans can align superhuman systems using their own (weaker) judgments - provided the right training recipe.