Weak-to-Strong Generalization
Free while signed in. Answers cite the passages they came from.
OpenAI's superalignment team shows that weak supervisors can still elicit capabilities from much stronger models - a first empirical signal for scalable oversight.
Weak-to-strong setup: A weak model (e.g., GPT-2) generates labels, and a strong pretrained model (e.g., GPT-4) is fine-tuned on those labels - an analog of humans supervising superhuman AI.
Better than the supervisor: Naively fine-tuning the strong model on weak-model labels often yields a model better than the supervisor itself, demonstrating useful capability elicitation.
~GPT-3.5 from GPT-2 supervision: Fine-tuning GPT-4 with GPT-2-level supervision recovers close to GPT-3.5-level performance on NLP tasks - a surprising amount of capability without strong labels.
Superalignment signal: Offers an early empirical footing for the bet that humans can align superhuman systems using their own (weaker) judgments - provided the right training recipe.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack