Synthetic Data Reduces Sycophancy
Free while signed in. Answers cite the passages they came from.
Google shows that fine-tuning on simple synthetic data can significantly reduce LLM sycophancy.
Sycophancy problem: Sycophancy occurs when LLMs align their responses with perceived user views even when those views are factually incorrect.
Synthetic anti-sycophancy data: Constructs simple synthetic examples where the correct answer contradicts the user's stated view, then fine-tunes models on them.
Meaningful reduction: Fine-tuning on this synthetic data measurably reduces sycophantic behavior without degrading overall helpfulness.
Broader lesson: Offers a cheap, targeted intervention for a specific alignment failure mode - a template for addressing other narrow failure modes through targeted synthetic data.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack