Survey of Aligned LLMs

Full-stack view: Covers preference data collection, RLHF variants, DPO-style direct methods, and alignment evaluation in one unified reference.

Taxonomy of methods: Organizes alignment techniques into clear families (outer alignment vs. inner alignment, value alignment vs. behavior alignment).

Practical pitfalls: Documents known failure modes like reward hacking, sycophancy, and mode collapse that practitioners should watch for.

Reference document: Frequently cited in alignment onboarding material as the first-pass overview for new researchers.