Illusion of Progress
Free while signed in. Answers cite the passages they came from.
The paper argues that common QA hallucination detectors look better than they are because evaluations lean on ROUGE. In human‑aligned tests, many detectors drop sharply. Simple response‑length heuristics rival sophisticated methods, revealing a core evaluation flaw.
ROUGE misaligns with humans. In a human study, LLM‑as‑Judge matches human labels much better than ROUGE.
Re‑scoring detectors collapses headline results. When replacing ROUGE with LLM‑as‑Judge, AUROC drops are large: up to −45.9% for Perplexity and −30.4% for Eigenscore on NQ‑Open with Mistral; PR‑AUC gaps are even larger. Correlation between ROUGE‑ and LLM‑based AUROC is only r = 0.55.
Length is the hidden confounder. Hallucinated answers are typically longer with higher variance. Many detectors are strongly correlated with length, not semantics. ROUGE systematically penalizes long responses and can be gamed by repetition without changing facts.
Simple baselines rival complex methods. Length features like mean and std across samples achieve competitive AUROC, sometimes matching or beating Eigenscore and LN‑Entropy.
Few‑shot helps format, not truth. Few‑shot examples reduce some ROUGE vs LLM‑as‑Judge discrepancies and stabilize outputs, but method rankings still shift and model effects persist; Semantic Entropy is relatively more stable.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack