LLMs Can Get “Brain Rot”!
Free while signed in. Answers cite the passages they came from.
The authors test a clear hypothesis: continual pretraining on trivial, highly engaging web text degrades LLM cognition in ways that persist even after mitigation. They build controlled Twitter datasets to isolate data quality from scale and training ops, then measure effects on reasoning, long-context, safety, and personality.
Setup that isolates data quality: Two orthogonal junk definitions: M1 uses engagement signals and short length to capture popular, bite-sized posts; M2 uses semantic cues like clickbait and superficial topics. Four instruct models are continually pretrained with matched token counts and then re-instruction tuned, enabling apples-to-apples comparisons with control data.
Non-trivial capability decay with dose response: Across models, junk exposure reduces ARC reasoning, long-context retrieval, and safety, with Hedges’ g exceeding 0.3. Increasing the M1 junk ratio drives smooth drops, for example, ARC-Challenge with CoT 74.9 to 57.2 and RULER CWE 84.4 to 52.3 from 0% to 100% junk.
Thought-skipping is the primary lesion: Error forensics on ARC CoT show failures dominated by no thinking, no plan, and skipping planned steps, explaining over 98% of errors. Popularity is a stronger predictor of this rot for reasoning than length, while length matters more for long-context.
Safety and “dark traits” worsen under M1: Junk training elevates risk on HH-RLHF and AdvBench and inflates narcissism and psychopathy scores, while lowering agreeableness. Personality and safety outcomes diverge between M1 and M2, highlighting that engagement signals capture a harmful non-semantic axis of quality.
Mitigations help but do not heal: External reflection with a stronger model reduces thought-skipping and recovers accuracy; self-reflection does not. Scaling instruction tuning and clean continual training improve scores yet fail to close the gap to baseline, indicating persistent representational drift.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack