Explaining Grokking

Ungrokking: A model can go from perfect generalization back to memorization when trained further on a smaller dataset below a critical threshold - the first demonstration of this reverse effect.

Semi-grokking: A randomly initialized network trained on the critical dataset size shows a grokking-like transition but partial, rather than the sharp full-grokking curve.

Theoretical predictions: These behaviors were predicted from theory before being demonstrated empirically - a rare example of predictive rather than post-hoc explanation in deep learning.

Generalization theory: Advances understanding of when and why neural networks transition from memorization to generalization, bridging empirical observation with principled prediction.