What Has a Foundation Model Found?

Probing for Deeper Understanding – The authors argue that a model's true understanding of a domain is revealed by its inductive bias, how it generalizes from limited data. Their probe measures this by fine-tuning a model on small, synthetic datasets and observing how it extrapolates.

From Kepler to Newton? Not Quite – They test a transformer trained on orbital mechanics. While it can predict planetary trajectories with high accuracy (like Kepler), it fails to learn the underlying Newtonian mechanics. When asked to predict force vectors, it produces nonsensical laws of gravity that change from one solar system to another.

Heuristics, Not World Models – Across various domains, including physics, lattice problems, and the game of Othello, the study finds that foundation models tend to develop task-specific heuristics rather than a coherent world model. For example, in Othello, the model learns to predict legal moves, but not the full board state.

A Path Forward – The inductive bias probe provides a new way to diagnose the shortcomings of foundation models. By understanding what a model's inductive bias is, we can better understand what it has not learned and guide the development of models that can uncover the deeper truths in data.