LLMs Represent Space and Time
Free while signed in. Answers cite the passages they came from.
MIT researchers find that LLMs internally encode linear representations of space and time across multiple scales.
Linear geographic representations: Activations contain linear representations of coordinates (latitude, longitude) of real-world entities, detectable via probes.
Multi-scale time: Similar linear representations exist for time at multiple scales (historical year, news date, etc.), suggesting a structured temporal axis.
Robust across prompts: The representations are robust to prompt variations and unified across different entity types (cities, events, people).
World-model evidence: Provides empirical support for the claim that LLMs build literal world models, not just surface-statistics imitators - a live debate in interpretability.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack