The AI-Enabled Scientific Frontier
Gabriel Manso, Emma Fu and Neil Thompson (MIT FutureTech) assemble 2,507 head-to-head comparisons between AI and other analysis methods across 27 disciplines from 2000 to early 2025.
Ask this paper
Against statistics: AI often outperforms traditional statistics at much higher compute cost, but in nearly a quarter of cases it is both more expensive and worse, a share that has been stable for a decade.
Against scientific computing: AI often underperforms physics-based computing at lower cost, and since 2020 its relative performance has been improving.
Position: The evidence supports treating AI as a context-dependent technology rather than a universal scientific method.
Use: The corpus gives a cross-domain cost-performance map that individual fields can use before generalizing from their own experience.
Abstract
As artificial intelligence's capabilities improve, it is increasingly viewed as a general scientific method. But how true are these claims? Does AI outperform all techniques, or only some, and how is this changing? To assess the claims, we assemble a corpus of 2,507 head-to-head comparisons between AI and other scientific analysis techniques across 27 scientific disciplines from papers published between 2000 and early 2025. We find a profound dichotomy. Relative to traditional statistics, AI often outperforms, but at a significantly higher computational cost. But there are also nearly a quarter of cases where AI is both more expensive and performs worse than traditional statistical techniques and this fraction has been stable for a decade. Relative to scientific computing, AI often underperforms, but at lower computational cost. This has begun to change: since 2020, AI's performance against scientific computing has notably strengthened and it now outperforms on more than half of comparisons. These patterns suggest that AI is therefore not a universal replacement for existing methods, but rather a valuable -- and improving -- part of a new AI-enabled scientific frontier.