Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool
Samuel Kushnir, Kimia Noorbakhsh and colleagues at MIT, Google, Google DeepMind and Stanford keep a performance-modeling library whose main branch contains almost no code: the repository is a DAG of natural-language design docs that coding sub-agents regenerate the implementation from.
Ask this paper
Every human change is a doc edit: Engineers do not patch the code directly. They edit a design doc in natural language, so the library documents itself by construction.
Why regenerate at all: ML performance modeling invalidates its own abstractions with every new generation of hardware and models. The authors argue that coding agents are now fast and cheap enough that regenerating a library costs less than paying down the tech debt of patching it.
Two ingredients keep regeneration reliable: The design docs are written around step-by-step worked examples that act as in-context demonstrations for the generating agents. The system is also anchored on a minimal, recursively defined operator IR with symbolic cost expressions in SymPy, with a fast analytical mode for large sweeps and a slower modulo-scheduling mode for fine-grained schedule studies.
Why it matters: Regenerated implementations reproduce hand-audited reference models to round-off precision, including DeepSeek-V3 serving on a TPU pod slice. That result supports keeping design docs as the durable artifact in fast-moving domains and regenerating the code as a build product.
Abstract
Machine-learning performance modeling is a uniquely hostile terrain for long-lived software: the assumptions baked into today's abstractions are invalidated by tomorrow's models and systems, forcing perpetual refactoring of performance-modeling frameworks. Meanwhile, AI coding agents have become fast and capable enough that regenerating an entire library is cheaper than paying down the tech debt of incrementally patching it. We describe SMART, a rigorous symbolic performance-modeling library for ML systems whose main branch contains almost no code: the repository is a DAG of self-contained natural-language design docs, coding sub-agents regenerate the implementation from only the docs on new version updates, and every human change is a natural-language edit to a doc--self-documenting by construction. Two ingredients make regeneration reliable: (i) a design-doc style built around step-by-step worked examples that act as in-context demonstrations for the generating agents, and (ii) a minimal, recursively defined operator IR with symbolic (SymPy) cost expressions, a fast analytical roll-up mode for large sweeps, and a slow modulo-scheduling mode for fine-grained schedule studies. Regenerated implementations reproduce hand-audited reference models--including DeepSeek-V3 serving on a TPU pod slice--to round-off precision, suggesting that design docs--not code--can be the durable artifact for ML-systems co-design tools.