The Time is Here for Just-in-Time Systems: Challenges and Opportunities
Shu Liu, Alexander Krentsel, Shubham Agarwal, Mert Cemri and colleagues (UC Berkeley with Bespoke Labs) argue that coding agents make it practical to synthesize a core system from scratch for each deployment, and they present Jitskit, a pipeline that builds key-value stores specialized to one workload, one set of resource limits and one set of required guarantees.
Ask this paper
Just-in-Time Systems: General-purpose systems such as key-value stores are built once and used everywhere, and that generality costs performance. A JIT system is generated for a specific environment, workload and set of required properties.
Jitskit: The pipeline takes a spec card describing a YCSB workload, deployment constraints such as compute resources, and properties such as consistency and durability. It refines the implementation against an evaluation test suite that grows during synthesis.
Results: The synthesized stores beat comparable state-of-the-art systems on 18 of 18 specs, by up to 4.6x over the best off-the-shelf baseline on the most favorable spec.
Plain agent baseline: Running Claude Code directly on the same task either reward-hacks the tests or produces systems up to 5.4x slower than Jitskit's output.
Abstract
Core systems like key-value stores have historically taken years to build, and are designed to be general so as to amortize cost across deployments, paying a significant performance cost. We argue that LLM-based coding agents now make a different approach tractable: Just-in-Time Systems, in which the entire system is synthesized from scratch, specialized to the environment, workload, and required system properties. We present a JIT system synthesis pipeline, Jitskit, and explore its effectiveness in synthesizing key-value stores from spec cards that span different YCSB workloads, deployment constraints (e.g., compute resources), and system properties (e.g., consistency and durability). Jitskit iteratively refines a system implementation to match the specification against an evolving evaluation test suite. The resulting synthesized systems are performant, beating comparable state-of-the-art systems on 18 of 18 specs tried, by up to 4.6x over the best off-the-shelf baseline on the most favorable spec. Naively running Claude Code either reward-hacks or underperforms Jitskit by up to 5.4x. We discuss the challenges we overcame in building Jitskit and our key takeaways.