VibeTensor
Free while signed in. Answers cite the passages they came from.
VibeTensor is an open-source deep learning system software stack from NVLabs that was fully generated by LLM-powered coding agents under high-level human guidance. The system implements a PyTorch-style eager tensor library with a C++20/CUDA core, Python and Node.js frontends, its own autograd engine, CUDA runtime, and caching allocator - demonstrating that coding agents can produce coherent system software spanning language bindings down to GPU memory management. - **Full-stack generated architecture:** The system includes a schema-lite dispatcher, reverse-mode autograd engine, stream-ordered caching allocator with diagnostics, CUDA graph support, and a stable C ABI for dynamically loaded operator plugins. The 28B LOC codebase spans 218 core C++ files and 225 Python test files, all generated without per-change manual diff review. - **AI-assisted development methodology:** A two-month development cycle used a simple loop: specify scoped goals, generate code, compile and test, then broaden validation. Tests as specifications and differential checks against PyTorch served as key guardrails, with multi-agent code review catching unsafe patterns. - **Kernel performance and training validation:** An accompanying AI-generated kernel suite shows mixed results: 1.54x faster than FlashAttention on NanoChat-style training (batch 32, seq 2048) but 0.67x on small-batch GQA prefill. End-to-end training on H100 and Blackwell GPUs converges correctly but runs 1.7-6.2x slower than PyTorch. - **The Frankenstein composition effect:** The paper identifies a key failure mode where individually correct generated subsystems compose into globally suboptimal designs - for example, a correctness-first autograd gate serializes execution and starves efficient backend kernels, highlighting challenges unique to AI-generated system software.
Full-stack generated architecture: The system includes a schema-lite dispatcher, reverse-mode autograd engine, stream-ordered caching allocator with diagnostics, CUDA graph support, and a stable C ABI for dynamically loaded operator plugins. The 28B LOC codebase spans 218 core C++ files and 225 Python test files, all generated without per-change manual diff review.
AI-assisted development methodology: A two-month development cycle used a simple loop: specify scoped goals, generate code, compile and test, then broaden validation. Tests as specifications and differential checks against PyTorch served as key guardrails, with multi-agent code review catching unsafe patterns.
Kernel performance and training validation: An accompanying AI-generated kernel suite shows mixed results: 1.54x faster than FlashAttention on NanoChat-style training (batch 32, seq 2048) but 0.67x on small-batch GQA prefill. End-to-end training on H100 and Blackwell GPUs converges correctly but runs 1.7-6.2x slower than PyTorch.
The Frankenstein composition effect: The paper identifies a key failure mode where individually correct generated subsystems compose into globally suboptimal designs - for example, a correctness-first autograd gate serializes execution and starves efficient backend kernels, highlighting challenges unique to AI-generated system software.
Abstract
VIBETENSOR is an open-source research system software stack for deep learning, generated by LLM-powered coding agents under high-level human guidance. In this paper, "fully generated" refers to code provenance: implementation changes were produced and applied as agent-proposed diffs; validation relied on agent-run builds, tests, and differential checks, without per-change manual diff review. It implements a PyTorch-style eager tensor library with a C++20 core (CPU+CUDA), a torch-like Python overlay via nanobind, and an experimental this http URL interface. Unlike thin bindings, VIBETENSOR includes its own tensor/storage system, schema-lite dispatcher, reverse-mode autograd, CUDA runtime (streams/events/graphs), a stream-ordered caching allocator with diagnostics, and a stable C ABI for dynamically loaded operator plugins. We view this release as a milestone for AI-assisted software engineering: it shows coding agents can generate a coherent deep learning runtime spanning language bindings down to CUDA memory management, validated primarily by builds and tests. We describe the architecture, summarize the workflow used to produce and validate the system, and evaluate the artifact. We report repository scale and test-suite composition, and summarize reproducible microbenchmarks from an accompanying AI-generated kernel suite, including fused attention versus PyTorch SDPA/FlashAttention. We also report end-to-end training sanity checks on 3 small workloads (sequence reversal, ViT, miniGPT) on NVIDIA H100 (Hopper, SM90) and Blackwell-class GPUs; multi-GPU results are Blackwell-only and use an optional CUTLASS-based ring-allreduce plugin gated on CUDA 13+ and sm103a toolchain support. Finally, we discuss failure modes in generated system software, including a "Frankenstein" composition effect where locally correct subsystems interact to yield globally suboptimal performance.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack