AgentPProf: Semantic Profiler for Long Horizon AI Agents
Yusheng Zheng and colleagues adapt systems profiling to agent trajectories with a semantic operation stack, so resource use can be attributed to task intent rather than to code paths.
Ask this paper
Operation stacks replace the call stack. Uniform operations represent all agent activity and stack into a hierarchy, which gives attribution at several granularities without stable code identifiers.
Recursive operation segmentation finds task boundaries. An agent's task occupies a contiguous span that decomposes into subtasks, so trajectories are split recursively at those boundaries.
0.764 B-cubed F1 against human annotations. On CodeTraceBench, which is the measurement that the automatic segmentation matches how a person would divide the trajectory.
Problem localization MAP rises up to 56 percent. Across three benchmarks, and profiles are pprof-compatible so existing flame graph tooling works on them.
Abstract
AI agents increasingly orchestrate long-running activities with users, tools, and system resources for days and weeks. To improve agent quality, safety, and cost efficiency, developers need to determine where failures happen, what triggers unsafe effects, and which tasks consume the most budget, then optimize those tasks. In systems software, profiling answers similar questions by aggregating resource consumption and attributing it to responsible code paths to identify hotspots. Yet existing agent observability tools focus on per-execution debugging and tracing rather than cross-run, long term profiling, making these questions difficult to answer at scale. Agent observability needs profiling, not only debugging, but profiling agents is challenging: the responsible entities are task intent like diagnose authentication, compare branches rather than code paths, and lack stable identifiers for aggregation. We propose a semantic operation stack model that adapts profiling to agent trajectories. Uniform operations represent all activities, and operation stacks replace the runtime call stack, enabling hierarchical attribution at different granularities. We observe that an agent's task occupies a contiguous span and decomposes into subtasks, so we introduce recursive operation segmentation, which recursively splits trajectories at task boundaries. AgentPProf is a profiler that aggregates agent trajectories into pprof-compatible profiles, enabling flame graph visualization and analysis. AgentPProf reaches 0.764 $B^3$ F1 against human annotations on CodeTraceBench. On three problem-localization benchmarks, the profile raises MAP by up to 56%, demonstrating that it effectively attributes resources, locates problems, and helps optimize token cost at practical profiling cost. AgentPProf is available at https://github.com/eunomia-bpf/agentsight.