TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems

Abstract

Agentic systems have been widely studied to automate coding tasks such as bug fixing and feature implementation. As these systems increasingly operate on complex codebases, understanding where and why they fail becomes essential for iterative refinement and operational reliability. Existing automated failure diagnosis approaches leverage \textit{task execution trajectories}, yet they struggle with trajectories produced by repository-level coding agents due to two key properties. First, these trajectories are often long, spanning many execution steps, making it difficult for LLMs to track the causal chain of failure over the execution history. Second, these trajectories are laden with noise, containing substantial low-signal observations such as redundant program structures and verbose code context, which can interfere with LLM reasoning. To address these challenges, we propose \textit{TrajAudit}, an automated failure diagnosis framework specifically for trajectories produced by repository-level coding agents. TrajAudit employs an investigator agent supported by two modules: one reduces failure-irrelevant noisy context through semantic saliency folding, and the other derives preliminary diagnostic guidance from test failure reports as prior knowledge to help LLMs focus on likely failure regions. The investigator agent can further invoke tools to inspect folded content on demand, enabling a focused investigation without losing access to the full trajectory context. We also introduce \textit{RootSE}, a benchmark of 102 real-world instances from repository-level coding tasks, each annotated with the earliest decisive error step and a justification. Experiments on RootSE show that TrajAudit outperforms the strongest baselines by 10.8\% and 21.6\% in exact failure localization accuracy in the with- and without-reference settings, respectively, demonstrating its effectiveness.