arXiv · 2611.10837
Patch correctness in automated program repair: A five-dimensional evidence framework and cross-benchmark audit from classical repair to coding agents
Abstract
Tests guide automated program repair and determine which candidate patches are reported as successful. Repository-level coding agents extend this feedback loop by generating tests and revising code after observing execution results. This technical report introduces a five-dimensional evidence profile and reporting protocol to audit how test observations support patch-correctness claims across classical repair systems and coding agents. We assemble an audit dataset of 39 benchmark and evaluation artifacts: 32 primary empirical studies, one institutional technical audit, and six prior taxonomic baselines. We code the 33 primary contributions along five dimensions: test provenance and exposure, behavioral scope, oracle basis and dependence, execution integrity, and label adjudication. The central distinction separates checks visible during repair search from evidence reserved for final assessment. Four worked comparative audits show how changed candidate populations, oracle rules, or execution budgets alter reported detection rates. Additional tests can expose missed faults while rejecting valid alternatives or reducing correct-repair yield under a fixed budget. Repository-level audits separate issue reproduction from candidate-patch discrimination and distinguish automated discrepancy detection from requirements-based adjudication. An evidence matrix links each coded decision to supporting passages and records shared benchmark ancestry. We specify a reporting protocol for repair benchmarks and maintenance pipelines that retains uncertain judgments and records validation cost alongside fault detection and valid-patch retention.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Shuai Wu, Yanna Feng, Yufang Li, Xue Li, Zhijun Wang, Bolun Liu, Weilin Cai. 2026-11-07. Patch correctness in automated program repair: A five-dimensional evidence framework and cross-benchmark audit from classical repair to coding agents. https://searcharxiv.com/abs/2611.10837
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.