Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.08651

A Case Study in Assuring AI-Written Software

Lindsey Ferris·Sierra Bonilla
View original source ↗Read source document ↗Export metadata ↓

Abstract

Software-engineering agents can enable people without formal software training to build systems they could not otherwise implement and simultaneously can produce more code than even experts can meaningfully inspect. In both cases, exhaustive code review is not reliable as the sole basis for human control. We report a case study of a production healthcare platform built through coding agents and governed by an operator without formal software-engineering training. Over time, its workflow grew into a human-led meta-agent system where one agent wrote code, other agents supervised and reviewed it, and project rules carried lessons forward. The operator found that tests, monitors and reviewing agents used to supervise the system were fallible. Some monitors measured proxies rather than outcomes, some audits failed silently, missing checks disappeared from reported results and one automated repair caused operational disruption. In this case, human control depended on keeping the intended outcome, the evidence used to judge it, the agents' permissions and the final human decision were all tied to the same underlying objective.

Explore related subjects

cs.SEcs.AIcs.MA

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Lindsey Ferris, Sierra Bonilla. 2026-10-06. A Case Study in Assuring AI-Written Software. https://arxiv.org/abs/2610.08651

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Theory building in software engineering: Operationalization

This work is part of a research project whose ultimate goal is to systematize a procedure for constructing theories in software engineering. The proposed methodology involves four phases: conceptualization, operationalization, testing, and application. In previous work, we described the conceptualization process. This paper presents a set of procedures for systematizing the operationalization phase in theory building. Specifically, it translates the concepts and propositions obtained from the previous conceptualization into constructs and empirically testable hypotheses. The operationalization phase is structured across three distinct dimensions: the practical procedural steps, their strict formal mathematical specification, and their evaluation through an illustrative empirical case study. Grounded in critical realism, we operationalize the qualitatively derived concepts and propositions into explicit constructs and logical hypotheses. It is important to emphasize that a novel strategy has been designed that reduces the number of hypotheses resulting from this process. In the use case described, more than 30% of the initial hypotheses have been eliminated because they are redundant. This paper is a pioneering contribution in offering comprehensive guidelines for theory operationalization using logical implication and describes a strategy based on graph theory that tackles the problem of hypothesis explosion and maintains the principle of parsimony.

cs.SE↗
arXiv2026

ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization

Large language models (LLMs) can translate natural-language problem descriptions into optimization code, but the code is prone to silent failures: it executes and returns a solver-feasible solution while encoding a semantically incorrect formulation. On compositional problems, the resulting feasibility-correctness gap reaches 90 percentage points. We introduce ReLoop, which combines two mechanisms. Structured generation decomposes code production into a four-stage reasoning chain (understand, formalize, synthesize, verify) to reduce formulation errors during generation. Behavioral verification detects the errors that remain by testing whether the formulation responds correctly to solver-based parameter perturbation, a signal that comes from the solver rather than from LLM self-review and requires no ground truth. The two mechanisms address different error structures: structured generation gives the largest gain on compositional problems (+8.5pp accuracy on RetailOpt-190 with Claude Opus 4.6), and behavioral verification gives its largest gain on localized defects (+4.4pp on MAMO-ComplexLP). With diagnostic execution recovery, ReLoop reaches 100% executable code on Claude Opus 4.6, and relative to direct generation it raises or preserves every reported metric of the three chat-tuned foundation models on all three benchmarks. For the narrowly fine-tuned SFT model we test, the chain-of-thought prompt conflicts with its learned output format and lowers its accuracy on MAMO-ComplexLP; we document and analyze this interaction. We release RetailOpt-190, 190 compositional retail optimization scenarios in which several constraints interact.

cs.SE↗
arXiv2026

Finding Memory Leaks in C/C++ Programs via Neuro-Symbolic Augmented Static Analysis

Memory leaks remain prevalent in real-world C/C++ software. Static analyzers such as CodeQL provide scalable program analysis but frequently miss such bugs because they cannot recognize project-specific custom memory-management functions and lack path-sensitive control-flow modeling. We present MemHint, a neuro-symbolic pipeline that addresses both limitations by combining LLMs' semantic understanding of code with Z3-based symbolic reasoning. MemHint parses the target codebase and applies an LLM to classify each function as a memory allocator, deallocator, or neither, producing function summaries that record which argument or return value carries memory ownership, extending the analyzer's built-in knowledge beyond standard primitives such as malloc and free. A Z3-based validation step checks each summary against the function's control-flow graph, discarding those whose claimed memory operation is unreachable on any feasible path. The validated summaries are injected into CodeQL and Infer via their respective extension mechanisms. Z3 path feasibility filtering then eliminates warnings on infeasible paths, and a final LLM-based validation step confirms whether each remaining warning is a genuine bug. On eight real-world C/C++ projects totaling over 3.6M lines of code, MemHint detects 54 unique memory leaks, all confirmed or fixed, at approximately $1.7 per detected bug, compared to 19 by vanilla CodeQL and 3 by vanilla Infer.

cs.SE↗