SEARCH · Searcharxiv

Search Searcharxiv

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

arXiv2026

A Governance Methodology Layer for AI-Assisted Software Development: Defect Taxonomy, Controlled Ablation, and a Test of Process-Over-Capability

Autonomous coding agents produce output that passes syntactic checks -- compilation, type safety, CI -- at high velocity. Yet syntactic correctness does not imply semantic correctness: design boundaries, security invariants, and maintainability contracts remain structurally invisible to automated pipelines. This paper makes four contributions. First, we present a defect-class taxonomy grounded in five AI agent permission and governance modules, separating defects structurally detectable by static analysis from those requiring semantic review. Second, we describe a runtime-decoupled governance gate -- a file-based protocol that reads generator output and emits a structured verdict without API coupling, hence portable across generators. Third, we formalize methodology-as-code: expressing a verification protocol as a version-controlled, executable artifact whose two layers separate portable methodology from host automation. Fourth, we report a controlled ablation experiment (E-ablation, N=5 artifacts, 8-item independent ground truth) comparing harness-structured review against a token-matched unstructured prompt. The structured condition records 62% lenient recall against 50%, with 25% strict against 0%. A severity-grade differential reported earlier does not survive blind re-grading and is withdrawn (Sec. 6.6). An independent-session re-test with blind scoring does not replicate that contrast: the conditions differ by one strict hit in 24 (6/24 against 5/24), the structured aggregate again 25% and the unstructured 0% not recurring (Sec. 6.7). Both conditions miss document-quality defects identified by a human QA reviewer, indicating complementarity between structured AI review and human process inspection. The re-test does not distinguish structured review from a detailed unstructured prompt here, so process design as the dominant factor remains a hypothesis, not a result of this paper.

cs.SE↗

arXiv2026

An Empirical Analysis of CodeQL False Positives and Query Refinements for Java Vulnerabilities

Static application security testing (SAST) tools help developers find vulnerabilities before deployment, but false positives create substantial triage effort. We study whether CodeQL false positives in Java security analysis form recurring, explainable patterns that can be reduced by refining the analysis. We run CodeQL's Java security query suite on 167 CVE instances from 110 projects, focusing on the ten queries with the highest false positive rates. We manually review 500 sampled false positive paths and locations and construct a source-level taxonomy. The five categories are Missed Path Constraint or Sanitization (36.6%), Benign Execution Context (29.4%), Missing Trust Boundary Modeling (27.6%), Imprecise Concurrency Modeling (5%), and Imprecise Sink Modeling (1.4%). Guided by these findings, we implement CodeQL refinements that detect and filter recurring false positive patterns at the query level. The refinements remove 81.8% of reviewed false positives. Across the full selected-query dataset, they remove 15.8% of reported paths and locations while retaining 7 of 8 true positives. This shows that many false positives can be reduced in the analysis, although fixed refinements often depend on project-specific context. To address this generalization gap, we evaluate whether agentic coding tools can adapt refinement patterns to new projects. Given our patterns as templates, the two tools succeed on 56% and 62% of tasks, with query compile-pass rates above 90%. Without this guidance, both succeed on only 28%, while compile rates fall to 30-36%. These results support a refinement-oriented SAST workflow in which recurring false positives are modeled in CodeQL queries and automatically adapted to different project contexts, reducing repeated triage.

cs.SE↗

arXiv2026

Understanding the (In)Security of Vibe-Coded Applications

Recent advances in large language models (LLMs) have enabled vibe coding, an emerging software development paradigm in which users create applications primarily through natural-language interactions with AI agents. Due to its low barrier to entry, vibe coding is rapidly gaining adoption in practice. Unlike conventional AI-assisted programming, where developers remain responsible for implementation and code review, vibe coding delegates a substantial portion of the development process to AI systems. This shift raises a fundamental question: how (in)secure are applications developed through vibe coding? In this paper, we conduct a systematic study of the security of real-world vibe-coded applications. We collect 9,041 open-source applications developed using popular AI agents (Claude Code and Lovable), and audit 200 publicly deployed applications, uncovering 1,186 vulnerabilities. Our study of these applications and vulnerabilities reveals several key findings: (1) insecurity is the norm rather than the exception: 91.0\% of audited applications contain at least one vulnerability, and 65.77\% of identified vulnerabilities are rated Critical or High severity, concentrated in broken access control, injection, and authentication failures; (2) these vulnerabilities are traceable to eight recurring failure modes rooted in three systematic limitations of AI agents: memory defects, objective defects, and knowledge defects; and (3) while improved agent harness and prompting strategies can reduce the incidence of vulnerabilities, they do not eliminate the underlying security risks. Overall, our study provides an empirical understanding of the security landscape of vibe-coded applications and lays the groundwork for addressing the security risks in the growing delegation of software development to AI systems.

cs.CR↗

arXiv2026

HoarePrompt: Structural Reasoning About Program Correctness in Natural Language

While software requirements are often expressed in natural language, verifying the correctness of a program against such requirements is a hard and underexplored problem. Large language models (LLMs) are promising candidates for addressing this challenge, however our experience shows that they are ineffective in this task, often failing to detect even straightforward bugs. To address this gap, we introduce HoarePrompt, a novel approach that adapts fundamental ideas from program verification to natural language artifacts. Inspired from the strongest postcondition calculus, HoarePrompt employs a systematic, step-by-step process in which an LLM generates natural language descriptions of reachable program states at various code points. To manage loops, we propose few-shot-driven k-induction, an adaptation of the k-induction method widely used in model checking. Once program states are described, HoarePrompt leverages the LLM to assess whether the program, annotated with these state descriptions, conforms to the natural language requirements. For evaluating the quality of classifiers of program correctness with respect to natural language requirements, we constructed CoCoClaNeL, a challenging dataset of solutions to programming competition problems. Our experiments show that HoarePrompt improves the MCC by 61% compared to directly using Zero-shot-CoT prompts for correctness classification. Furthermore, HoarePrompt outperforms a classifier that assesses correctness via LLM-based test generation by an MCC increase of 106%. The inductive reasoning mechanism contributes a 26% boost to MCC, underscoring its effectiveness in managing loops.

cs.SE↗

arXiv2026

Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation

Large language models (LLMs) can generate executable code from natural language descriptions, but the resulting programs frequently contain bugs due to hallucinations. In the absence of formal specifications, existing approaches attempt to assess correctness using LLM-generated proxies such as tests or auto-formalized specifications. However, these proxies are produced by the same imperfect models and thus often corroborate rather than catch errors, especially when the model exhibits correlated errors. We introduce semantic triangulation, a theory-grounded framework that decorrelates model errors by transforming the original problem into a dissociative variant - one likely requiring a fundamentally different algorithm - and checks consistency between independently sampled solutions to both problems. We identify theoretical requirements for this framework, and we prove that under a formal model of LLM hallucinations, these properties confer higher confidence in program correctness. We instantiate the framework through four concrete triangulation methods based on problem inversion, decomposition, and solution enumeration. Evaluated on LiveCodeBench and CodeElo across GPT-4o, DeepSeek-V3, and Gemini 2.5 Flash, our tool increases the probability of selecting a correct program by 16% over baselines (test generation, metamorphic testing, and auto-formalized specifications) and achieves 7% higher reliability and 7% higher F1 score in selection-or-abstention scenarios, while being the only method that consistently handles inexact problems admitting multiple valid solutions.

cs.SE↗

arXiv2026

Towards Requirements Engineering for GenAI-Enabled Software: Bridging Responsibility Gaps through Human Oversight Requirements

Context: Responsibility gaps, long-recognized challenges in socio-technical systems where accountability becomes diffuse or ambiguous, have become increasingly pronounced in GenAI-enabled software. This study seeks to establish a coherent perspective for a systematic analysis of responsibility gaps from a human oversight requirements standpoint, encompassing how these responsibility gaps should be conceptualized, identified, and represented throughout the requirements engineering process. Methods: The proposed design methodology is structured across three analytical layers. At the conceptualization layer, it establishes a conceptual framing that defines the key elements of responsibility across the human and system dimensions and explains how potential responsibility gaps emerge from their interactions. At the methodological layer, it introduces a deductive pipeline for identifying responsibility gaps by analyzing interactions between these dimensions and deriving corresponding oversight requirements within established requirements engineering frameworks. At the artifact layer, it formalizes the results in a Deductive Backbone Table, a reusable representation that traces the reasoning path from responsibility gaps identification to human oversight requirements derivation. Results: A user study compared the proposed methodology with a baseline goal-oriented requirements engineering approach. Participant-perception results indicated generally favorable views of the enhanced approach across the evaluated dimensions, while expert ratings of the generated requirement artifacts showed higher mean scores for six of the seven artifact-quality items, with the strongest descriptive improvements observed for consistency and representational traceability.

cs.SE↗

arXiv2026

A Large-Scale Dynamic Characterization of Flaky Tests in Quantum Software: The Qiskit Terra Case Study

Flaky tests, tests that pass or fail nondeterministically without changes to code or environment, pose a threat to software reliability. While classical software engineering has developed a body of techniques to study flakiness, evidence for quantum software remains limited. Prior work relies on static analysis or small sets of manually reported incidents, leaving open questions about their prevalence, characteristics, and detectability. This paper presents a large-scale dynamic characterization of flaky tests in quantum software through a longitudinal case study of the Qiskit Terra core library. We executed the Qiskit Terra test suite 10,000 times across 23 releases in controlled environments. For each release, we measured test-outcome variability, identified flaky tests, estimated empirical failure probabilities, analyzed recurrence across versions, used Wilson confidence intervals to quantify baseline rerun budgets, and mapped flaky tests to Terra subcomponents. Across 27,026 qualified test identifiers, we identified 62 unique flaky tests. Although flakiness rates were low (0-0.17%), recurrence was substantial: 52 of 62 flaky tests (83.87%) reappeared in multiple releases, while only 10 tests (16.13%) were confined to a single release. Failure probabilities spanned several orders of magnitude, with a median of $9 \times 10^{-4}$ and 34 tests (54.84%) at or below $10^{-3}$, implying that thousands to tens of thousands of executions may be required for confident detection. These results show that flakiness observed in the studied Terra releases is rare under our controlled protocol but difficult to detect under typical continuous integration budgets. To support future research and replication, we release a dataset of per-test execution outcomes.

cs.SE↗

arXiv2026

SWE-Tester: Training Open-Source LLMs for Issue Reproduction in Real-World Repositories

Software testing is crucial for ensuring the correctness and reliability of software systems. Automated generation of issue reproduction tests from natural language issue descriptions enhances developer productivity by simplifying root cause analysis, promotes test-driven development -- "test first, write code later", and can be used for improving the effectiveness of automated issue resolution systems like coding agents. Existing methods proposed for this task predominantly rely on closed-source LLMs, with limited exploration of open models. To address this, we propose SWE-Tester -- a novel pipeline for training open-source LLMs to generate issue reproduction tests. First, we curate a high-quality training dataset of 41K instances from 2.6K open-source GitHub repositories and use it to train LLMs of varying sizes and families. The fine-tuned models achieve absolute improvements of up to 10\% in success rate and 21\% in change coverage on SWT-Bench Verified. Further analysis shows consistent improvements with increased inference-time compute, more data, and larger models. These results highlight the effectiveness of our framework for advancing open-source LLMs in this domain.

cs.SE↗

arXiv2026

Defusing Logic Bombs in Symbolic Execution with LLM-Generated Ghost Code

Symbolic execution is a powerful program analysis technique, but its effectiveness is fundamentally limited by solver-hostile program fragments, complex numerical reasoning, and unbounded heap structures. Recent work proposed replacing constraint solvers with large language models (LLMs) to bypass these limitations, but such approaches struggle to analyze real-world codebases, where deep execution paths require globally consistent reasoning across many interacting constraints. We present Gordian, a hybrid symbolic execution framework that uses LLMs selectively to generate lightweight ghost code that aids an SMT solver in handling solver-hostile code fragments, while preserving its precise, global reasoning capability. In particular, we propose three types of ghost code: (1) inversion of difficult code fragments with iterative bidirectional constraint propagation, (2) modeling via solver-friendly surrogates while preserving relevant behavior, and (3) semantic partitioning of unbounded heap spaces. We implemented Gordian on top of the KLEE symbolic execution engine and evaluated it on synthetic "logic bombs" capturing distinct symbolic reasoning challenges, a popular mathematical library FDLibM, and four structured-input programs (libexpat, jq, bc and libyaml). Across benchmarks, Gordian improves coverage by 28.5-115.2% over traditional symbolic execution baseline and by 74.1-189.8% over LLM-based symbolic execution baselines, while reducing LLM token usage by an average of 91-96%. This highlights the practicality and effectiveness of this approach in real-world settings.

cs.PL↗

arXiv2026

Complexity and Scale in AI-Assisted Workflow Management: A Federated Learning Case Study

Federated learning over medical images is a demanding workflow application. Each round fans out across parallel client jobs and converges on an aggregation step that feeds the next round. At scale this yields 101 sub-workflows and 2,679 jobs on GPUs at four sites, which takes an expert months to build, mostly on workflow mechanics rather than science. We ask how far AI assistance can automate such workflows. An LLM agent, grounded in a released plugin of Pegasus-specific skills, first produces a reviewable specification of checkable constraints and acceptance criteria, then generates the executable workflow. A validation loop repairs runtime failures, checks code against those constraints, and regenerates the implementation from the specification alone. We evaluate three LLM agents, report end-to-end runs on the FABRIC testbed, and show how conformance checking against the specification caught three silent errors that failure-driven debugging missed, including one that trained 1,700 jobs on random tensors.

cs.SE↗

arXiv2026

Builder, Defender, Breaker: Measurable Independence and Bounded Autonomy When Generative Models Build, Defend and Test Software

Generative models now write application code, harden and monitor it, and probe it for exploitable flaws, so that one family of models increasingly plays builder, defender and breaker at once. The prevailing view treats full autonomy as the natural end point of assistance. This article argues for a narrower and more defensible position than a blanket requirement for human oversight. We define the shared generative substrate as the set of upstream dependencies (training corpus, model family, alignment procedure, vendor, toolchain) that induce correlated errors, and we define independence as a measurable property of pairs of lifecycle roles rather than an attribute of any participant. A coincident-failure model in the tradition of Eckhardt and Lee shows why organisational independence, the proxy on which verification standards have relied, no longer implies statistical independence once roles share a substrate; it also shows that heterogeneous models, deterministic analysers and formal verification can restore independence for the fault classes they cover. What remains for humans is specific: authority over the specification against which every machine oracle is judged, accountability that current governance instruments do not permit to be delegated, and last-resort authority to halt. We translate this into an operational framework with five autonomy levels, three distinct human roles and five decision criteria (consequence, reversibility, time-criticality, verifiability and adversarial exposure), apply it to build, defend and test actions, and specify how independence and oversight effectiveness can be measured. The central claims are stated as testable hypotheses with a protocol for refuting them.

cs.CR↗

arXiv2026

TianoForge: An Automated Bug Triage Approach for the TianoCore UEFI Firmware Development Community

We propose a novel approach to bug triage in the TianoCore open-source UEFI firmware development ecosystem. This integrated approach, called TianoForge, deploys the state of the art in artificial intelligence, specifically machine learning, to enable automated bug triage. This includes invalid bug report detection, duplicate bug report detection, bug report prioritization, and bug report assignment. We use various Generative Pretrained Transformer (GPT) Large Language Models (LLMs) with and without Retrieval Augmented Generation (RAG) to automate these tasks. Given the crucial role of bug triage in software maintenance and the huge number of untriaged issues in the TianoCore community, in particular, their primary project, EDK II, we expect a significant impact on the efficiency of TianoCore software maintenance processes, primarily bug triage and resolution. Our experimental study shows that TianoForge reduces the average bug triage time from around 11 days to approximately 7 minutes, which is a 99.95% reduction.

cs.SE↗

arXiv2026

An Empirical Study of the TianoCore Community

We investigate the software security and maintenance practices adopted by stakeholders in the TianoCore community and identify opportunities to improve firmware development workflows. We conduct a survey and a limited interview study with participants representing independent firmware vendors, original equipment manufacturers, security experts, firmware developers, and academic researchers. This open-source development community maintains a reference implementation for the core of the UEFI firmware. We highlight important gaps in the current state of firmware development within the TianoCore ecosystem and identify key areas in which improved security practices, greater adoption of memory-safe technologies, and increased automation of manual processes could strengthen the maintenance and security of the UEFI firmware.

cs.SE↗

arXiv2026

Enhancing Bug Report Templates in the TianoCore UEFI Firmware Development Community

We propose enhancing the bug report templates in the GitHub Issues issue tracking system used by the TianoCore open-source community with the aim of improving the bug triage and resolution process. We analyze the bug repository data and find patterns of information that are useful for bug triage and fixing. However, some of them are only occasionally included in the free-form text of bug reports. Therefore, we propose adding a few new fields to the existing TianoCore bug report template. In this study, we focus on the key TianoCore project, EDK II, which constitutes the core of the UEFI firmware across various firmware vendors and original equipment manufacturers. This study is currently a work-in-progress. So far, we have interviewed a few developers to obtain their feedback and adjust the proposed approach. We are planning more interviews with the TianoCore community to conduct A/B tests and validate our approach to achieve effective and efficient bug triage and resolution.

cs.SE↗

arXiv2026

SPIDER4TianoCore: Enhancing Patch-Propagation for the TianoCore UEFI Firmware Development Ecosystem

We propose and demonstrate SPIDER4TianoCore, a packaged Python command-line tool that provides integration-stage patch-status evidence for the TianoCore/UEFI firmware supply chain. Given an upstream pre-patch and post-patch pair and prepared downstream targets, the tool reports Vulnerable, Already Patched, Not Applicable, or Uncertain with supporting evidence for maintainer review. Our work is inspired by SPIDER's patch-propagation framing, but SPIDER4TianoCore does not itself prove that a patch is safe to propagate. We evaluate the engine on 20 prepared target/CVE pairs from eight public downstream EDK II repositories and two CVEs. The analyzers produce 10 high-confidence pre-patch matches and four high-confidence post-patch matches, conservatively abstain on six targets, and make no confidently wrong classifications relative to the recorded manual patch-state labels. These preliminary results demonstrate reproducible evidence generation for prepared targets rather than general downstream accuracy.

cs.SE↗

arXiv2026

Vectorizer: Vectorizing NumPy Programs with Shape-Guided Rewrite

NumPy is a widely used Python library for numerical scientific computing, known for its declarative APIs and its optimized implementations. However, writing efficient NumPy programs, which often entails using vectorized array operations instead of explicit Python loops, may not be straightforward. This can be difficult for programmers who are accustomed to imperative array traversal, especially when vectorized API invocations require careful reasoning about shapes, broadcasting, and advanced indexing. This paper presents a rewrite-based approach for vectorizing Numpy programs with explicit loops over array data. Our approach vectorizes loops from the inside out, using array shapes and dataflow analysis to guide a source-to-source transformation that replaces loop bodies with vectorized statements. Following a set of rewrite rules that are correct by construction, our approach is consistently fast. We have implemented the approach as a tool called Vectorizer and evaluated it on 150 benchmarks collected from prior work and Stack Overflow. The evaluation shows that Vectorizer vectorizes 142 of the 150 benchmarks directly and 2 more after minor changes to the original benchmarks, with only 0.53 seconds on average to rewrite each one. The resulting programs are, on average, 74.83x faster than the original loop-based implementations.

cs.CL↗

arXiv2026

Qiushi Engine on AstaBench E2E-Bench-Hard

This report analyzes Qiushi Engine v0.8 across all 40 test tasks in AstaBench E2E-Bench-Hard, a benchmark that requires autonomous agents to carry a research question through experimental design, code implementation, actual execution, result analysis, and report delivery. Qiushi Engine is model-configurable; this evaluation selected DeepSeek deepseek-v4pro-preview as the model backend. The official AstaBench leaderboard records a score of 0.816 and an average benchmark cost of USD 15.209 per task, while the full-precision local recomputation is $81.59 \pm 1.87$. Four tasks satisfied every rubric item, yielding a full-task completion rate of 4/40 = 10% -- 7 percentage points above, and about 3.3 times, the approximately 3% best rate reported for AstaBench's official agents. Across 507 required rubric items, 416 were satisfied (82.1%). Official scoring archives and 40 Meta-Trace records show sustained production and verification of reports, code, and experimental artifacts; the principal gaps lie in repeated runs, external dependencies, specified metrics, and ablation studies. The report explains the benchmark, system workflow, aggregate results, representative cases, and limits of interpretation.

cs.AI↗

arXiv2026

Agent ATO: Visualizing Agent Interaction Timelines from Logs

AI coding agents are becoming part of developers' workflows, but their behavior is difficult to understand from final code changes alone. During a task, agents interact with software repositories through sequences of actions such as searching for files, reading code, editing programs, and running tests or build commands. These interactions, together with token usage, are often recorded in console logs, but raw logs are difficult for developers to inspect. In this paper, we propose Agent ATO (Agentic Trajectory Observer), a tool for visualizing AI coding agent interaction timelines from console logs. Agent ATO extracts agent interactions, classifies them by command or tool type, and visualizes them as timelines. In addition to an all-interaction timeline, Agent ATO provides filtered timelines that emphasize file discovery, file reading, file editing, and execution while preserving surrounding context. We illustrate how Agent ATO may help developers inspect and compare agent actions using selected runs from two repair tasks. Future work will apply Agent ATO to more agents, tasks, and development environments, and will evaluate whether it reduces the effort needed to compare trajectories.

cs.SE↗