arXiv subjects

Zhaoqiang Guo

Publications and source records attributed to Zhaoqiang Guo.

9 recordsLinked to original sources

arXiv2026

LLM-based Low-Level Integration Test Generation for Java

Large language models (LLMs) show promise for automated test generation, but most approaches target unit tests with mocked dependencies. Low-level integration testing instead exercises a class with its real, in-project dependencies, exposing faults involving object construction, API call sequences, and component interactions. Generating such tests is challenging because LLMs may lack project-specific knowledge (not knowing) or violate provided constraints (not following). We present IntTestGen, an LLM-based approach that combines context-enriched generation with constraint-enforced fixing. It mines dependency usage patterns from project code to guide test generation, then applies symbol-, protocol-, and iteration-level constraints during repair using a ClassIndex, a Markov typestate model, and experience memory. We evaluate IntTestGen against the state-of-the-art LLM-based baseline PANTA and search-based baseline EvoSuite on Defects4J and Deps4J, a new post-cutoff benchmark of recent Java repositories. Across the two benchmarks, IntTestGen improves line coverage by 19.99 and 22.69 percentage points, branch coverage by 24.90 and 15.78 points, and mutation score by 13.67 and 0.17 points, respectively. It also covers 378 and 55 additional lines of dependency code. Ablation results confirm that all major components contribute to performance.

cs.SE↗

arXiv2026

Call-Chain-Aware LLM-Based Test Generation for Java Projects

Large language models (LLMs) have recently shown strong potential for generating project-level unit tests. However, existing state-of-the-art approaches primarily rely on execution-path information to guide prompt construction, which is often insufficient for complex software systems with rich inter-class dependencies, deep call chains, and intricate object initialization requirements. In this paper, we present CAT, a novel call-chain-aware LLM-based test generation approach that explicitly incorporates call-chain and dependency contexts into prompts through dedicated static analysis. To construct executable, semantically valid test contexts, CAT systematically models caller--callee relationships, object constructors, and third-party dependencies, and supports iterative test fixing when generation failures occur. We evaluate CAT on the widely used Defects4J benchmark and on four real-world GitHub projects released after the LLM's cut-off date. The results show that, across projects in Defects4J, CAT improves line and branch coverage by 18.04% and 21.74%, respectively, over the state-of-the-art approach PANTA, while consistently achieving superior performance on post-cutoff real-world projects. An ablation study further demonstrates the importance of call-chain and dependency contexts in CAT.

cs.SE↗

arXiv2026

A Roadmap on Modern Code Review: Challenges and Opportunities

Over the past decade, modern code review (MCR) has been established as a cornerstone of software quality assurance and a vital channel for knowledge transfer within development teams. However, the manual inspection of increasingly complex systems remains a cognitively demanding and resource-intensive activity, often leading to significant workflow bottlenecks. This paper presents a comprehensive roadmap for the evolution of MCR, consolidating over a decade of research (2013-2025) into a unified taxonomy comprising improvement techniques, which focus on the technical optimization and automation of downstream review tasks, and understanding studies, which investigate the underlying socio-technical mechanisms and empirical phenomena of the review process. By diagnosing the current landscape through a strategic SWOT analysis, we examine the transformative impact of generative AI and identify critical gaps between burgeoning AI capabilities and industrial realities. We envision a future where MCR evolves from a human-driven task into a symbiotic partnership between developers and intelligent systems. Our roadmap charts this course by proposing three pivotal paradigm shifts, Context-Aware Proactivity, Value-Driven Evaluation, and Human-Centric Symbiosis, aiming to guide researchers and practitioners in transforming MCR into an intelligent, inclusive, and strategic asset for the AI-driven future.

cs.SE↗

arXiv2025

Large Language Models for Unit Test Generation: Achievements, Challenges, and Opportunities

Automated unit test generation is critical for software quality but traditional structure-driven methods often lack the semantic understanding required to produce realistic inputs and oracles. Large language models (LLMs) address this limitation by leveraging their extensive data-driven knowledge of code semantics and programming patterns. To analyze the state of the art in this domain, we conducted a systematic literature review of 115 publications published between May 2021 and August 2025. We propose a taxonomy based on the unit test generation lifecycle that divides the process into a generative phase for creating test artifacts and a quality assurance phase for refining them. Our analysis reveals that prompt engineering has emerged as the dominant utilization approach and accounts for 89% of the studies due to its flexibility. We find that iterative validation and repair loops have become the standard mechanism to ensure robust usability by significantly improving compilation and execution pass rates. However, critical challenges remain regarding the weak fault detection capabilities and the lack of standardized benchmarks. We conclude with a roadmap for future research that emphasizes the progression toward autonomous testing agents and hybrid systems combining LLMs with traditional software engineering tools.

cs.SE↗

arXiv2025

PALM: Synergizing Program Analysis and LLMs to Enhance Rust Unit Test Coverage

Unit testing is essential for ensuring software reliability and correctness. Classic Search-Based Software Testing (SBST) methods and concolic execution-based approaches for generating unit tests often fail to achieve high coverage due to difficulties in handling complex program units, such as branching conditions and external dependencies. Recent work has increasingly utilized large language models (LLMs) to generate test cases, improving the quality of test generation by providing better context and correcting errors in the model's output. However, these methods rely on fixed prompts, resulting in relatively low compilation success rates and coverage. This paper presents PALM, an approach that leverages large language models (LLMs) to enhance the generation of high-coverage unit tests. PALM performs program analysis to identify branching conditions within functions, which are then combined into path constraints. These constraints and relevant contextual information are used to construct prompts that guide the LLMs in generating unit tests. We implement the approach and evaluate it in 15 open-source Rust crates. Experimental results show that within just two or three hours, PALM can significantly improve test coverage compared to classic methods, with increases in overall project coverage exceeding 50% in some instances and its generated tests achieving an average coverage of 72.30%, comparable to human effort (70.94%), highlighting the potential of LLMs in automated test generation. We submitted 91 PALM-generated unit tests targeting new code. Of these submissions, 80 were accepted, 5 were rejected, and 6 remain pending review. The results demonstrate the effectiveness of integrating program analysis with AI and open new avenues for future research in automated software testing.

cs.SE↗

arXiv2023

Toward a consistent performance evaluation for defect prediction models

In defect prediction community, many defect prediction models have been proposed and indeed more new models are continuously being developed. However, there is no consensus on how to evaluate the performance of a newly proposed model. In this paper, we aim to propose MATTER, a fraMework towArd a consisTenT pErformance compaRison, which makes model performance directly comparable across different studies. We take three actions to build a consistent evaluation framework for defect prediction models. First, we propose a simple and easy-to-use unsupervised baseline model ONE (glObal baseliNe modEl) to provide "a single point of comparison". Second, we propose using the SQA-effort-aligned threshold setting to make a fair comparison. Third, we suggest reporting the evaluation results in a unified way and provide a set of core performance indicators for this purpose, thus enabling an across-study comparison to attain real progress. The experimental results show that MATTER can serve as an effective framework to support a consistent performance evaluation for defect prediction models and hence can help determine whether a newly proposed defect prediction model is practically useful for practitioners and inform the real progress in the road of defect prediction. Furthermore, when applying MATTER to evaluate the representative defect prediction models proposed in recent years, we find that most of them (if not all) are not superior to the simple baseline model ONE in terms of the SQA-effort awareness prediction performance. This reveals that the real progress in defect prediction has been overestimated. We hence recommend that, in future studies, when any new defect prediction model is proposed, MATTER should be used to evaluate its actual usefulness (on the same benchmark test data sets) to advance scientific progress in defect prediction.

cs.SE↗

arXiv2021

An extensive empirical study of inconsistent labels in multi-version-project defect data sets

The label quality of defect data sets has a direct influence on the reliability of defect prediction models. In this study, for multi-version-project defect data sets, we propose an approach to automatically detecting instances with inconsistent labels (i.e. the phenomena of instances having the same source code but different labels over multiple versions of a software project) and understand their influence on the evaluation and interpretation of defect prediction models. Based on five multi-version-project defect data sets (either widely used or the most up-to-date in the literature) collected by diverse approaches, we find that: (1) most versions in the investigated defect data sets contain inconsistent labels with varying degrees; (2) the existence of inconsistent labels in a training data set may considerably change the prediction performance of a defect prediction model as well as can lead to the identification of substantially different true defective modules; and (3) the importance ranking of independent variables in a defect prediction model can be substantially shifted due to the existence of inconsistent labels. The above findings reveal that inconsistent labels in defect data sets can profoundly change the prediction ability and interpretation of a defect prediction model. Therefore, we strongly suggest that practitioners should detect and exclude inconsistent labels in defect data sets to avoid their potential negative influence on defect prediction models. What is more, it is necessary for researchers to improve existing defect label collection approaches to reduce inconsistent labels. Furthermore, there is a need to re-examine the experimental conclusions of previous studies using multi-version-project defect data sets with a high ratio of inconsistent labels.

cs.SE↗

arXiv2020

Prioritizing documentation effort: Can we do better?

Code documentations are essential for software quality assurance, but due to time or economic pressures, code developers are often unable to write documents for all modules in a project. Recently, a supervised artificial neural network (ANN) approach is proposed to prioritize important modules for documentation effort. However, as a supervised approach, there is a need to use labeled training data to train the prediction model, which may not be easy to obtain in practice. Furthermore, it is unclear whether the ANN approach is generalizable, as it is only evaluated on several small data sets. In this paper, we propose an unsupervised approach based on PageRank to prioritize documentation effort. This approach identifies "important" modules only based on the dependence relationships between modules in a project. As a result, the PageRank approach does not need any training data to build the prediction model. In order to evaluate the effectiveness of the PageRank approach, we use six additional large data sets to conduct the experiments in addition to the same data sets collected from open-source projects as used in prior studies. The experimental results show that the PageRank approach is superior to the state-of-the-art ANN approach in prioritizing important modules for documentation effort. In particular, due to the simplicity and effectiveness, we advocate that the PageRank approach should be used as an easy-to-implement baseline in future research on documentation effort prioritization, and any new approach should be compared with it to demonstrate its effectiveness.

cs.SE↗

arXiv2019

MAT: A simple yet strong baseline for identifying self-admitted technical debt

In the process of software evolution, developers often sacrifice the long-term code quality to satisfy the short-term goals due to specific reasons, which is called technical debt. In particular, self-admitted technical debt (SATD) refers to those that were intentionally introduced and remarked by code comments. Those technical debts reduce the quality of software and increase the cost of subsequent software maintenance. Therefore, it is necessary to find out and resolve these debts in time. Recently, many approaches have been proposed to identify SATD. However, those approaches either have a low accuracy or are complex to implementation in practice. In this paper, we propose a simple unsupervised baseline approach that fuzzily matches task annotation tags (MAT) to identify SATD. MAT does not need any training data to build a prediction model. Instead, MAT only examines whether any of four task tags (i.e. TODO, FIXME, XXX, and HACK) appears in the comments of a target project to identify SATD. In this sense, MAT is a natural baseline approach, which has a good understandability, in SATD identification. In order to evaluate the usefulness of MAT, we use 10 open-source projects to conduct the experiment. The experimental results reveal that MAT has a surprisingly excellent performance for SATD identification compared with the state-of-the-art approaches. As such, we suggest that, in the future SATD identification studies, MAT should be considered as an easy-to-implement baseline to which any new approach should be compared against to demonstrate its usefulness.

cs.SE↗