arXiv subjects

Peng Liang

Publications and source records attributed to Peng Liang.

At least 73 records · Page 4Linked to original sources

arXiv2025

Demystifying Issues, Causes and Solutions in LLM Open-Source Projects

With the advancements of Large Language Models (LLMs), an increasing number of open-source software projects are using LLMs as their core functional component. Although research and practice on LLMs are capturing considerable interest, no dedicated studies explored the challenges faced by practitioners of LLM open-source projects, the causes of these challenges, and potential solutions. To fill this research gap, we conducted an empirical study to understand the issues that practitioners encounter when developing and using LLM open-source software, the possible causes of these issues, and potential solutions. We collected all closed issues from 15 LLM open-source projects and labelled issues that met our requirements. We then randomly selected 994 issues from the labelled issues as the sample for data extraction and analysis to understand the prevalent issues, their underlying causes, and potential solutions. Our study results show that (1) Model Issue is the most common issue faced by practitioners, (2) Model Problem, Configuration and Connection Problem, and Feature and Method Problem are identified as the most frequent causes of the issues, and (3) Optimize Model is the predominant solution to the issues. Based on the study results, we provide implications for practitioners and researchers of LLM open-source projects.

cs.SE↗

arXiv2025

Knowledge-Based Multi-Agent Framework for Automated Software Architecture Design

Architecture design is a critical step in software development. However, creating a high-quality architecture is often costly due to the significant need for human expertise and manual effort. Recently, agents built upon Large Language Models (LLMs) have achieved remarkable success in various software engineering tasks. Despite this progress, the use of agents to automate the architecture design process remains largely unexplored. To address this gap, we envision a Knowledge-based Multi-Agent Architecture Design (MAAD) framework. MAAD uses agents to simulate human roles in the traditional software architecture design process, thereby automating the design process. To empower these agents, MAAD incorporates knowledge extracted from three key sources: 1) existing system designs, 2) authoritative literature, and 3) architecture experts. By envisioning the MAAD framework, we aim to advance the full automation of application-level system development.

cs.SE↗

arXiv2025

Unveiling Security Weaknesses in Autonomous Driving Systems: An In-Depth Empirical Study

The advent of Autonomous Driving Systems (ADS) has marked a significant shift towards intelligent transportation, with implications for public safety and traffic efficiency. While these systems integrate a variety of technologies and offer numerous benefits, their security is paramount, as vulnerabilities can have severe consequences for safety and trust. This study aims to systematically investigate potential security weaknesses in the codebases of prominent open-source ADS projects using CodeQL, a static code analysis tool. The goal is to identify common vulnerabilities, their distribution and persistence across versions to enhance the security of ADS. We selected three representative open-source ADS projects, Autoware, AirSim, and Apollo, based on their high GitHub star counts and Level 4 autonomous driving capabilities. Using CodeQL, we analyzed multiple versions of these projects to identify vulnerabilities, focusing on CWE categories such as CWE-190 (Integer Overflow or Wraparound) and CWE-20 (Improper Input Validation). We also tracked the lifecycle of these vulnerabilities across software versions. This approach allows us to systematically analyze vulnerabilities in projects, which has not been extensively explored in previous ADS research. Our analysis revealed that specific CWE categories, particularly CWE-190 (59.6%) and CWE-20 (16.1%), were prevalent across the selected ADS projects. These vulnerabilities often persisted for over six months, spanning multiple version iterations. The empirical assessment showed a direct link between the severity of these vulnerabilities and their tangible effects on ADS performance. These security issues among ADS still remain to be resolved. Our findings highlight the need for integrating static code analysis into ADS development to detect and mitigate common vulnerabilities.

cs.SE↗

arXiv2025

Exploring Data Management Challenges and Solutions in Agile Software Development: A Literature Review and Practitioner Survey

Context: Managing data related to a software product and its development poses significant challenges for software projects and agile development teams. These include integrating data from diverse sources and ensuring data quality amidst continuous change and adaptation. Objective: The paper systematically explores data management challenges and potential solutions in agile projects, aiming to provide insights into data management challenges and solutions for both researchers and practitioners. Method: We employed a mixed-methods approach, including a systematic literature review (SLR) to understand the state-of-research followed by a survey with practitioners to reflect on the state-of-practice. The SLR reviewed 45 studies, identifying and categorizing data management aspects along with their associated challenges and solutions. The practitioner survey captured practical experiences and solutions from 32 industry practitioners who were significantly involved in data management to complement the findings from the SLR. Results: Our findings identified major data management challenges in practice, such as managing data integration processes, capturing diverse data, automating data collection, and meeting real-time analysis requirements. To address the challenges, solutions such as automation tools, decentralized data management practices, and ontology-based approaches have been identified. The solutions enhance data integration, improve data quality, and enable real-time decision-making by providing flexible frameworks tailored to agile project needs. Conclusion: The study pinpointed significant challenges and actionable solutions in data management for agile software development. Our findings provide practical implications for practitioners and researchers, emphasizing the development of effective data management practices and tools to address those challenges and improve project success.

cs.SE↗

arXiv2025

Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study

Modern code generation tools utilizing AI models like Large Language Models (LLMs) have gained increased popularity due to their ability to produce functional code. However, their usage presents security challenges, often resulting in insecure code merging into the code base. Thus, evaluating the quality of generated code, especially its security, is crucial. While prior research explored various aspects of code generation, the focus on security has been limited, mostly examining code produced in controlled environments rather than open source development scenarios. To address this gap, we conducted an empirical study, analyzing code snippets generated by GitHub Copilot and two other AI code generation tools (i.e., CodeWhisperer and Codeium) from GitHub projects. Our analysis identified 733 snippets, revealing a high likelihood of security weaknesses, with 29.5% of Python and 24.2% of JavaScript snippets affected. These issues span 43 Common Weakness Enumeration (CWE) categories, including significant ones like CWE-330: Use of Insufficiently Random Values, CWE-94: Improper Control of Generation of Code, and CWE-79: Cross-site Scripting. Notably, eight of those CWEs are among the 2023 CWE Top-25, highlighting their severity. We further examined using Copilot Chat to fix security issues in Copilot-generated code by providing Copilot Chat with warning messages from the static analysis tools, and up to 55.5% of the security issues can be fixed. We finally provide the suggestions for mitigating security issues in generated code.

cs.SE↗

arXiv2025

MPLinker: Multi-template Prompt-tuning with Adversarial Training for Issue-commit Link Recovery

In recent years, the pre-training, prompting and prediction paradigm, known as prompt-tuning, has achieved significant success in Natural Language Processing (NLP). Issue-commit Link Recovery (ILR) in Software Traceability (ST) plays an important role in improving the reliability, quality, and security of software systems. The current ILR methods convert the ILR into a classification task using pre-trained language models (PLMs) and dedicated neural networks. these methods do not fully utilize the semantic information embedded in PLMs, resulting in not achieving acceptable performance. To address this limitation, we introduce a novel paradigm: Multi-template Prompt-tuning with adversarial training for issue-commit Link recovery (MPLinker). MPLinker redefines the ILR task as a cloze task via template-based prompt-tuning and incorporates adversarial training to enhance model generalization and reduce overfitting. We evaluated MPLinker on six open-source projects using a comprehensive set of performance metrics. The experiment results demonstrate that MPLinker achieves an average F1-score of 96.10%, Precision of 96.49%, Recall of 95.92%, MCC of 94.04%, AUC of 96.05%, and ACC of 98.15%, significantly outperforming existing state-of-the-art methods. Overall, MPLinker improves the performance and generalization of ILR models, and introduces innovative concepts and methods for ILR. The replication package for MPLinker is available at https://github.com/WTU-intelligent-software-development/MPLinker

cs.SE↗

arXiv2024

Automated Detection of Inter-Language Design Smells in Multi-Language Deep Learning Frameworks

Nowadays, most DL frameworks (DLFs) use multilingual programming of Python and C/C++, facilitating the flexibility and performance of the DLF. However, inappropriate interlanguage interaction may introduce design smells involving multiple programming languages (PLs), i.e., Inter-Language Design Smells (ILDS). Despite the negative impact of ILDS on multi-language DLFs, there is a lack of an automated approach for detecting ILDS in multi-language DLFs and a comprehensive understanding on ILDS in such DLFs. This work automatically detects ILDS in multi-language DLFs written in the combination of Python and C/C++, and to obtain a understanding on such ILDS in DLFs. We first developed an approach to automatically detecting ILDS in the multi-language DLFs written in the combination of Python and C/C++, including a number of ILDS and their detection rules defined based on inter-language communication mechanisms and code analysis. We then developed the CPSMELL tool that implements detection rules for automatically detecting such ILDS, and manually validated the accuracy of the tool. Finally, we performed a study to evaluate the ILDS in multi-language DLFs. We proposed seven ILDS and achieved an accuracy of 98.17% in the manual validation of CPSMELL in 5 popular multi-language DLFs. The study results revealed that among the 5 DLFs, TensorFlow, PyTorch, and PaddlePaddle exhibit relatively high prevalence of ILDS; each smelly file contains around 5 ILDS instances on average, with ILDS Long Lambda Function For Inter-language Binding and Unused Native Entity being relatively prominent; throughout the evolution process of the 5 DLFs, some ILDS were resolved to a certain extent, but the overall count of ILDS instances shows an upward trend. The automated detection of the proposed ILDS achieved a high accuracy, and the study provides a comprehensive understanding on ILDS in the multi-language DLFs.

cs.SE↗

arXiv2024

QADL: Prototype of Quantum Architecture Description Language

Quantum Software (QSW) uses the principles of quantum mechanics, specifically programming quantum bits (qubits) that manipulate quantum gates, to implement quantum computing systems. QSW has become a specialized field of software development, requiring specific notations, languages, patterns, and tools for mapping the behavior of qubits and the structure of quantum gates to components and connectors of QSW architectures. To support declarative modeling of QSW, we aim to enable architecture-driven development, where software engineers can design, program, and evaluate quantum software systems by abstracting complex details through high-level components and connectors. We introduce QADL (Quantum Architecture Description Language), which provides a specification language, design space, and execution environment for architecting QSW. Inspired by classical ADLs, QADL offers (1) a graphical interface to specify and design QSW components, (2) a parser for syntactical correctness, and (3) an execution environment by integrating QADL with IBM Qiskit. The initial evaluation of QADL is based on usability assessments by a team of quantum physicists and software engineers, using quantum algorithms such as Quantum Teleportation and Grover's Search. QADL offers a pioneering specification language and environment for QSW architecture. A demo is available at https://youtu.be/xaplHH_3NtQ.

quant-ph↗

arXiv2024

An Exploratory Study on Automatic Identification of Assumptions in the Development of Deep Learning Frameworks

Stakeholders constantly make assumptions in the development of deep learning (DL) frameworks. These assumptions are related to various types of software artifacts (e.g., requirements, design decisions, and technical debt) and can turn out to be invalid, leading to system failures. Existing approaches and tools for assumption management usually depend on manual identification of assumptions. However, assumptions are scattered in various sources (e.g., code comments, commits, pull requests, and issues) of DL framework development, and manually identifying assumptions has high costs. This study intends to evaluate different classification models for the purpose of identification with respect to assumptions from the point of view of developers and users in the context of DL framework projects (i.e., issues, pull requests, and commits) on GitHub. First, we constructed a new and largest dataset (i.e., the AssuEval dataset) of assumptions collected from the TensorFlow and Keras repositories on GitHub. Then we explored the performance of seven non-transformers based models (e.g., Support Vector Machine, Classification and Regression Trees), the ALBERT model, and three decoder-only models (i.e., ChatGPT, Claude, and Gemini) for identifying assumptions on the AssuEval dataset. The study results show that ALBERT achieves the best performance (f1-score: 0.9584) for identifying assumptions on the AssuEval dataset, which is much better than the other models (the 2nd best f1-score is 0.8858, achieved by the Claude 3.5 Sonnet model). Though ChatGPT, Claude, and Gemini are popular models, we do not recommend using them to identify assumptions in DL framework development because of their low performance. Fine-tuning ChatGPT, Claude, Gemini, or other language models (e.g., Llama3, Falcon, and BLOOM) specifically for assumptions might improve their performance for assumption identification.

cs.SE↗

arXiv2024

RAG-Enhanced Commit Message Generation

Commit message is one of the most important textual information in software development and maintenance. However, it is time-consuming to write commit messages manually. Commit Message Generation (CMG) has become a research hotspot. Recently, several pre-trained language models (PLMs) and large language models (LLMs) with code capabilities have been introduced, demonstrating impressive performance on code-related tasks. Meanwhile, prior studies have explored the utilization of retrieval techniques for CMG, but it is still unclear what effects would emerge from combining advanced retrieval techniques with various generation models. This paper proposed REACT, a REtrieval-Augmented framework for CommiT message generation. It integrates advanced retrieval techniques with different PLMs and LLMs, to enhance the performance of these models on the CMG task. Specifically, a hybrid retriever is designed and used to retrieve the most relevant code diff and commit message pair as an exemplar. Then, the retrieved pair is utilized to guide and enhance the CMG task by PLMs and LLMs through fine-tuning and in-context learning. The experimental results show that REACT significantly enhances these models' performance on the CMG task, improving the BLEU score of CodeT5 by up to 55%, boosting Llama 3's BLEU score by 102%, and substantially surpassing all baselines.

cs.SE↗

arXiv2024

Architecture Decisions in Quantum Software Systems: An Empirical Study on Stack Exchange and GitHub

Quantum computing provides a new dimension in computation, utilizing the principles of quantum mechanics to potentially solve complex problems that are currently intractable for classical computers. However, little research has been conducted about the architecture decisions made in quantum software development, which have a significant influence on the functionality, performance, scalability, and reliability of these systems. The study aims to empirically investigate and analyze architecture decisions made during the development of quantum software systems, identifying prevalent challenges and limitations by using the posts and issues from Stack Exchange and GitHub. We used a qualitative approach to analyze the obtained data from Stack Exchange Sites and GitHub projects. Specifically, we collected data from 385 issues (from 87 GitHub projects) and 70 posts (from three Stack Exchange sites) related to architecture decisions in quantum software development. The results show that in quantum software development (1) architecture decisions are articulated in six linguistic patterns, the most common of which are Solution Proposal and Information Giving, (2) the two major categories of architectural decisions are Implementation Decision and Technology Decision, (3) Softwar Development Tools are the most common application domain among the twenty application domains identified, (4) Maintainability is the most frequently considered quality attribute, and (5) Design Issues and High Error Rates are the major limitations and challenges that practitioners face when making architecture decisions in quantum software development. Our results show that the limitations and challenges encountered in architecture decision-making during the development of quantum software systems are strongly linked to the particular features (e.g., quantum entanglement, superposition, and decoherence) of those systems.

cs.SE↗

arXiv2024

Exploring the Problems, their Causes and Solutions of AI Pair Programming: A Study on GitHub and Stack Overflow

With the recent advancement of Artificial Intelligence (AI) and Large Language Models (LLMs), AI-based code generation tools become a practical solution for software development. GitHub Copilot, the AI pair programmer, utilizes machine learning models trained on a large corpus of code snippets to generate code suggestions using natural language processing. Despite its popularity in software development, there is limited empirical evidence on the actual experiences of practitioners who work with Copilot. To this end, we conducted an empirical study to understand the problems that practitioners face when using Copilot, as well as their underlying causes and potential solutions. We collected data from 473 GitHub issues, 706 GitHub discussions, and 142 Stack Overflow posts. Our results reveal that (1) Operation Issue and Compatibility Issue are the most common problems faced by Copilot users, (2) Copilot Internal Error, Network Connection Error, and Editor/IDE Compatibility Issue are identified as the most frequent causes, and (3) Bug Fixed by Copilot, Modify Configuration/Setting, and Use Suitable Version are the predominant solutions. Based on the results, we discuss the potential areas of Copilot for enhancement, and provide the implications for the Copilot users, the Copilot team, and researchers.

cs.SE↗

arXiv2024

Copilot-in-the-Loop: Fixing Code Smells in Copilot-Generated Python Code using Copilot

As one of the most popular dynamic languages, Python experiences a decrease in readability and maintainability when code smells are present. Recent advancements in Large Language Models have sparked growing interest in AI-enabled tools for both code generation and refactoring. GitHub Copilot is one such tool that has gained widespread usage. Copilot Chat, released in September 2023, functions as an interactive tool aimed at facilitating natural language-powered coding. However, limited attention has been given to understanding code smells in Copilot-generated Python code and Copilot Chat's ability to fix the code smells. To this end, we built a dataset comprising 102 code smells in Copilot-generated Python code. Our aim is to first explore the occurrence of code smells in Copilot-generated Python code and then evaluate the effectiveness of Copilot Chat in fixing these code smells employing different prompts. The results show that 8 out of 10 types of code smells can be detected in Copilot-generated Python code, among which Multiply-Nested Container is the most common one. For these code smells, Copilot Chat achieves a highest fixing rate of 87.1%, showing promise in fixing Python code smells generated by Copilot itself. In addition, the effectiveness of Copilot Chat in fixing these smells can be improved by providing more detailed prompts.

cs.SE↗

arXiv2024

Depends-Kotlin: A Cross-Language Kotlin Dependency Extractor

Since Google introduced Kotlin as an official programming language for developing Android apps in 2017, Kotlin has gained widespread adoption in Android development. However, compared to Java, there is limited support for Kotlin code dependency analysis, which is the foundation to software analysis. To bridge this gap, we develop Depends-Kotlin to extract entities and their dependencies in Kotlin source code. Not only does Depends-Kotlin support extracting entities' dependencies in Kotlin code, but it can also extract dependency relations between Kotlin and Java. Using three open-source Kotlin-Java mixing projects as our subjects, Depends-Kotlin demonstrates high accuracy and performance in resolving Kotlin-Kotlin and Kotlin-Java dependencies relations. The source code of Depends-Kotlin and the dataset used have been made available at https: //github.com/XYZboom/depends-kotlin. We also provide a screen-cast presenting Depends-Kotlin at https://youtu.be/ZPq8SRhgXzM.

cs.SE↗

arXiv2024

Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps

Fairness is one of the socio-technical concerns that must be addressed in software systems. Considering the popularity of mobile software applications (apps) among a wide range of individuals worldwide, mobile apps with unfair behaviors and outcomes can affect a significant proportion of the global population, potentially more than any other type of software system. Users express a wide range of socio-technical concerns in mobile app reviews. This research aims to investigate fairness concerns raised in mobile app reviews. Our research focuses on AI-based mobile app reviews as the chance of unfair behaviors and outcomes in AI-based mobile apps may be higher than in non-AI-based apps. To this end, we first manually constructed a ground-truth dataset, including 1,132 fairness and 1,473 non-fairness reviews. Leveraging the ground-truth dataset, we developed and evaluated a set of machine learning and deep learning models that distinguish fairness reviews from non-fairness reviews. Our experiments show that our best-performing model can detect fairness reviews with a precision of 94%. We then applied the best-performing model on approximately 9.5M reviews collected from 108 AI-based apps and identified around 92K fairness reviews. Next, applying the K-means clustering technique to the 92K fairness reviews, followed by manual analysis, led to the identification of six distinct types of fairness concerns (e.g., 'receiving different quality of features and services in different platforms and devices' and 'lack of transparency and fairness in dealing with user-generated content'). Finally, the manual analysis of 2,248 app owners' responses to the fairness reviews identified six root causes (e.g., 'copyright issues') that app owners report to justify fairness concerns.

cs.SE↗

arXiv2024

An Exploratory Study on Just-in-Time Multi-Programming-Language Bug Prediction

Context: An increasing number of software systems are written in multiple programming languages (PLs), which are called multi-programming-language (MPL) systems. MPL bugs (MPLBs) refers to the bugs whose resolution involves multiple PLs. Despite high complexity of MPLB resolution, there lacks MPLB prediction methods. Objective: This work aims to construct just-in-time (JIT) MPLB prediction models with selected prediction metrics, analyze the significance of the metrics, and then evaluate the performance of cross-project JIT MPLB prediction. Method: We develop JIT MPLB prediction models with the selected metrics using machine learning algorithms and evaluate the models in within-project and cross-project contexts with our constructed dataset based on 18 Apache MPL projects. Results: Random Forest is appropriate for JIT MPLB prediction. Changed LOC of all files, added LOC of all files, and the total number of lines of all files of the project currently are the most crucial metrics in JIT MPLB prediction. The prediction models can be simplified using a few top-ranked metrics. Training on the dataset from multiple projects can yield significantly higher AUC than training on the dataset from a single project for cross-project JIT MPLB prediction. Conclusions: JIT MPLB prediction models can be constructed with the selected set of metrics, which can be reduced to build simplified JIT MPLB prediction models, and cross-project JIT MPLB prediction is feasible.

cs.SE↗

arXiv2024

How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts

Since the emergence of GPT-3, Large Language Models (LLMs) have caught the eyes of researchers, practitioners, and educators in the field of software engineering. However, there has been relatively little investigation regarding the performance of LLMs in assisting with requirements analysis and UML modeling. This paper explores how LLMs can assist novice analysts in creating three types of typical UML models: use case models, class diagrams, and sequence diagrams. For this purpose, we designed the modeling tasks of these three UML models for 45 undergraduate students who participated in a requirements modeling course, with the help of LLMs. By analyzing their project reports, we found that LLMs can assist undergraduate students as novice analysts in UML modeling tasks, but LLMs also have shortcomings and limitations that should be considered when using them.

cs.SE↗

arXiv2024

Experimental Study on Deuterium-Deuterium Thermonuclear Fusion with Interface Confinement

Nuclear fusion is recognized as the energy of the future, and huge efforts and capitals have been put into the research of controlled nuclear fusion in the past decades. The most challenging thing for controlled nuclear fusion is to generate and keep a super high temperature. Here, a sonication system, combining with micro-scale fluid control techniques, was built to generate cavitation within a limited region. As bubbles being rapidly compressed, high temperature plasma generated interior leads to particle emissions, where a Cs2LiYCl6: Ce3+ (CLYC) scintillator was used to collect the emission events. The pulse shape discrimination methods applied on captured signals revealed that only gamma ray events were observed in sonication with normal water as excepted, while obvious separation of neutron and gamma ray events was surprisingly identified in sonication with deuterated water. This result suggested that neutrons were emitted from the sonicated deuterated water, i.e. deuterium-deuterium thermonuclear fusion was initiated. This study provides an alternative and feasible approach to achieve controllable nuclear fusion and makes great sense for future researches on the application of fusion energy.

nucl-ex↗