arXiv subjects

Lyuye Zhang

Publications and source records attributed to Lyuye Zhang.

23 records · Page 2Linked to original sources

arXiv2025

LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completion

Large language models (LLMs), pre-trained or fine-tuned on large code corpora, have shown effectiveness in generating code completions. However, in LLM-based code completion, LLMs may struggle to use correct and up-to-date Application Programming Interfaces (APIs) due to the rapid and continuous evolution of libraries. While existing studies have highlighted issues with predicting incorrect APIs, the specific problem of deprecated API usage in LLM-based code completion has not been thoroughly investigated. To address this gap, we conducted the first evaluation study on deprecated API usage in LLM-based code completion. This study involved seven advanced LLMs, 145 API mappings from eight popular Python libraries, and 28,125 completion prompts. The study results reveal the status quo (i.e., API usage plausibility and deprecated usage rate) of deprecated API and replacing API usage in LLM-based code completion from the perspectives of model, prompt, and library, and indicate the root causes behind. Based on these findings, we propose two lightweight fixing approaches, REPLACEAPI and INSERTPROMPT, which can serve as baseline approaches for future research on mitigating deprecated API usage in LLM-based completion. Additionally, we provide implications for future research on integrating library evolution with LLM-driven software development.

cs.SE↗

arXiv2024

Empirical Analysis of Vulnerabilities Life Cycle in Golang Ecosystem

Open-source software (OSS) greatly facilitates program development for developers. However, the high number of vulnerabilities in open-source software is a major concern, including in Golang, a relatively new programming language. In contrast to other commonly used OSS package managers, Golang presents a distinctive feature whereby commits are prevalently used as dependency versions prior to their integration into official releases. This attribute can prove advantageous to users, as patch commits can be implemented in a timely manner before the releases. However, Golang employs a decentralized mechanism for managing dependencies, whereby dependencies are upheld and distributed in separate repositories. This approach can result in delays in the dissemination of patches and unresolved vulnerabilities. To tackle the aforementioned concern, a comprehensive investigation was undertaken to examine the life cycle of vulnerability in Golang, commencing from its introduction and culminating with its rectification. To this end, a framework was established by gathering data from diverse sources and systematically amalgamating them with an algorithm to compute the lags in vulnerability patching. It turned out that 66.10% of modules in the Golang ecosystem were affected by vulnerabilities. Within the vulnerability life cycle, we found two kinds of lag impeding the propagation of vulnerability fixing. By analyzing reasons behind non-lagged and lagged vulnerabilities, timely releasing and indexing patch versions could significantly enhance ecosystem security.

cs.SE↗

arXiv2023

Mitigating Persistence of Open-Source Vulnerabilities in Maven Ecosystem

Vulnerabilities from third-party libraries (TPLs) have been unveiled to threaten the Maven ecosystem. Despite patches being released promptly after vulnerabilities are disclosed, the libraries and applications in the community still use the vulnerable versions, which makes the vulnerabilities persistent in the Maven ecosystem (e.g., the notorious Log4Shell still greatly influences the Maven ecosystem nowadays from 2021). Both academic and industrial researchers have proposed user-oriented standards and solutions to address vulnerabilities, while such solutions fail to tackle the ecosystem-wide persistent vulnerabilities because it requires a collective effort from the community to timely adopt patches without introducing breaking issues. To seek an ecosystem-wide solution, we first carried out an empirical study to examine the prevalence of persistent vulnerabilities in the Maven ecosystem. Then, we identified affected libraries for alerts by implementing an algorithm monitoring downstream dependents of vulnerabilities based on an up-to-date dependency graph. Based on them, we further quantitatively revealed that patches blocked by upstream libraries caused the persistence of vulnerabilities. After reviewing the drawbacks of existing countermeasures, to address them, we proposed a solution for range restoration (Ranger) to automatically restore the compatible and secure version ranges of dependencies for downstream dependents. The automatic restoration requires no manual effort from the community, and the code-centric compatibility assurance ensures smooth upgrades to patched versions. Moreover, Ranger along with the ecosystem monitoring can timely alert developers of blocking libraries and suggest flexible version ranges to rapidly unblock patch versions. By evaluation, Ranger could restore 75.64% of ranges which automatically remediated 90.32% of vulnerable downstream projects.

cs.SE↗

arXiv2023

Compatible Remediation on Vulnerabilities from Third-Party Libraries for Java Projects

With the increasing disclosure of vulnerabilities in open-source software, software composition analysis (SCA) has been widely applied to reveal third-party libraries and the associated vulnerabilities in software projects. Beyond the revelation, SCA tools adopt various remediation strategies to fix vulnerabilities, the quality of which varies substantially. However, ineffective remediation could induce side effects, such as compilation failures, which impede acceptance by users. According to our studies, existing SCA tools could not correctly handle the concerns of users regarding the compatibility of remediated projects. To this end, we propose Compatible Remediation of Third-party libraries (CORAL) for Maven projects to fix vulnerabilities without breaking the projects. The evaluation proved that CORAL not only fixed 87.56% of vulnerabilities which outperformed other tools (best 75.32%) and achieved a 98.67% successful compilation rate and a 92.96% successful unit test rate. Furthermore, we found that 78.45% of vulnerabilities in popular Maven projects could be fixed without breaking the compilation, and the rest of the vulnerabilities (21.55%) could either be fixed by upgrades that break the compilations or even be impossible to fix by upgrading.

cs.SE↗

arXiv2022

Has My Release Disobeyed Semantic Versioning? Static Detection Based on Semantic Differencing

To enhance the compatibility in the version control of Java Third-party Libraries (TPLs), Maven adopts Semantic Versioning (SemVer) to standardize the underlying meaning of versions, but users could still confront abnormal execution and crash after upgrades even if compilation and linkage succeed. It is caused by semantic breaking (SemB) issues, such that APIs directly used by users have identical signatures but inconsistent semantics across upgrades. To strengthen compliance with SemVer rules, developers and users should be alerted of such issues. Unfortunately, it is challenging to detect them statically, because semantic changes in the internal methods of APIs are difficult to capture. Dynamic testing can confirmingly uncover some, but it is limited by inadequate coverage. To detect SemB issues over compatible upgrades (Patch and Minor) by SemVer rules, we conduct an empirical study on 180 SemB issues to understand the root causes, inspired by which, we propose Sembid (Semantic Breaking Issue Detector) to statically detect such issues of TPLs for developers and users. Since APIs are directly used by users, Sembid detects and reports SemB issues based on APIs. For a pair of APIs, Sembid walks through the call chains originating from the API to locate breaking changes by measuring semantic diff. Then, Sembid checks if the breaking changes can affect API's output along call chains. The evaluation showed Sembid achieved 90.26% recall and 81.29% precision and outperformed other API checkers on SemB API detection. We also revealed Sembid detected over 3 times more SemB APIs with better coverage than unit tests, the commonly used solution. Furthermore, we carried out an empirical study on 1,629,589 APIs from 546 version pairs of top Java libraries and found there were 2-4 times more SemB APIs than those with signature-based issues.

cs.SE↗