arXiv subjects

Laerte Xavier

Publications and source records attributed to Laerte Xavier.

7 recordsLinked to original sources

arXiv2025

Using Copilot Agent Mode to Automate Library Migration: A Quantitative Assessment

Keeping software systems up to date is essential to avoid technical debt, security vulnerabilities, and the rigidity typical of legacy systems. However, updating libraries and frameworks remains a time consuming and error-prone process. Recent advances in Large Language Models (LLMs) and agentic coding systems offer new opportunities for automating such maintenance tasks. In this paper, we evaluate the update of a well-known Python library, SQLAlchemy, across a dataset of ten client applications. For this task, we use the Github's Copilot Agent Mode, an autonomous AI systema capable of planning and executing multi-step migration workflows. To assess the effectiveness of the automated migration, we also introduce Migration Coverage, a metric that quantifies the proportion of API usage points correctly migrated. The results of our study show that the LLM agent was capable of migrating functionalities and API usages between SQLAlchemy versions (migration coverage: 100%, median), but failed to maintain the application functionality, leading to a low test-pass rate (39.75%, median).

cs.SE↗

arXiv2024

Automatic Library Migration Using Large Language Models: First Results

Despite being introduced only a few years ago, Large Language Models (LLMs) are already widely used by developers for code generation. However, their application in automating other Software Engineering activities remains largely unexplored. Thus, in this paper, we report the first results of a study in which we are exploring the use of ChatGPT to support API migration tasks, an important problem that demands manual effort and attention from developers. Specifically, in the paper, we share our initial results involving the use of ChatGPT to migrate a client application to use a newer version of SQLAlchemy, an ORM (Object Relational Mapping) library widely used in Python. We evaluate the use of three types of prompts (Zero-Shot, One-Shot, and Chain Of Thoughts) and show that the best results are achieved by the One-Shot prompt, followed by the Chain Of Thoughts. Particularly, with the One-Shot prompt we were able to successfully migrate all columns of our target application and upgrade its code to use new functionalities enabled by SQLAlchemy's latest version, such as Python's asyncio and typing modules, while preserving the original code behavior.

cs.SE↗

arXiv2024

Comments or Issues: Where to Document Technical Debt?

Self-Admitted Technical Debt (SATD) is a form of Technical Debt where developers document the debt using source code comments (SATD-C) or issues (SATD-I). However, it is still unclear the circumstances that drive developers to choose one or another. In this paper, we survey authors of both types of debts using a large-scale dataset containing 74K SATD-C and 20K SATD-I instances, extracted from 190 GitHub projects. As a result, we provide 13 guidelines to support developers to decide when to use comments or issues to report Technical Debt.

cs.SE↗

arXiv2024

On the Quality of AI-Generated Source Code Comments: A Comprehensive Evaluation

This paper investigates the quality of source code comments automatically generated by Large Language Models (LLMs). While AI-based comment generation has emerged as a promising solution to reduce developers' documentation effort, prior studies have been limited by small datasets or by relying solely on traditional Information Retrieval (IR) metrics, which are insufficient to capture documentation quality. To address these limitations, we conducted a large-scale empirical study on 142 classes and 273 methods created after the training cut-off of the evaluated models. For each code element, we generated Javadoc comments using three LLMs (GPT-3.5 Turbo, GPT-4o, and DeepSeek-V3). A qualitative assessment of the comments-performed independently by two experts-showed that 58.8% were equivalent to, and 27.7% superior to, the original comments. A quantitative analysis using BLEU, ROUGE-L, and METEOR confirmed that IR-based metrics do not reliably reflect human evaluations, revealing the need for new documentation-specific metrics. Finally, correlation analyses indicated slightly positive relationships between code properties (size, complexity, coupling) and comment quality, confirming that LLMs benefit from richer contextual information.

cs.SE↗

arXiv2022

Together or Apart? Investigating a mediator bot to aggregate bot's comments on pull requests

Software bots connect users and tools, streamlining the pull request review process in social coding platforms. However, bots can introduce information overload into developers' communication. Information overload is especially problematic for newcomers, who are still exploring the project and may feel overwhelmed by the number of messages. Inspired by the literature of other domains, we designed and evaluated FunnelBot, a bot that acts as a mediator between developers and other bots in the repository. We conducted a within-subject study with 25 newcomers to capture their perceptions and preferences. Our results provide insights for bot developers who want to mitigate noise and create bots for supporting newcomers, laying a foundation for designing better bots.

cs.SE↗

arXiv2020

Beyond the Code: Mining Self-Admitted Technical Debt in Issue Tracker Systems

Self-admitted technical debt (SATD) is a particular case of Technical Debt (TD) where developers explicitly acknowledge their sub-optimal implementation decisions. Previous studies mine SATD by searching for specific TD-related terms in source code comments. By contrast, in this paper we argue that developers can admit technical debt by other means, e.g., by creating issues in tracking systems and labelling them as referring to TD. We refer to this type of SATD as issue-based SATD or just SATD-I. We study a sample of 286 SATD-I instances collected from five open source projects, including Microsoft Visual Studio and GitLab Community Edition. We show that only 29% of the studied SATD-I instances can be tracked to source code comments. We also show that SATD-I issues take more time to be closed, compared to other issues, although they are not more complex in terms of code churn. Besides, in 45% of the studied issues TD was introduced to ship earlier, and in almost 60% it refers to Design flaws. Finally, we report that most developers pay SATD-I to reduce its costs or interests (66%). Our findings suggest that there is space for designing novel tools to support technical debt management, particularly tools that encourage developers to create and label issues containing TD concerns.

cs.SE↗

arXiv2018

Why and How Java Developers Break APIs

Modern software development depends on APIs to reuse code and increase productivity. As most software systems, these libraries and frameworks also evolve, which may break existing clients. However, the main reasons to introduce breaking changes in APIs are unclear. Therefore, in this paper, we report the results of an almost 4-month long field study with the developers of 400 popular Java libraries and frameworks. We configured an infrastructure to observe all changes in these libraries and to detect breaking changes shortly after their introduction in the code. After identifying breaking changes, we asked the developers to explain the reasons behind their decision to change the APIs. During the study, we identified 59 breaking changes, confirmed by the developers of 19 projects. By analyzing the developers' answers, we report that breaking changes are mostly motivated by the need to implement new features, by the desire to make the APIs simpler and with fewer elements, and to improve maintainability. We conclude by providing suggestions to language designers, tool builders, software engineering researchers and API developers.

cs.SE↗