Searcharxiv⌕ Search

arXiv subjects

Benoît Vanderose

Publications and source records attributed to Benoît Vanderose.

6 recordsLinked to original sources

An Empirical Study of Maintainer-Identified AI Slop Pull Requests in Free and Open Source Software Projects

The FOSS ecosystem has always relied on outside contributions, notably through issues or Pull Requests (PRs), to sustain itself. With the advance of GenAI, the barrier to contributing to FOSS projects has never been lower, leading to the emergence of AI slop PRs: low-quality, low-value generated PRs. Past studies have examined AI slop PRs through theoretical, platform-level, or practitioner-account frames, but not at the project level. To this end, this study investigates the incidence of maintainer-identified AI slop PRs in FOSS projects by focusing on PRs flagged as slop by maintainers. Using a mixed-methods approach, we trace the origins of maintainer-identified AI slop PRs, quantify their volume in FOSS projects, describe their characteristics, and highlight maintainers' reasons for flagging or closing them. Our results show that the volume of maintainer-identified AI slop PRs increased in mid-2025, peaking in May 2026. These PRs are mostly additive, rarely merged, and typically closed within 3 days. The authors of these PRs disclosed the use of AI in 14% of cases, with 1% disclosing after a maintainer's request. Most PRs are closed or marked as slop without any reason provided by the maintainer. Based on our open coding, reasons range from concrete elements, like build failures, to ideological or intuitive (i.e., "looks AI to me"). This exploratory study captures a phenomenon in its early stage and highlights GenAI's potential impact on FOSS, emphasizing the pressure it puts on maintainers and the need to shift current open-source development practices.

cs.SE↗

Systematic Detection of Energy Regressions in Software Version Histories

Context: Green software engineering is emerging as a crucial response to the rising energy impact of information technology, especially in continuous development. However, challenges remain in devising automated methods to efficiently identify energy regressions across commits. In particular, little effort has been put into automatically detecting regressions at the commit level by identifying statistically significant changes in energy consumption. Objectives: This paper presents EnergyTrackr, an approach for detecting energy regressions across multiple commits. Methods: We empirically evaluated EnergyTrackr on 1,932 commits from five Java projects, assessing its ability to identify significant energy changes. We also compared a straightforward linear algorithm that analyzes the entire commit history with a more efficient dichotomous search algorithm. Results: The evaluation demonstrates the effectiveness of EnergyTrackr in identifying significant energy changes across commits. Our findings highlight the efficiency of the dichotomous algorithm in uncovering overall software energy trends, achieving results comparable to the linear approach while reducing resource consumption and processing time. Conclusion: We expect EnergyTrackr to help developers accurately monitor energy regressions and improvements in their projects and optimize their source code to reduce software energy consumption.

cs.SE↗

How Developers Experience Debugging Unfamiliar Codebases with Code Tours Generated and Evaluated by Local LLMs

Code tours are interactive, onboarding documentation to guide developers through a codebase. Large Language Models (LLMs) can automatically synthesize code tours. Prior work on code tour generation has not studied developer experience or trust calibration when debugging unfamiliar codebases with code tours generated and evaluated by open-weight LLMs. This study surveys how the properties of components in open-weight LLM-authored code tours influence developers' experiences when debugging unfamiliar codebases. We built a pipeline that generated and evaluated code tours from real reproducible bugs. 26 developers with varying backgrounds participated in a user study. In total, 26 code tours were authored from real Java bugs mined from 2025 GitHub commits, with each tour independently judged by two different LLMs, resulting in 52 evaluated configurations. Participants thought aloud as they explored each tour. Three authors qualitatively coded the interviews to identify recurring themes. Developers generally preferred tours that scaled detail with the code length, avoided merely restating code, were easily scannable, and adopted a guiding tone. However, some preferences were mutually exclusive, such as the use of imperative mood. Stack traces were often insufficient to identify all steps developers found relevant. Developers also trusted descriptions they perceived as human-written more than those they believed were AI-generated. Finally, LLM-generated annotations of tour quality were unreliable: sycophancy, confabulation, and incoherence were pervasive. This work lays a basis for future research on fine-tuning open-weight models for code tour generation, personalizing generation to accommodate diverging preferences, selecting relevant steps beyond stack traces, calibrating users' trust to avoid both disuse and misuse, and improving open-weight LLMs' ability to be more trustworthy evaluators

cs.SE↗

Systematic Detection of Energy Regression and Corresponding Code Patterns in Java Projects

Green software engineering is emerging as a crucial response to information technology's rising energy impact, especially in continuous development. However, there remain challenges in devising automated methods for identifying energy regressions across commits and their associated code change patterns. In particular, little effort has been put into automatically detecting regressions at the commit level by identifying statistically significant changes in energy consumption. In this paper, we introduce EnergyTrackr, an approach designed to detect energy regressions across multiple commits that can then be used to identify code anti-patterns potentially contributing to the increase of software energy consumption over time. We describe our empirical evaluation, including repository mining and source code analysis, made on 3,232 commits from three Java projects, and show the approach's ability to identify significant energy changes. We also highlight recurring anti-patterns such as missing early exits or costly dependency upgrades. We expect EnergyTrackr to assist developers in accurately monitoring energy regressions and improvements within their projects, identifying code anti-patterns, and helping them optimize their source code to reduce software energy consumption.

cs.SE↗

Measuring Agile Agreement: Development and Validation of the Manifesto and Principle Scales

While the importance of human factors in agile software development is widely acknowledged, the measurement of an individual's "agile agreement" remains an ill-defined and challenging area. A key limitation in existing research is the failure to distinguish between agreement with the abstract, high-level values of the Agile Manifesto and agreement with the concrete, day-to-day practices derived from the 12 Principles. This paper addresses this methodological gap by presenting the design and validation of two distinct instruments: the novel Manifesto Agreement Scale (MAS), and the Principle Agreement Scale (PAS), which is a systematic adaptation and refinement of a prior instrument. We detail the systematic process of item creation and selection, survey design, and validation. The results demonstrate that both scales possess important internal consistency and construct validity. A convergence and divergence analysis, including Proportional Odds Logistic Regression, a Bland-Altman plot, and an Intraclass Correlation Coefficient (ICC), reveals that while the two scales are moderately correlated, they are not interchangeable and capture distinct dimensions of agile agreement. The primary contribution of this work is a pair of publicly available instruments, validated within a specific demographic of Belgian IT professionals. These scales represent a critical initial step toward facilitating a more nuanced measurement of agile agreement, distinguishing agile agreement across various levels of perception and aiding in a more refined interpretation of person-agile fit.

cs.SE↗

Towards debiasing code review support

Cognitive biases appear during code review. They significantly impact the creation of feedback and how it is interpreted by developers. These biases can lead to illogical reasoning and decision-making, violating one of the main hypotheses supporting code review: developers' accurate and objective code evaluation. This paper explores harmful cases caused by cognitive biases during code review and potential solutions to avoid such cases or mitigate their effects. In particular, we design several prototypes covering confirmation bias and decision fatigue. We rely on a developer-centered design approach by conducting usability tests and validating the prototype with a user experience questionnaire (UEQ) and participants' feedback. We show that some techniques could be implemented in existing code review tools as they are well accepted by reviewers and help prevent behavior detrimental to code review. This work provides a solid first approach to treating cognitive bias in code review.

cs.SE↗