Searcharxiv⌕ Search

arXiv subjects

Gema Rodríguez-Pérez

Publications and source records attributed to Gema Rodríguez-Pérez.

15 recordsLinked to original sources

When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation

Large Language Models (LLMs) are widely used as programming assistants, yet it remains unclear whether and how user's demographic information impacts the technical quality of generated code. We conduct a large-scale empirical study of persona-induced bias in LLM-based code generation, focusing a proprietary model (Gemini 2.5 Pro) and an open-weight model (GPT-OSS-120B). Using 18 demographic personas spanning nationality, gender, and experience level, we compare persona-induced prompts against a neutral baseline. Across 35,000+ generated programs, we analyze demographic marker leakage in reasoning and responses, as well as differences in functional correctness, maintainability, code style, and security. Our results show that demographic cues are frequently reflected in LLM reasoning and outputs. Demographic markers appear in up to 65% of responses and 70% of reasoning traces, despite being semantically irrelevant to the tasks. On LiveCodeBench, persona prompting were associated with lower correctness scores of the Gemini model by an average of 1.54 percentage points, with one persona exhibiting a decrease of 3.6% (odds ratio = 0.51). In contrast, the accuracy of the GPT-OSS model improved by 3.4 - 5.7% across all personas (odds ratios = 1.8 - 3.0). Maintainability and code style metrics show statistically significant but negligible effect sizes (all Cliff's δ < 0.15), and security vulnerabilities exhibit no systematic persona-specific patterns. Overall, our results show that the presence of demographic information about users is associated with measurable variation in LLM reasoning and code quality even in purely technical tasks, and that these effects hold across models. Our work highlights an under-examined risk in LLM-assisted software development.

cs.SE↗

Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation

Large Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code. While prompt wording and structure are known to influence model performance, the impact of psychologically inspired prompt framings remains unexplored. This study investigates whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective prompt framing, which may, in turn, affect LLM behaviour in coding tasks. Drawing on Yukl & Falbe's well-known taxonomy, we operationalized eight influence tactics (like rational persuasion, ingratiation, and exchange) into reproducible prompt templates. These prompt templates were evaluated across five leading open-weight LLMs using two widely adopted benchmarks: LiveCodeBench and SWE-bench Verified. We assessed the resulting code output on four key software quality dimensions: functional correctness, quality, maintainability, and security. Our results show that certain influence-induced prompt framings, particularly those emphasizing urgency, were associated with reduced correctness and security. This work presents the first large-scale empirical study of influence-induced prompt framing in software engineering tasks, offering insights into how linguistic cues may shape LLM outputs. We conclude with practical insights for designing transparent and interpretable human-AI interactions in code generation.

cs.SE↗

Maintenance Signals in AI-Assisted GitHub Repositories: Evidence from GenAI Adopters

Generative artificial intelligence (GenAI) can reduce code-generation effort, but it may shift work to documentation, validation, debugging, and maintenance. We study observable maintenance-cost signals among GenAI adopters on GitHub by analyzing 622 users who publicly signal adoption, 179 repositories with visible AI-assistance configuration files, 179 matched traditional repositories, and 248 issues created in AI-assisted repositories. AI-assisted repositories span diverse project types and contain longer README files with more headers and code blocks, while traditional repositories contain more external URLs. Issues concerning GenAI technology often involve external dependencies, such as API rate limits and reliance on GenAI provider APIs. These findings suggest that AI assistance shifts maintenance costs toward verifying generated content, managing external AI dependencies, and validating AI-specific behavior.

cs.SE↗

Towards Understanding Barriers and Mitigation Strategies of Software Engineers with Non-traditional Educational and Occupational Backgrounds

The traditional path to a software engineering career usually involves a post-secondary diploma in Software Engineering, Computer Science, or a related field. However, many individuals working as software engineers take a non-traditional path to their careers, starting from other industries or fields of study. This paper explores the barriers that individuals with non-traditional educational and occupational backgrounds face when pursuing a software engineering career and proposes potential strategies to overcome those barriers. A two-stage methodology was used, consisting of an exploratory study followed by a follow-up survey. The exploratory study consisted of a grounded-theory-based qualitative analysis of relevant Reddit data to yield a framework around the barriers and possible mitigation strategies. These findings were then supplemented through a follow-up survey. Understanding these barriers and what strategies could be effective is an important step towards making software engineering more accessible to individuals with non-traditional backgrounds. In addition to fostering functional diversity, this might also serve to tackle labor shortages within the software engineering industry.

cs.SE↗

InEx-Bug: A Human Annotated Dataset of Intrinsic and Extrinsic Bugs in the NPM Ecosystem

Understanding the causes of software defects is essential for reliable software maintenance and ecosystem stability. However, existing bug datasets do not distinguish between issues originating within a project from those caused by external dependencies or environmental factors. In this paper we present InEx-Bug, a manually annotated dataset of 377 GitHub issues from 103 NPM repositories, categorizing issues as Intrinsic (internal defect), Extrinsic (dependency/environment issue), Not-a-Bug, or Unknown. Beyond labels, the dataset includes rich temporal and behavioral metadata such as maintainer participation, code changes, and reopening patterns. Analyses show Intrinsic bugs resolve faster (median 8.9 vs 10.2 days), are close more often (92% vs 78%), and require code changes more frequently (57% vs 28%) compared to Extrinsic bugs. While Extrinsic bugs exhibit higher reopen rates (12% vs 4%) and delayed recurrence (median 157 vs 87 days). The dataset provides a foundation for further studying Intrinsic and Extrinsic defects in the NPM ecosystem.

cs.SE↗

More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requests

Large Language Model (LLM) Agents are advancing quickly, with the increasing leveraging of LLM Agents to assist in development tasks such as code generation. While LLM Agents accelerate code generation, studies indicate they may introduce adverse effects on development. However, existing metrics solely measure pass rates, failing to reflect impacts on long-term maintainability and readability, and failing to capture human intuitive evaluations of PR. To increase the comprehensiveness of this problem, we investigate and evaluate the characteristics of LLM to know the pull requests' characteristics beyond the pass rate. We observe the code quality and maintainability within PRs based on code metrics to evaluate objective characteristics and developers' reactions to the pull requests from both humans and LLM's generation. Evaluation results indicate that LLM Agents frequently disregard code reuse opportunities, resulting in higher levels of redundancy compared to human developers. In contrast to the quality issues, our emotions analysis reveals that reviewers tend to express more neutral or positive emotions towards AI-generated contributions than human ones. This disconnect suggests that the surface-level plausibility of AI code masks redundancy, leading to the silent accumulation of technical debt in real-world development environments. Our research provides insights for improving human-AI collaboration.

cs.SE↗

What About Our Bug? A Study on the Responsiveness of NPM Package Maintainers

Background: Widespread use of third-party libraries makes ecosystems like Node Package Manager (npm) critical to modern software development. However, this interconnected chain of dependencies also creates challenges: bugs in one library can propagate downstream, potentially impacting many other libraries that rely on it. We hypothesize that maintainers may not always decide to fix a bug, especially if the maintainer decides it falls out of their responsibility within the chain of dependencies. Aims: To confirm this hypothesis, we investigate the responsiveness of 30,340 bug reports across 500 of the most depended-upon npm packages. Method: We adopt a mixed-method approach to mine repository issue data and perform qualitative open coding to analyze reasons behind unaddressed bug reports. Results: Our findings show that maintainers are generally responsive, with a median project-level responsiveness of 70% (IQR: 55%-89%), reflecting their commitment to support downstream developers. Conclusions: We present a taxonomy of the reasons some bugs remain unresolved. The taxonomy includes contribution practices, dependency constraints, and library-specific standards as reasons for not being responsive. Understanding maintainer behavior can inform practices that promote a more robust and responsive open-source ecosystem that benefits the entire community.

cs.SE↗

CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs

Understanding how developers interact with code generation tools (CGTs) requires detailed, real-time data on programming behavior which is often difficult to collect without disrupting workflow. We present \textit{CodeWatcher}, a lightweight, unobtrusive client-server system designed to capture fine-grained interaction events from within the Visual Studio Code (VS Code) editor. \textit{CodeWatcher} logs semantically meaningful events such as insertions made by CGTs, deletions, copy-paste actions, and focus shifts, enabling continuous monitoring of developer activity without modifying user workflows. The system comprises a VS Code plugin, a Python-based RESTful API, and a MongoDB backend, all containerized for scalability and ease of deployment. By structuring and timestamping each event, \textit{CodeWatcher} enables post-hoc reconstruction of coding sessions and facilitates rich behavioral analyses, including how and when CGTs are used during development. This infrastructure is crucial for supporting research on responsible AI, developer productivity, and the human-centered evaluation of CGTs. Please find the demo, diagrams, and tool here: https://osf.io/j2kru/overview.

cs.SE↗

Reflection on Code Contributor Demographics and Collaboration Patterns in the Rust Community

Open-source software communities thrive on global collaboration and contributions from diverse participants. This study explores the Rust programming language ecosystem to understand its contributors' demographic composition and interaction patterns. Our objective is to investigate the phenomenon of participation inequality in key Rust projects and the presence of diversity among them. We studied GitHub pull request data from the year leading up to the release of the latest completed Rust community annual survey in 2023. Specifically, we extracted information from three leading repositories: Rust, Rust Analyzer, and Cargo, and used social network graphs to visualize the interactions and identify central contributors and sub-communities. Social network analysis has shown concerning disparities in gender and geographic representation among contributors who play pivotal roles in collaboration networks and the presence of varying diversity levels in the sub-communities formed. These results suggest that while the Rust community is globally active, the contributor base does not fully reflect the diversity of the wider user community. We conclude that there is a need for more inclusive practices to encourage broader participation and ensure that the contributor base aligns more closely with the diverse global community that utilizes Rust.

cs.SE↗

Unveiling Ruby: Insights from Stack Overflow and Developer Survey

Ruby is a widely used open-source programming language, valued for its simplicity, especially in web development. Despite its popularity, with over one million users on GitHub, little is known about the issues faced by Ruby developers. This study aims to investigate the key topics, trends, and difficulties faced by Ruby developers by analyzing over 498,000 Ruby-related questions on Stack Overflow (SO), followed by a survey of 154 Ruby developers. We employed BERTopic modeling and manual analysis to develop a taxonomy of 35 topics, grouped into six main categories. Our findings reveal that Web Application Development is the most commonly discussed category, while Ruby Gem Installation and Configuration Issues emerged as the most challenging topic. Analysis of trends on SO showed a steady decline. A survey of 154 Ruby developers demonstrated that 31.6% of the participants find the Core Ruby Concepts category particularly difficult, while Application Quality and Security is found to be difficult for over 40% of experienced developers. Notably, a comparison between survey responses and SO metrics highlights a misalignment, suggesting that perceived difficulty and objective indicators from SO differ; emphasizing the need for improved metrics to better capture developer challenges. Our study provides insights about the challenges Ruby developers face and strong implications for researchers.

cs.SE↗

Buggin: Automatic intrinsic bugs classification model using NLP and ML

Recent studies have shown that bugs can be categorized into intrinsic and extrinsic types. Intrinsic bugs can be backtracked to specific changes in the version control system (VCS), while extrinsic bugs originate from external changes to the VCS and lack a direct bug-inducing change. Using only intrinsic bugs to train bug prediction models has been reported as beneficial to improve the performance of such models. However, there is currently no automated approach to identify intrinsic bugs. To bridge this gap, our study employs Natural Language Processing (NLP) techniques to automatically identify intrinsic bugs. Specifically, we utilize two embedding techniques, seBERT and TF-IDF, applied to the title and description text of bug reports. The resulting embeddings are fed into well-established machine learning algorithms such as Support Vector Machine, Logistic Regression, Decision Tree, Random Forest, and K-Nearest Neighbors. The primary objective of this paper is to assess the performance of various NLP and machine learning techniques in identifying intrinsic bugs using the textual information extracted from bug reports. The results demonstrate that both seBERT and TF-IDF can be effectively utilized for intrinsic bug identification. The highest performance scores were achieved by combining TF-IDF with the Decision Tree algorithm and utilizing the bug titles (yielding an F1 score of 78%). This was closely followed by seBERT, Support Vector Machine, and bug titles (with an F1 score of 77%). In summary, this paper introduces an innovative approach that automates the identification of intrinsic bugs using textual information derived from bug reports.

cs.SE↗

A Preliminary Study of Multilingual Code Language Models for Code Generation Task Using Translated Benchmarks

Evaluating the performance of Code Language Models (CLMs) for software engineering tasks, especially in multilingual and low-resource programming language settings, poses significant challenges. These challenges are primarily due to the lack of high-quality benchmarks across various programming languages and the imbalanced nature of the CLMs training corpus. Although recent advances in one of the common downstream tasks, code generation, have shown promise by introducing translated benchmarks using different methodologies, there is a current lack of empirical evidence assessing these benchmarks. To address this gap, we conducted a preliminary study to evaluate the performance of Poly-Coder, a pioneering open-source, multilingual CLM built for code generation. We utilized two existing state-of-the-art translations of the popular code generation benchmark, HumanEval, facilitated by the OctoPack and MultiPL-E studies. Our results suggest that the outcomes observed in these translated benchmarks align well with evaluation metrics used during the training phase, such as perplexity, thereby validating their effectiveness in estimating the performance of CLMs. However, we identified several inconsistencies in the CLMs' performance across the translated benchmarks and encountered challenges in replicating the results. These initial insights highlight the need for more comprehensive empirical studies to fully understand translated benchmarks' methodological approaches, limitations, and reproducibility. Such studies are essential to ensure their reliability before they are widely adopted.

cs.SE↗

Investigating the Efficacy of Large Language Models for Code Clone Detection

Large Language Models (LLMs) have demonstrated remarkable success in various natural language processing and software engineering tasks, such as code generation. The LLMs are mainly utilized in the prompt-based zero/few-shot paradigm to guide the model in accomplishing the task. GPT-based models are one of the popular ones studied for tasks such as code comment generation or test generation. These tasks are `generative' tasks. However, there is limited research on the usage of LLMs for `non-generative' tasks such as classification using the prompt-based paradigm. In this preliminary exploratory study, we investigated the applicability of LLMs for Code Clone Detection (CCD), a non-generative task. By building a mono-lingual and cross-lingual CCD dataset derived from CodeNet, we first investigated two different prompts using ChatGPT to detect Type-4 code clones in Java-Java and Java-Ruby pairs in a zero-shot setting. We then conducted an analysis to understand the strengths and weaknesses of ChatGPT in CCD. ChatGPT surpasses the baselines in cross-language CCD attaining an F1-score of 0.877 and achieves comparable performance to fully fine-tuned models for mono-lingual CCD, with an F1-score of 0.878. Also, the prompt and the difficulty level of the problems has an impact on the performance of ChatGPT. Finally we provide insights and future directions based on our initial analysis

cs.SE↗

A Fine-grained Data Set and Analysis of Tangling in Bug Fixing Commits

Context: Tangled commits are changes to software that address multiple concerns at once. For researchers interested in bugs, tangled commits mean that they actually study not only bugs, but also other concerns irrelevant for the study of bugs. Objective: We want to improve our understanding of the prevalence of tangling and the types of changes that are tangled within bug fixing commits. Methods: We use a crowd sourcing approach for manual labeling to validate which changes contribute to bug fixes for each line in bug fixing commits. Each line is labeled by four participants. If at least three participants agree on the same label, we have consensus. Results: We estimate that between 17% and 32% of all changes in bug fixing commits modify the source code to fix the underlying problem. However, when we only consider changes to the production code files this ratio increases to 66% to 87%. We find that about 11% of lines are hard to label leading to active disagreements between participants. Due to confirmed tangling and the uncertainty in our data, we estimate that 3% to 47% of data is noisy without manual untangling, depending on the use case. Conclusion: Tangled commits have a high prevalence in bug fixes and can lead to a large amount of noise in the data. Prior research indicates that this noise may alter results. As researchers, we should be skeptics and assume that unvalidated data is likely very noisy, until proven otherwise.

cs.SE↗

On the Relationship Between the Developer's Perceptible Race and Ethnicity and the Evaluation of Contributions in OSS

Open Source Software (OSS) projects are typically the result of collective efforts performed by developers with different backgrounds. Although the quality of developers' contributions should be the only factor influencing the evaluation of the contributions to OSS projects, recent studies have shown that diversity issues are correlated with the acceptance or rejection of developers' contributions. This paper assists this emerging state-of-the-art body on diversity research with the first empirical study that analyzes how developers' perceptible race and ethnicity relates to the evaluation of the contributions in OSS. We performed a large-scale quantitative study of OSS projects in GitHub. We extracted the developers' perceptible race and ethnicity from their names in GitHub using the Name-Prism tool and applied regression modeling of contributions (i.e, pull requests) data from GHTorrent and GitHub. We observed that among the developers whose perceptible race and ethnicity was captured by the tool, only 16.56% were perceptible as Non-White developers; contributions from perceptible White developers have about 6-10% higher odds of being accepted when compared to contributions from perceptible Non-White developers; and submitters with perceptible non-white races and ethnicities are more likely to get their pull requests accepted when the integrator is estimated to be from their same race and ethnicity rather than when the integrator is estimated to be White. Our initial analysis shows a low number of Non-White developers participating in OSS. Furthermore, the results from our regression analysis lead us to believe that there may exist differences between the evaluation of the contributions from different perceptible races and ethnicities. Thus, our findings reinforce the need for further studies on racial and ethnic diversity in software engineering to foster healthier OSS communities.

cs.SE↗