arXiv subjects

Oscar Chaparro

Publications and source records attributed to Oscar Chaparro.

At least 19 recordsLinked to original sources

arXiv2026

On the Reliability of Code Comprehension Proxies

Prior work on code comprehension uses different comprehension proxies---for example, Likert-scale ratings or answers to input-output questions about program snippets, usually collected from students, to approximate whether code is comprehensible to software engineers, but the relative reliability of these proxies is not known. This paper investigates the relative reliability of a collection of proxies common in the extant literature with a pair of human studies. First, we conducted an expert-consensus study with a panel of five professional software engineers to establish a ground-truth comprehensibility ranking of eight code snippets by adapting the Delphi expert-consensus protocol. The Delphi protocol is widely used for expert consensus under conditions of uncertainty in other domains such as medicine and national-security forecasting, but to our knowledge, this is its first application to code comprehension research. Second, we conducted a study with 44 student participants who completed comprehension tasks, allowing us to measure 14 comprehension proxies derived from the literature on the same set of eight code snippets. Finally, we conducted a correlation analysis on the results, concluding that proxies 1) derived from input-output questions and 2) that measure response time rather than accuracy are especially reliable. We also found that proxies derived from questions about program syntax (rather than semantics) are especially unreliable, regardless of measurement strategy, which draws into question the reliability of parts of the existing comprehensibility literature.

cs.SE↗

arXiv2026

From Absolute to Relative Code Comprehensibility Prediction

Automatically predicting code comprehensibility could support tasks such as refactoring and code review. Existing metrics correlate poorly with human comprehension, motivating ML models that predict comprehensibility directly from code and developer features. Prior models predict absolute comprehensibility (AC), a comprehensibility value for an isolated snippet, but perform poorly since AC is a subjective proxy for a complex cognitive process. We propose relative comprehensibility (RC) as an alternative task: given two snippets, predict which is easier to understand or whether they are comparable. We hypothesize RC is easier to learn, since it only requires identifying distinguishing features between snippets rather than estimating absolute values. Using 150 Java snippets and 12,540 human comprehensibility measurements from two prior studies, we compare AC and RC prediction across classical ML models, a CNN, and two LLMs, evaluating both snippet-wise (aggregate) and developer-wise (individual-judgment) predictions. AC models rarely beat simple baselines (at most 33.4\% average relative improvement), while snippet-wise RC models outperform baselines in 96.8\% of configurations, with gains up to 159.8\% consistent across architectures, though developer-wise results are more variable. We surveyed 38 practitioners and found both AC and RC useful, with a stronger preference for RC in comparison-oriented tasks like refactoring and review.

cs.SE↗

arXiv2026

Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation

Multimodal large language models (MLLMs) are increasingly used to translate webpage screenshots into front-end code, but repeated UI patterns may sway them toward visually incorrect yet pattern-consistent outputs. In this work, we test how repeated webpage patterns hurt MLLM accuracy on an objective screenshot-to-code fill-in-the-blank task. We introduce the first benchmark for visual pattern-completion bias, where one localized element in a repeated UI pattern is perturbed and the model must recover the masked width or font-size value from the screenshot and HTML context. Starting from 30 webpages curated from the Design2Code dataset, we build 1,440 evaluated screenshots spanning structural card and text-style patterns under standard and noise-overlaid conditions. We evaluate five frontier MLLMs and find that all are strongly biased toward the repeated baseline. Mean bias rate reaches 69.78% on card-width perturbations and 80.22% on text font-size perturbations, while mean accuracy is only 21.17% and 7.89%, respectively. Codex-5.3 performs best but still drops from 68.61% accuracy on cards to 13.89% on text, while Flash-3.0 reaches 96.11% bias on text. Noise, subtler perturbations, and boundary positions further increase bias rate. Reasoning analysis further shows that greater reasoning effort correlates with lower bias, yet qualitative evidence reveals that models can identify the anomalous element and still override it with the pattern-consistent answer. Our results identify a concrete failure mode in multimodal code generation and show that its severity is strongly associated with visual saliency

cs.SE↗

arXiv2026

On Automated and Explainable Provenance of AI-Generated Code

Generative AI for code generation has transformed software development, but it has also introduced a critical transparency problem: the origins of AI-generated code are opaque to the developers who use it, the organizations that deploy it, and the compliance professionals responsible for ensuring its legal and quality standards. Existing mitigations flag problematic outputs after the fact without explaining why a model produced them or how future generation could be improved. We present a research vision, grounded in a U.S. NSF-funded research grant, that argues that the next generation of CodeGenAI tools must be built on a foundation of explainable provenance: automated, post-hoc traceability that links generated code back to the prompt components, training data instances, global data features, and internal model components that caused its generation. We grounded this vision in empirical evidence from studies of software developers, model users, and compliance/legal professionals, which show that provenance information is a practical necessity that current tools do not provide. We characterize the problem across four traceability dimensions, outline a research program combining large-scale empirical studies with post-hoc causal and interpretability techniques, and identify the key open challenges that the community must address to realize this vision.

cs.SE↗

arXiv2026

Recovering Fine-Grained Code Change Rationale from Multiple Software Artifacts

Understanding the reasons behind past code changes is critical for refactoring, code review, and debugging. However, code change rationale is often fragmented, inconsistently documented, and scattered across heterogeneous artifacts. We address this challenge with two contributions. First, we conduct an empirical study of nine rationale components from an established taxonomy and trace where they are documented across artifacts associated with 63 commits from five widely used open-source Java projects. Seven components appear in practice, and rationale is highly fragmented: commit messages and pull requests primarily capture GOAL, while NEED and ALTERNATIVE are more often found in issues and pull requests. No single artifact type consistently captures all components, which presents the need for cross-document reasoning. Second, we introduce ARGUS, an LLM-based approach that identifies sentences expressing GOAL, NEED, and ALTERNATIVE across a commit's artifacts and synthesizes them into concise rationale summaries. ARGUS achieved 51.4% overall precision and 93.2% recall for rationale identification and generated summaries rated as accurate relative to reference summaries. Experiments across different LLMs showed varying identification performance but consistently accurate summaries. A user study with 12 Java programmers found these summaries useful for understanding unfamiliar code changes and supporting code review, documentation, debugging, and maintenance.

cs.SE↗

arXiv2026

Rethinking Issue Resolution for AI/ML Systems

We advocate for AI/ML issue resolution frameworks tailored to maintenance workflows and the nature of modern AI/ML systems. Existing issue resolution frameworks largely emerged for traditional software maintenance practices and do not explicitly account for characteristics common in AI/ML systems, such as stochastic behavior, experimentation-driven workflows, and heterogeneous artifacts beyond source code. To identify the unique characteristics of issue resolution in AI/ML systems and motivate the need for tailored frameworks, we conducted a qualitative study of issue resolution workflows documented in 100 issue reports and pull requests across four widely used AI/ML systems: TensorFlow, scikit-learn, MLflow, and AutoGPT. Our findings suggest that issue resolution in AI/ML systems involves: recurring AI/ML-related activities that span multiple resolution stages; iterative experimentation and adaptive verification; and coordinated changes across artifacts such as datasets, prompts, and model configurations. We also observed challenges related to reproducibility, nondeterministic behavior, and artifact coordination. Building on these findings, we present a vision for AI/ML issue resolution frameworks and discuss research directions and tooling support needed to realize this vision.

cs.SE↗

arXiv2026

Automatically Enhancing the Quality of Android App Bug Reports

Most defects in mobile applications are visually observable on the device screen. Since automated mechanisms for detecting and reporting such defects are often unavailable, users, testers, and developers must manually submit bug reports. However, these reports are frequently incomplete, ambiguous, or inaccurate, often lacking the information needed to understand, reproduce, and diagnose defects. This challenge is particularly prominent for UI-centric defects, where the relevant application behavior is difficult for end users to describe precisely. We formulate automatic bug report enhancement as the problem of connecting user-written bug reports with application execution. We present BugScribe, an LLM-powered approach that links bug report information with app-specific UI execution information to infer and generate accurate, complete, and correct Observed Behavior (OB), Expected Behavior (EB), and Steps to Reproduce (S2Rs). BugScribe employs a component-specific grounding strategy that provides the most relevant context to an LLM for generating each bug report component. To support BugScribe's design and evaluation, we develop a bug report quality model and use it to identify the most effective context for each component. We evaluate BugScribe on 48 bug reports from 26 Android applications with manually constructed ground truth. Our results show that BugScribe generates higher-quality bug report components than the original reports and three LLM-based baselines, improving S2R quality by 44.1%--82.3% and OB/EB quality by 3.8%--35.2%.

cs.SE↗

arXiv2026

Verifier Warnings Do Not Improve Comprehensibility Prediction

Proponents of software verification suggest that code simplicity is linked to the effort to verify code, hypothesizing that formal verifiers produce fewer false positive warnings and require less manual intervention when analyzing simpler code. A recent meta-analysis study found empirical support for this hypothesis: a small correlation between the sum of verifier warnings and human-derived code comprehensibility metrics. Based on this finding, we conjectured that using the sum of verifier tool (verifier) warnings to represent program semantic information as an input feature to machine learning (ML) models for code comprehensibility prediction can enhance their performance, when combined with traditional syntactic and developer features. To test this conjecture, we performed a control-treatment experiment incorporating the verifier warning sum feature into machine learning models from the literature, and conducted a comparative analysis of their performance against models trained only on syntactic and developer features. We found no significant difference in the prediction performance of models with and without the warnings feature. Our findings suggest that while a correlation exists, the verifier warning sum offers limited discriminative power: combining syntactic and developer features is just as effective for predicting human-judged code comprehensibility.

cs.SE↗

arXiv2026

"Don't Be Afraid, Just Learn": Insights from Industry Practitioners to Prepare Software Engineers in the Age of Generative AI

Although tension between university curricula and industry expectations has existed in some form for decades, the rapid integration of generative AI (GenAI) tools into software development has recently widened the gap between the two domains. To better understand this disconnect, we surveyed 51 industry practitioners (software developers, technical leads, upper management, \etc) and conducted 11 follow-up interviews focused on hiring practices, required job skills, perceived shortcomings in university curricula, and views on how university learning outcomes can be improved. Our results suggest that GenAI creates demand for new skills (\eg prompting and output evaluation), while strengthening the importance of soft-skills (\eg problem solving and critical thinking) and traditional competencies (\eg architecture design and debugging). We synthesize these findings into actionable recommendations for academia (\eg how to incorporate GenAI into curricula and evaluation redesign). Our work offers empirical guidance to help educators prepare students for modern software engineering environments.

cs.SE↗

arXiv2026

Evaluating Language Model Applications for Identifying Solution-Related Content in Issue Report Discussions

During issue resolution, software developers rely on issue reports to discuss solutions for defects, feature requests, and other changes. These discussions contain proposed solutions--from design changes to code implementations--as well as their evaluations. Locating solution-related content is essential for investigating reopened issues, addressing regressions, reusing solutions, and understanding code change rationale. Manually understanding long discussions to identify such content can be difficult and time-consuming. This paper automates solution identification using language models as supervised classifiers. We investigate three applications--embeddings, prompting, and fine-tuning--across three classifier types: traditional ML models (MLMs), pre-trained language models (PLMs), and large language models (LLMs). Using 356 Mozilla Firefox issues, we created a dataset to train and evaluate six MLMs, four PLMs, and two LLMs across 68 configurations. Results show that MLMs with LLM embeddings outperform TF-IDF features, prompting underperforms, and fine-tuned LLMs achieve the highest performance, with LLAMAft reaching 0.716 F1 score. Ensembles of the best models further improve results (0.737 F1). Misclassifications often arise from misleading clues or missing context, highlighting the need for context-aware classifiers. Models trained on Mozilla transfer to other projects, with a small amount of project-specific data, further enhancing results. This work supports software maintenance, issue understanding, and solution reuse.

cs.SE↗

arXiv2026

Developers' Perspectives on Software Licensing: Current Practices, Challenges, and Tools

Most modern software products incorporate open-source components, requiring development teams to maintain compliance with each component's licenses. Noncompliance can have significant financial, legal, and reputational repercussions. Although some organizations may seek advice from legal practitioners to assist with licensing tasks, developers still play a key role in this process. To this end, it is essential to understand how developers approach licensing compliance tasks, the challenges they encounter, and the tools they use. This work studies these aspects of software licensing practices through a study - conducted by a joint team of software engineering and legal researchers - consisting of a survey with 58 software developers and 7 follow-up interviews. The study resulted in 13 key findings regarding the current state of practice. We discuss the implications of our findings and offer directions for future research, as well as actionable recommendations.

cs.SE↗

arXiv2026

Prompting in Practice: Investigating Software Practitioners' Use of Generative AI Tools

The use of generative AI (GenAI) tools has fundamentally transformed software development. Central to this shift is prompt engineering, the practice of crafting textual prompts to guide GenAI tools in generating useful content. Although prompt engineering has emerged as a critical skill, prior research has focused primarily on cataloging of prompting techniques, with limited attention to how software practitioners employ GenAI within real-world development workflows. To address this gap, this study presents a systematic investigation of practitioners' integration of GenAI tools into software development, drawing on a rigorous survey that examines prompting strategies, conversation patterns, and reliability assessments across core software development tasks. We surveyed 72 software practitioners who actively use GenAI to characterize AI usage patterns throughout the development process. By combining qualitative and quantitative analyses of the survey responses, we identified 13 key findings that describe how prompting is performed in practice. Our study shows that while code generation is nearly universal, proficiency strongly correlates with the use of GenAI for more nuanced tasks such as debugging and code review. Practitioners also tend to favor iterative multi-turn conversations to single-shot prompting. Documentation tasks are perceived as most reliable, while complex code generation and debugging remain major challenges. Our findings provide an empirical view of practitioner practices, ranging from basic code generation to deeper integration of GenAI into development workflows, enabling us to offer recommendations for improving both GenAI tools and the ways practitioners interact with them.

cs.SE↗

arXiv2026

Challenges and Practices in Quantum Software Testing and Debugging: Insights from Practitioners

Quantum software engineering is an emerging discipline with distinct challenges, particularly in testing and debugging. As quantum computing transitions from theory to implementation, developers face issues not present in classical software development, such as probabilistic execution, limited observability, shallow abstractions, and low awareness of quantum-specific tools. To better understand current practices, we surveyed 26 quantum software developers from academia and industry and conducted follow-up interviews focused on testing, debugging, and recurring challenges. All participants reported engaging in testing, with unit testing (88%), regression testing (54%), and acceptance testing (54%) being the most common. However, only 31% reported using quantum-specific testing tools, relying instead on classical and manual methods. Debugging practices were similarly grounded in classical strategies, such as print statements, circuit visualizations, and simulators, which respondents noted do not scale well. The most frequently cited sources of bugs were classical in nature: library updates (81%), developer errors (69%), and compatibility issues (62%)-often worsened by limited abstraction in existing quantum SDKs. These findings highlight the urgent need for better-aligned testing and debugging tools integrated more seamlessly into the workflows of quantum developers. We present these results in detail and offer actionable recommendations grounded in the real-world needs of practitioners.

cs.SE↗

arXiv2025

An Empirical Analysis of Machine Learning Model and Dataset Documentation, Supply Chain, and Licensing Challenges on Hugging Face

The last decade has seen widespread adoption of Machine Learning (ML) components in software systems. This has occurred in nearly every domain, from natural language processing to computer vision. These ML components range from relatively simple neural networks to complex and resource-intensive large language models. However, despite this widespread adoption, little is known about the supply chain relationships that produce these models, which can have implications for compliance and security. In this work, we conducted an extensive analysis of 760,460 models and 175,000 datasets extracted from the popular model-sharing site Hugging Face. First, we evaluate the current state of documentation in the Hugging Face supply chain, report real-world examples of shortcomings, and offer actionable suggestions for improvement. Next, we analyze the underlying structure of the existing supply chain. Finally, we explore the current licensing landscape against what was reported in previous work and discuss the unique challenges posed in this domain. Our results motivate multiple research avenues, including the need for better license management for ML models/datasets, better support for model documentation, and automated inconsistency checking and validation. We make our research infrastructure and dataset available to facilitate future research.

cs.SE↗

arXiv2025

LadyBug: A GitHub Bot for UI-Enhanced Bug Localization in Mobile Apps

This paper introduces LadyBug, a GitHub bot that automatically localizes bugs for Android apps by combining UI interaction information with text retrieval. LadyBug connects to an Android app's GitHub repository, and is triggered when a bug is reported in the corresponding issue tracker. Developers can then record a reproduction trace for the bug on a device or emulator and upload the trace to LadyBug via the GitHub issue tracker. This enables LadyBug to utilize both the text from the original bug description, and UI information from the reproduction trace to accurately retrieve a ranked list of files from the project that most likely contain the reported bug. We empirically evaluated LadyBug using an automated testing pipeline and benchmark called RedWing that contains 80 fully-localized and reproducible bug reports from 39 Android apps. Our results illustrate that LadyBug outperforms text-retrieval-based baselines and that the utilization of UI information leads to a substantial increase in localization accuracy. LadyBug is an open-source tool, available at https://github.com/LadyBugML/ladybug. A video showing the capabilities of Ladybug can be viewed here: https://youtu.be/hI3tzbRK0Cw

cs.SE↗

arXiv2025

Developer Perspectives on Licensing and Copyright Issues Arising from Generative AI for Software Development

Despite the utility that Generative AI (GenAI) tools provide for tasks such as writing code, the use of these tools raises important legal questions and potential risks, particularly those associated with copyright law. As lawmakers and regulators engage with those questions, the views of users can provide relevant perspectives. In this paper, we provide: (1) a survey of 574 developers on the licensing and copyright aspects of GenAI for coding, as well as follow-up interviews; (2) a snapshot of developers' views at a time when GenAI and perceptions of it are rapidly evolving; and (3) an analysis of developers' views, yielding insights and recommendations that can inform future regulatory decisions in this evolving field. Our results show the benefits developers derive from GenAI, how they view the use of AI-generated code as similar to using other existing code, the varied opinions they have on who should own or be compensated for such code, that they are concerned about data leakage via GenAI, and much more, providing organizations and policymakers with valuable insights into how the technology is being used and what concerns stakeholders would like to see addressed.

cs.SE↗

arXiv2025

Decoding the Issue Resolution Process in Practice via Issue Report Analysis: A Case Study of Firefox

Effectively managing and resolving software issues is critical for maintaining and evolving software systems. Development teams often rely on issue trackers and issue reports to track and manage the work needed during issue resolution, ranging from issue reproduction and analysis to solution design, implementation, verification, and deployment. Despite the issue resolution process being generally known in the software engineering community as a sequential list of activities, it is unknown how developers implement this process in practice and how they discuss it in issue reports. This paper aims to enhance our understanding of the issue resolution process implemented in practice by analyzing the issue reports of Mozilla Firefox. We qualitatively and quantitatively analyzed the discussions found in 356 Firefox issue reports, to identify the sequences of stages that developers go through to address various software problems. We analyzed the sequences to identify the overall resolution process at Firefox and derived a catalog of 47 patterns that represent instances of the process. We analyzed the process and patterns across multiple dimensions, including pattern complexity, issue report types, problem categories, and issue resolution times, resulting in various insights about Mozilla's issue resolution process. We discuss these findings and their implications for different stakeholders on how to better assess and improve the issue resolution process.

cs.SE↗

arXiv2025

Bridging the Quantum Divide: Aligning Academic and Industry Goals in Software Engineering

This position paper examines the substantial divide between academia and industry within quantum software engineering. For example, while academic research related to debugging and testing predominantly focuses on a limited subset of primarily quantum-specific issues, industry practitioners face a broader range of practical concerns, including software integration, compatibility, and real-world implementation hurdles. This disconnect mainly arises due to academia's limited access to industry practices and the often confidential, competitive nature of quantum development in commercial settings. As a result, academic advancements often fail to translate into actionable tools and methodologies that meet industry needs. By analyzing discussions within quantum developer forums, we identify key gaps in focus and resource availability that hinder progress on both sides. We propose collaborative efforts aimed at developing practical tools, methodologies, and best practices to bridge this divide, enabling academia to address the application-driven needs of industry and fostering a more aligned, sustainable ecosystem for quantum software development.

cs.SE↗