arXiv subjects

Gregorio Robles

Publications and source records attributed to Gregorio Robles.

At least 19 recordsLinked to original sources

arXiv2026

You can contribute if you... An Empirical Framework of AI Contribution Policies in OSS

Artificial intelligence is reshaping open source software (OSS) contribution by lowering the cost of producing code, documentation, issue reports, and review interactions. This creates opportunities for broader participation, but also disrupts how maintainers assess contributor effort, competence, and accountability. In response, OSS projects are beginning to regulate AI-mediated contribution through contribution guidelines and other project documentation. This paper presents an empirical study of these emerging policies. We analyze project policies on AI-mediated contributions by evaluating their underlying rationales, rules, and expectations. Our analysis shows that these policies seek to protect scarce maintainer attention, preserve accountability, sustain meaningful review interactions, address legal and quality concerns, and maintain pathways for newcomer learning. Based on these findings, we introduce the AI Contribution Governance Framework, which organizes recurring concerns and governance mechanisms across projects. The framework helps OSS communities develop AI contribution policies and provides researchers with a vocabulary for studying how AI is changing collaborative software production.

cs.SE↗

arXiv2026

A CEFR-Inspired Classification Framework with Fuzzy C-Means To Automate Assessment of Programming Skills in Scratch

Context: Schools, training platforms, and technology firms increasingly need to assess programming proficiency at scale with transparent, reproducible methods that support personalized learning pathways. Objective: This study introduces a pedagogical framework for Scratch project assessment, aligned with the Common European Framework of Reference (CEFR), providing universal competency levels for students and teachers alongside actionable insights for curriculum design. Method: We apply Fuzzy C-Means clustering to 2008246 Scratch projects evaluated via Dr.Scratch, implementing an ordinal criterion to map clusters to CEFR levels (A1-C2), and introducing enhanced classification metrics that identify transitional learners, enable continuous progress tracking, and quantify classification certainty to balance automated feedback with instructor review. Impact: The framework enables diagnosis of systemic curriculum gaps-notably a "B2 bottleneck" where only 13.3% of learners reside due to the cognitive load of integrating Logic Synchronization, and Data Representation--while providing certainty--based triggers for human intervention.

cs.CY↗

arXiv2026

Accountability in Open Source Software Ecosystems: Workshop Report

Open source software ecosystems are composed of a variety of stakeholders including but not limited to non-profit organizations, volunteer contributors, users, and corporations. The needs and motivations of these stakeholders are often diverse, unknown, and sometimes even conflicting given the engagement and investment of both volunteers and corporate actors. Given this, it is not clear how open source communities identify and engage with their stakeholders, understand their needs, and hold themselves accountable to those needs. We convened 24 expert scholars and practitioners studying and working with open source software communities for an exploratory workshop discussion on these ideas. The workshop titled "Accountability and Open Source Software Ecosystems" was organized on Oct 14-15 on campus in Carnegie Mellon University, Pittsburgh, PA. The purpose of this in-person workshop was to initiate conversations that explore important and urgent questions related to the role of accountability in open source software ecosystems, and to inspire an exciting research agenda and meaningful stakeholder engagement ideas for practitioners.

cs.SE↗

arXiv2026

Understanding Specification-Driven Code Generation with LLMs: An Empirical Study Design

Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their behavior in structured, specification-driven processes remains poorly understood. This paper presents an empirical study design using CURRANTE, a Visual Studio Code extension that enables a human-in-the-loop workflow for LLM-assisted code generation. The tool guides developers through three sequential stages--Specification, Tests, and Function--allowing them to define requirements, generate and refine test suites, and produce functions that satisfy those tests. Participants will solve medium-difficulty problems from the LiveCodeBench dataset, while the tool records fine-grained interaction logs, effectiveness metrics (e.g., pass rate, all-pass completion), efficiency indicators (e.g., time-to-pass), and iteration behaviors. The study aims to analyze how human intervention in specification and test refinement influences the quality and dynamics of LLM-generated code. The results will provide empirical insights into the design of next-generation development environments that align human reasoning with model-driven code generation.

cs.SE↗

arXiv2025

The Software Infrastructure Attitude Scale (SIAS): A Questionnaire Instrument for Measuring Professionals' Attitudes Toward Technical and Sociotechnical Infrastructure

Context: Recent software engineering (SE) research has highlighted the need for sociotechnical research, implying a demand for customized psychometric scales. Objective: We define the concepts of technical and sociotechnical infrastructure in software engineering, and develop and validate a psychometric scale that measures attitudes toward them. Method: Grounded in theories of infrastructure, attitudes, and prior work on psychometric measurement, we defined the target constructs and generated scale items. The scale was administered to 225 software professionals and evaluated using a split sample. We conducted an exploratory factor analysis (EFA) on one half of the sample to uncover the underlying factor structure and performed a confirmatory factor analysis (CFA) on the other half to validate the structure. Further analyses with the whole sample assessed face, criterion-related, and discriminant validity. Results: EFA supported a two-factor structure (technical and sociotechnical infrastructure), accounting for 65% of the total variance with strong loadings. CFA confirmed excellent model fit. Face and content validity were supported by the item content reflecting cognitive, affective, and behavioral components. Both subscales were correlated with job satisfaction, perceived autonomy, and feedback from the job itself, supporting convergent validity. Regression analysis supported criterion-related validity, while the Heterotrait-Monotrait ratio of correlations (HTMT), the Fornell-Larcker criterion, and model comparison all supported discriminant validity. Discussion: The resulting scale is a valid instrument for measuring attitudes toward technical and sociotechnical infrastructure in software engineering research. Our work contributes to ongoing efforts to integrate psychological measurement rigor into empirical and behavioral software engineering research.

cs.SE↗

arXiv2025

Staying or Leaving? How Job Satisfaction, Embeddedness and Antecedents Predict Turnover Intentions of Software Professionals

Context: Voluntary turnover is common in the software industry, increasing recruitment and onboarding costs and the risk of losing organizational and tacit knowledge. Objective: This study investigates how job satisfaction, work-life balance, job embeddedness, and their antecedents, including job quality, personality traits, attitudes toward technical and sociotechnical infrastructure, and perceptions of organizational justice, relate to software professionals' turnover intentions. Method: We conducted a geographically diverse cross-sectional survey of software professionals (N = 224) and analyzed the data using partial least squares structural equation modeling (PLS-SEM). Our model includes both reflective and formative constructs and tests 15 hypotheses grounded in occupational psychology and software engineering literature. Results: Job satisfaction and embeddedness were significantly negatively associated with software professionals' turnover intentions, while work-life balance showed no direct effect. The strongest antecedents for job satisfaction were work-life balance and job quality, while organizational justice was the strongest predictor of job embeddedness. Discussion: The resulting PLS-SEM model has considerably higher explanatory power for key outcome variables than previous work conducted in the software development context, highlighting the importance of both psychological (e.g., job satisfaction, job embeddedness) and organizational (e.g., organizational justice, job quality) factors in understanding turnover intentions of software professionals. Our results imply that improving job satisfaction and job embeddedness is the key to retaining software professionals. In turn, enhancing job quality, supporting work-life balance, and ensuring high organizational justice can improve job satisfaction and embeddedness, indirectly reducing turnover intentions.

cs.SE↗

arXiv2025

SmartDoc: A Context-Aware Agentic Method Comment Generation Plugin

Context: The software maintenance phase involves many activities such as code refactoring, bug fixing, code review or testing. Program comprehension is key to all these activities, as it demands developers to grasp the knowledge (e.g., implementation details) required to modify the codebase. Methods as main building blocks in a program can offer developers this knowledge source for code comprehension. However, reading entire method statements can be challenging, which necessitates precise and up-to-date comments. Objective: We propose a solution as an IntelliJ IDEA plugin, named SmartDoc, that assists developers in generating context-aware method comments. Method: This plugin acts as an Artificial Intelligence (AI) agent that has its own memory and is augmented by target methods' context. When a request is initiated by the end-user, the method content and all its nested method calls are used in the comment generation. At the beginning, these nested methods are visited and a call graph is generated. This graph is then traversed using depth-first search (DFS), enabling the provision of full-context to enrich Large Language Model (LLM) prompts. Result: The product is a software, as a plugin, developed for Java codebase and installable on IntelliJ IDEA. This plugin can serve concurrently for methods whose comments are being updated , and it shares memory across all flows to avoid redundant calls. o measure the accuracy of this solution, a dedicated test case is run to record SmartDoc generated comments and their corresponding ground truth. For each collected result-set, three metrics are computed, BERTScore, BLEU and ROUGE-1. These metrics will determine how accurate the generated comments are in comparison to the ground truth. Result: The obtained accuracy, in terms of the precision, recall and F1, is promising, and lies in the range of 0.80 to 0.90 for BERTScore.

cs.SE↗

arXiv2025

How Do Code Smells Affect Skill Growth in Scratch Novice Programmers?

Context. Code smells, which are recurring anomalies in design or style, have been extensively researched in professional code. However, their significance in block-based projects created by novices is still largely unknown. Block-based environments such as Scratch offer a unique, data-rich setting to examine how emergent design problems intersect with the cultivation of computational-thinking (CT) skills. Objective. This research explores the connection between CT proficiency and design-level code smells--issues that may hinder software maintenance and evolution--in programs created by Scratch developers. We seek to identify which CT dimensions align most strongly with which code smells and whether task context moderates those associations. Method. A random sample of aprox. 2 million public Scratch projects is mined. Using open-source linters, we extract nine CT scores and 40 code smell indicators from these projects. After rigorous pre-processing, we apply descriptive analytics, robust correlation tests, stratified cross-validation, and exploratory machine-learning models; qualitative spot-checks contextualize quantitative patterns. Impact. The study will deliver the first large-scale, fine-grained map linking specific CT competencies to concrete design flaws and antipatterns. Results are poised to (i) inform evidence-based curricula and automated feedback systems, (ii) provide effect-size benchmarks for future educational interventions, and (iii) supply an open, pseudonymized dataset and reproducible analysis pipeline for the research community. By clarifying how programming habits influence early skill acquisition, the work advances both computing-education theory and practical tooling for sustainable software maintenance and evolution.

cs.SE↗

arXiv2025

Contextual Fairness-Aware Practices in ML: A Cost-Effective Empirical Evaluation

As machine learning (ML) systems become central to critical decision-making, concerns over fairness and potential biases have increased. To address this, the software engineering (SE) field has introduced bias mitigation techniques aimed at enhancing fairness in ML models at various stages. Additionally, recent research suggests that standard ML engineering practices can also improve fairness; these practices, known as fairness-aware practices, have been cataloged across each stage of the ML development life cycle. However, fairness remains context-dependent, with different domains requiring customized solutions. Furthermore, existing specific bias mitigation methods may sometimes degrade model performance, raising ongoing discussions about the trade-offs involved. In this paper, we empirically investigate fairness-aware practices from two perspectives: contextual and cost-effectiveness. The contextual evaluation explores how these practices perform in various application domains, identifying areas where specific fairness adjustments are particularly effective. The cost-effectiveness evaluation considers the trade-off between fairness improvements and potential performance costs. Our findings provide insights into how context influences the effectiveness of fairness-aware practices. This research aims to guide SE practitioners in selecting practices that achieve fairness with minimal performance costs, supporting the development of ethical ML systems.

cs.SE↗

arXiv2024

Towards Identifying Code Proficiency through the Analysis of Python Textbooks

Python, one of the most prevalent programming languages today, is widely utilized in various domains, including web development, data science, machine learning, and DevOps. Recent scholarly efforts have proposed a methodology to assess Python competence levels, similar to how proficiency in natural languages is evaluated. This method involves assigning levels of competence to Python constructs, for instance, placing simple 'print' statements at the most basic level and abstract base classes at the most advanced. The aim is to gauge the level of proficiency a developer must have to understand a piece of source code. This is particularly crucial for software maintenance and evolution tasks, such as debugging or adding new features. For example, in a code review process, this method could determine the competence level required for reviewers. However, categorizing Python constructs by proficiency levels poses significant challenges. Prior attempts, which relied heavily on expert opinions and developer surveys, have led to considerable discrepancies. In response, this paper presents a new approach to identifying Python competency levels through the systematic analysis of introductory Python programming textbooks. By comparing the sequence in which Python constructs are introduced in these textbooks with the current state of the art, we have uncovered notable discrepancies in the order of introduction of Python constructs. Our study underscores a misalignment in the sequences, demonstrating that pinpointing proficiency levels is not trivial. Insights from the study serve as pivotal steps toward reinforcing the idea that textbooks serve as a valuable source for evaluating developers' proficiency, and particularly in terms of their ability to undertake maintenance and evolution tasks.

cs.SE↗

arXiv2024

The Role of Code Proficiency in the Era of Generative AI

At the current pace of technological advancements, Generative AI models, including both Large Language Models and Large Multi-modal Models, are becoming integral to the developer workspace. However, challenges emerge due to the 'black box' nature of many of these models, where the processes behind their outputs are not transparent. This position paper advocates for a 'white box' approach to these generative models, emphasizing the necessity of transparency and understanding in AI-generated code to match the proficiency levels of human developers and better enable software maintenance and evolution. We outline a research agenda aimed at investigating the alignment between AI-generated code and developer skills, highlighting the importance of responsibility, security, legal compliance, creativity, and social value in software development. The proposed research questions explore the potential of white-box methodologies to ensure that software remains an inspectable, adaptable, and trustworthy asset in the face of rapid AI integration, setting a course for research that could shape the role of code proficiency into 2030 and beyond.

cs.SE↗

arXiv2023

Investigating the Impact of Vocabulary Difficulty and Code Naturalness on Program Comprehension

Context: Developers spend most of their time comprehending source code during software development. Automatically assessing how readable and understandable source code is can provide various benefits in different tasks, such as task triaging and code reviews. While several studies have proposed approaches to predict software readability and understandability, most of them only focus on local characteristics of source code. Besides, the performance of understandability prediction is far from satisfactory. Objective: In this study, we aim to assess readability and understandability from the perspective of language acquisition. More specifically, we would like to investigate whether code readability and understandability are correlated with the naturalness and vocabulary difficulty of source code. Method: To assess code naturalness, we adopted the cross-entropy metric, while we use a manually crafted list of code elements with their assigned advancement levels to assess the vocabulary difficulty. We will conduct a statistical analysis to understand their correlations and analyze whether code naturalness and vocabulary difficulty can be used to improve the performance of code readability and understandability prediction methods. The study will be conducted on existing datasets.

cs.SE↗

arXiv2023

The Software Heritage License Dataset (2022 Edition)

Context: When software is released publicly, it is common to include with it either the full text of the license or licenses under which it is published, or a detailed reference to them. Therefore public licenses, including FOSS (free, open source software) licenses, are usually publicly available in source code repositories.Objective: To compile a dataset containing as many documents as possible that contain the text of software licenses, or references to the license terms. Once compiled, characterize the dataset so that it can be used for further research, or practical purposes related to license analysis.Method: Retrieve from Software Heritage-the largest publicly available archive of FOSS source code-all versions of all files whose names are commonly used to convey licensing terms. All retrieved documents will be characterized in various ways, using automated and manual analyses.Results: The dataset consists of 6.9 million unique license files. Additional metadata about shipped license files is also provided, making the dataset ready to use in various contexts, including: file length measures, MIME type, SPDX license (detected using ScanCode), and oldest appearance. The results of a manual analysis of 8102 documents is also included, providing a ground truth for further analysis. The dataset is released as open data as an archive file containing all deduplicated license files, plus several portable CSV files with metadata, referencing files via cryptographic checksums.Conclusions: Thanks to the extensive coverage of Software Heritage, the dataset presented in this paper covers a very large fraction of all software licenses for public code. We have assembled a large body of software licenses, characterized it quantitatively and qualitatively, and validated that it is mostly composed of licensing information and includes almost all known license texts. The dataset can be used to conduct empirical studies on open source licensing, training of automated license classifiers, natural language processing (NLP) analyses of legal texts, as well as historical and phylogenetic studies on FOSS licensing. It can also be used in practice to improve tools detecting licenses in source code.

cs.SE↗

arXiv2023

Open Source Software in the Public Sector: 25 years and still in its infancy

The proliferation of Open Source Software (OSS) adoption and collaboration has surged within industry, resulting in its ubiquitous presence in commercial offerings and shared digital infrastructure. However, in the public sector, both awareness and adoption of OSS is still in its infancy due to a number of obstacles including regulatory, cultural, and capacity-related challenges. This special issue is a call for action, highlighting the necessity for both research and practice to narrow the gap, selectively transfer and adapt existing knowledge, as well as generate new knowledge to enable the public sector to fully harness the potential benefits OSS has to offer.

cs.SE↗

arXiv2023

The Life and Death of Software Ecosystems

Software ecosystems have gained a lot of attention in recent times. Industry and developers gather around technologies and collaborate to their advancement; when the boundaries of such an effort go beyond certain amount of projects, we are witnessing the appearance of Free/Libre and Open Source Software (FLOSS) ecosystems. In this chapter, we explore two aspects that contribute to a healthy ecosystem, related to the attraction (and detraction) and the death of ecosystems. To function and survive, ecosystems need to attract people, get them on-boarded and retain them. In Section One we explore possibilities with provocative research questions for attracting and detracting contributors (and users): the lifeblood of FLOSS ecosystems. Then in the Section Two, we focus on the death of systems, exploring some presumed to be dead systems and their state in the afterlife.

cs.SE↗

arXiv2023

Public Sector Open Source Software Projects -- How is development organized?

Background: Open Source Software (OSS) started as an effort of communities of volunteers, but its practices have been adopted far beyond these initial scenarios. For instance, the strategic use of OSS in industry is constantly growing nowadays in different verticals, including energy, automotive, and health. For the public sector, however, the adoption has lagged behind even if benefits particularly salient in the public sector context such as improved interoperability, transparency, and digital sovereignty have been pointed out. When Public Sector Organisations (PSOs) seek to engage with OSS, this introduces challenges as they often lack the necessary technical capabilities, while also being bound and influenced by regulations and practices for public procurement. Aim: We aim to shed light on how public sector OSS projects, i.e., projects initiated, developed and governed by public sector organizations, are developed and structured. We conjecture, based on the challenges of PSOs, that the way development is organized in these type of projects to a large extent disalign with the commonly adopted bazaar model (popularized by Eric Raymond), which implies that development is carried out collaboratively in a larger community. Method: We plan to contrast public sector OSS projects with a set of earlier reported case studies of bazaar OSS projects, including Mockus et al.'s reporting of the Apache web server and Mozilla browser OSS projects, along with the replications performed on the FreeBSD, JBossAS, JOnAS, and Apache Geronimo OSS projects. To enable comparable results, we will replicate the methodology used by Mockus et al. on a purposefully sampled subset of public sector OSS projects. The subset will be identified and characterized quantitatively by mining relevant software repositories, and qualitatively investigated through interviews with individuals from involved organizations.

cs.SE↗

arXiv2022

Can instability variations warn developers when open-source projects boost?

Although architecture instability has been studied and measured using a variety of metrics, a deeper analysis of which project parts are less stable and how such instability varies over time is still needed. While having more information on architecture instability is, in general, useful for any software development project, it is especially important in Open Source Software (OSS) projects where the supervision of the development process is more difficult to achieve. In particular, we are interested when OSS projects grow from a small controlled environment (i.e., the cathedral phase) to a community-driven project (i.e., the bazaar phase). In such a transition, the project often explodes in terms of software size and number of contributing developers. Hence, the complexity of the newly added features, and the frequency of the commits and files modified may cause significant variations of the instability of the structure of the classes and packages. Consequently, in this registered report we suggest ways to analyze the instability in OSS projects, especially during that sensitive phase where they become community-driven. We intend to suggest ways to predict the evolution of the instability in several OSS projects. Our preliminary results show that it seems possible to provide meaningful estimations that can be useful for OSS teams before a project grows in excess.

cs.SE↗

arXiv2022

pycefr: Python Competency Level through Code Analysis

Python is known to be a versatile language, well suited both for beginners and advanced users. Some elements of the language are easier to understand than others: some are found in any kind of code, while some others are used only by experienced programmers. The use of these elements lead to different ways to code, depending on the experience with the language and the knowledge of its elements, the general programming competence and programming skills, etc. In this paper, we present pycefr, a tool that detects the use of the different elements of the Python language, effectively measuring the level of Python proficiency required to comprehend and deal with a fragment of Python code. Following the well-known Common European Framework of Reference for Languages (CEFR), widely used for natural languages, pycefr categorizes Python code in six levels, depending on the proficiency required to create and understand it. We also discuss different use cases for pycefr: identifying code snippets that can be understood by developers with a certain proficiency, labeling code examples in online resources such as Stackoverflow and GitHub to suit them to a certain level of competency, helping in the onboarding process of new developers in Open Source Software projects, etc. A video shows availability and usage of the tool: https://tinyurl.com/ypdt3fwe.

cs.SE↗