SearcharxivSearch

arXiv subjects

Daniel Graziotin

Publications and source records attributed to Daniel Graziotin.

At least 19 recordsLinked to original sources

GitSkills: A Dataset of Agent Skills on GitHub

An agent skill is a folder containing a $\mathrm{SKILL.md}$ file with instructions for a language-model agent, optionally accompanied by scripts and reference files. The agent loads the skill when it judges that a task matches the skill description. Anthropic introduced the format in October 2025 as an open specification. Nine months later, public GitHub repositories hold millions of skill files. Skills are unlike the artifacts that software engineering researchers usually mine: they are written mainly in natural language, a model selects them probabilistically at run time, and no compiler or type checker verifies the selection. Skills also have no central registry or package manager; developers reuse them by copying folders between repositories. How developers write, reuse, and maintain skills is therefore an empirical question, and no existing dataset records this population. We present GitSkills, a dataset of 3,797,117 $\mathrm{SKILL.md}$ files collected from 282,200 public repositories in July 2026. The dataset retains every file occurrence with its repository, path, and content hash. We group identical files into 1,877,981 distinct contents and enrich one representative per group with the full text, parsed front matter, folder contents, repository metadata, and, for a subset, the commit history of the file. A single self-contained SQLite file supports research on the adoption, reuse, structure, authorship, maintenance, and security of agent skills.

cs.SE

Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review

Background: Large language models (LLMs) are increasingly used to automate code review, but the reasoning behind their decisions remains hard to understand. Developers struggle to assess the validity of LLM-generated reviews, making it difficult to gauge how much trust to place in them. The role of Explainable AI (XAI) in code review and its impact on trust remain underexplored. Objective: We study the influence of XAI on developer trust in AI-assisted code reviews. Method: We conducted a within-subjects user study with 34 participants, comparing three LLM-based code review systems with varying levels of XAI support: Condition A (detailed explanation and review feedback), Condition B (review feedback only), and Condition C (no explanations). Participants reviewed real-world code change requests alongside the AI-generated reviews. We measured trust perceptions, agreement with the AI recommendation, the reasoning given for each decision, and the time taken. Results: The level of explanation significantly influences both trust and agreement with AI recommendations, but in different ways. Full explanations (A) yield the highest perceived trust (M = 3.99/5) but not the highest agreement, whereas moderate explanations (B) achieve the highest agreement (89.22%). This could suggest that more explanation prompts developers to question AI recommendations more frequently. No explanations (C) results in the lowest trust and agreement. Explanation level did not significantly affect review time. The most commonly cited reasons for decisions were code readability and correctness. Conclusion: Incorporating XAI into code review significantly changes trust perceptions and agreement with AI recommendations. These results inform the design and evaluation of trustworthy AI-based code review systems, as well as studies on the human factors of AI-assisted software development.

cs.SE

Attributes to Support the Formulation of Practically Relevant Research Problems in Software Engineering

[Background] A well-formulated research problem is essential for achieving practical relevance in Software Engineering (SE), yet there is a lack of structured guidance in this early phase. [Aims] Our goal is to introduce and evaluate seven attributes identified in the SE literature as relevant for formulating research problems (practical problem, context, implications/impacts, practitioners, evidence, objective, and research questions) in terms of their perceived importance and completeness, and learn how they can be applied. [Method] We conducted a workshop with 42 senior SE researchers during the ISERN 2024 meeting. The seven attributes were presented using a Problem Vision board filled with a research example. Participants discussed attributes in groups, shared written feedback, and individually completed a survey assessing their importance, completeness, and suggestions for improvement. [Results] The findings confirm the importance of the seven attributes in the formulation of industry-oriented research problems. Qualitative feedback illustrated how they can be applied in practice and revealed suggestions to refine them, such as incorporating financial criteria (e.g., ROI) into implications/impacts and addressing feasibility and constraints under evidence. [Conclusion] The results reaffirm the importance of the seven attributes in supporting a reflective and context-aware problem formulation. Adapting their use to specific research contexts can help to improve the alignment between academic research and industry needs.

cs.SE

Large Language Models in Thematic Analysis: Prompt Engineering, Evaluation, and Guidelines for Qualitative Software Engineering Research

As artificial intelligence advances, large language models (LLMs) are entering qualitative research workflows, yet no reproducible methods exist for integrating them into established approaches like thematic analysis (TA), one of the most common qualitative methods in software engineering research. Moreover, existing studies lack systematic evaluation of LLM-generated qualitative outputs against established quality criteria. We designed and iteratively refined prompts for Phases 2-5 of Braun and Clarke's reflexive TA, then tested outputs from multiple LLMs against codes and themes produced by experienced researchers. Using 15 interviews on software engineers' well-being, we conducted blind evaluations with four expert evaluators who applied rubrics derived directly from Braun and Clarke's quality criteria. Evaluators preferred LLM-generated codes 61% of the time, finding them analytically useful for answering the research question. However, evaluators also identified limitations: LLMs fragmented data unnecessarily, missed latent interpretations, and sometimes produced themes with unclear boundaries. Our contributions are threefold. First, a reproducible approach integrating refined, documented prompts with an evaluation framework to operationalize Braun and Clarke's reflexive TA. Second, an empirical comparison of LLM- and human-generated codes and themes in software engineering data. Third, guidelines for integrating LLMs into qualitative analysis while preserving methodological rigour, clarifying when and how LLMs can assist effectively and when human interpretation remains essential.

cs.SE

ReqBrain: Task-Specific Instruction Tuning of LLMs for AI-Assisted Requirements Generation

Requirements elicitation and specification remains a labor-intensive, manual process prone to inconsistencies and gaps, presenting a significant challenge in modern software engineering. Emerging studies underscore the potential of employing large language models (LLMs) for automated requirements generation to support requirements elicitation and specification; however, it remains unclear how to implement this effectively. In this work, we introduce ReqBrain, an Al-assisted tool that employs a fine-tuned LLM to generate authentic and adequate software requirements. Software engineers can engage with ReqBrain through chat-based sessions to automatically generate software requirements and categorize them by type. We curated a high-quality dataset of ISO 29148-compliant requirements and fine-tuned five 7B-parameter LLMs to determine the most effective base model for ReqBrain. The top-performing model, Zephyr-7b-beta, achieved 89.30\% Fl using the BERT score and a FRUGAL score of 91.20 in generating authentic and adequate requirements. Human evaluations further confirmed ReqBrain's effectiveness in generating requirements. Our findings suggest that generative Al, when fine-tuned, has the potential to improve requirements elicitation and specification, paving the way for future extensions into areas such as defect identification, test case generation, and agile user story creation.

cs.SE

Introducing Repository Stability

Drawing from engineering systems and control theory, we introduce a framework to understand repository stability, which is a repository activity capacity to return to equilibrium following disturbances - such as a sudden influx of bug reports, key contributor departures, or a spike in feature requests. The framework quantifies stability through four indicators: commit patterns, issue resolution, pull request processing, and community engagement, measuring development consistency, problem-solving efficiency, integration effectiveness, and sustainable participation, respectively. These indicators are synthesized into a Composite Stability Index (CSI) that provides a normalized measure of repository health proxied by its stability. Finally, the framework introduces several important theoretical properties that validate its usefulness as a measure of repository health and stability. At a conceptual phase and open to debate, our work establishes mathematical criteria for evaluating repository stability and proposes new ways to understand sustainable development practices. The framework bridges control theory concepts with modern collaborative software development, providing a foundation for future empirical validation.

cs.SE

Mining a Decade of Event Impacts on Contributor Dynamics in Ethereum: A Longitudinal Study

We analyze developer activity across 10 major Ethereum repositories (totaling 129884 commits, 40550 issues) spanning 10 years to examine how events such as technical upgrades, market events, and community decisions impact development. Through statistical, survival, and network analyses, we find that technical events prompt increased activity before the event, followed by reduced commit rates afterwards, whereas market events lead to more reactive development. Core infrastructure repositories like Go-Ethereum exhibit faster issue resolution compared to developer tools, and technical events enhance core team collaboration. Our findings show how different types of events shape development dynamics, offering insights for project managers and developers in maintaining development momentum through major transitions. This work contributes to understanding the resilience of development communities and their adaptation to ecosystem changes.

cs.SE

Towards Emotionally Intelligent Software Engineers: Understanding Students' Self-Perceptions After a Cooperative Learning Experience

[Background] Emotional Intelligence (EI) can impact Software Engineering (SE) outcomes through improved team communication, conflict resolution, and stress management. SE workers face increasing pressure to develop both technical and interpersonal skills, as modern software development emphasizes collaborative work and complex team interactions. Despite EI's documented importance in professional practice, SE education continues to prioritize technical knowledge over emotional and social competencies. [Objective] This paper analyzes SE students' self-perceptions of their EI after a two-month cooperative learning project, using Mayer and Salovey's four-ability model to examine how students handle emotions in collaborative development. [Method] We conducted a case study with 29 SE students organized into four squads within a project-based learning course, collecting data through questionnaires and focus groups that included brainwriting and sharing circles, then analyzing the data using descriptive statistics and open coding. [Results] Students demonstrated stronger abilities in managing their own emotions compared to interpreting others' emotional states. Despite limited formal EI training, they developed informal strategies for emotional management, including structured planning and peer support networks, which they connected to improved productivity and conflict resolution. [Conclusion] This study shows how SE students perceive EI in a collaborative learning context and provides evidence-based insights into the important role of emotional competencies in SE education.

cs.SE

Overwhelmed Software Developers

We have conducted a qualitative psychology study to explore the experience of feeling overwhelmed in the realm of software development. Through the candid confessions of two participants who have recently faced overwhelming challenges, we have identified seven distinct categories: communication-induced, disturbance-related, organizational, variety, technical, temporal, and positive overwhelm. While most types of overwhelm tend to deteriorate productivity and increase stress levels, developers sometimes perceive overwhelm as a catalyst for heightened focus, self-motivation, and productivity. Stress was often found to be a common companion of overwhelm. Our findings align with previous studies conducted in diverse disciplines. However, we believe that software developers possess unique traits that may enable them to navigate through the storm of overwhelm more effectively.

cs.SE

Requirements for Organizational Resilience: Engineering Developer Happiness

Can the right requirements boost developer satisfaction and happiness? We believe they can. In keeping with this issue's theme, "Well-Being for Resilience: Developers Thrive," we discuss the connection between the three keywords, well-being, resilience, and thriving. How could requirements engineering foster these qualities? While there hasn't been much research on this topic, we see opportunities for future work. Let's initiate the discussion!

cs.SE

Understanding and measuring software engineer behavior: What can we learn from the behavioral sciences?

This paper explores the intricate challenge of understanding and measuring software engineer behavior. More specifically, we revolve around a central question: How can we enhance our understanding of software engineer behavior? Grounded in the nuanced complexities addressed within Behavioral Software Engineering (BSE), we advocate for holistic methods that integrate quantitative measures, such as psychometric instruments, and qualitative data from diverse sources. Furthermore, we delve into the relevance of this challenge within national and international contexts, highlighting the increasing interest in understanding software engineer behavior. Real-world initiatives and academic endeavors are also examined to underscore the potential for advancing this research agenda and, consequently, refining software engineering practices based on behavioral aspects. Lastly, this paper addresses different ways to evaluate the progress of this challenge by leveraging methodological skills derived from behavioral sciences, ultimately contributing to a deeper understanding of software engineer behavior and software engineering practices.

cs.SE

Overwhelmed software developers: An Interpretative Phenomenological Analysis

In this paper, we report on an Interpretive Phenomenological Analysis (IPA) study on experiencing overwhelm in a software development context. The objectives of our study are, hence, to understand the experiences developers have when being overwhelmed, how this impacts their productivity and which role stress plays in the process. To this end, we interviewed two software developers who have experienced overwhelm recently. Throughout a qualitative analysis of the shared experiences, we uncover seven categories of overwhelm (communication, disturbance, organizational, variety, technical, temporal, and positive overwhelm). While the first six themes all are related to negative outcomes, including low productivity and stress, the participants reported that overwhelm can sometimes be experienced to be positive and pleasant, and it can increase their mental focus, self ambition, and productivity. Stress was the most mentioned feeling experienced when overwhelmed. Our findings, for the most, are along the same direction of similar studies from other disciplines and with other participants. However, there may be unique attributes to software developers that mitigate the negative experiences of overwhelm.

cs.SE

Qualitative software engineering research -- reflections and guidelines

Researchers are increasingly recognizing the importance of human aspects in software development. Since qualitative methods are used to explore human behavior in-depth, we believe that studies using such methods will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and the knowledge on how to use them like in social sciences. The purpose of this study was to extend the software engineering community's current body of knowledge regarding available qualitative methods and their quality assurance frameworks, and to provide recommendations and guidelines for their use. With the support of an epistemological argument and a survey of the literature, we suggest that future research would benefit from (1) utilizing a broader set of research methods, (2) more strongly emphasizing reflexivity, and (3) employing qualitative guidelines and quality criteria. We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research, namely interpretative phenomenological analysis, narrative analysis, and discourse analysis. Furthermore, we discuss the meaning of reflexivity in relation to the software engineering context and suggest means of fostering it. Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.

cs.SE

Psychometric Instruments in Software Engineering Research on Personality: Status Quo After Fifty Years

Context: Although software development is a human activity, Software Engineering (SE) research has focused mostly on processes and tools, making human factors underrepresented. This kind of research may be improved using knowledge from human-focused disciplines. An example of missed opportunities is how SE employs psychometric instruments. Objective: Provide an overview of psychometric instruments in SE research regarding personality and provide recommendations for adopting them. Method: We conducted a systematic mapping to build an overview of instruments used within SE for assessing personality and reviewed their use from a multidisciplinary perspective of SE and social science. Results: We contribute with a secondary study covering fifty years of research (1970 to 2020). One of the most adopted instruments (MBTI) faces criticism within social sciences, and we identified discrepancies between its application and existing recommendations. We emphasize that several instruments refer to the Five-Factor Model, which despite its relevance in social sciences, has no specific advice for its application within SE. We discuss general advice for its proper application. Conclusion: The findings show that the adoption of psychometric instruments regarding personality in SE needs to be improved, ideally with the support of social science researchers. We believe that the review presented in this study can help to understand limitations and to evolve in this direction.

cs.SE

Evidence Profiles for Validity Threats in Program Comprehension Experiments

Searching for clues, gathering evidence, and reviewing case files are all techniques used by criminal investigators to draw sound conclusions and avoid wrongful convictions. Similarly, in software engineering (SE) research, we can develop sound methodologies and mitigate threats to validity by basing study design decisions on evidence. Echoing a recent call for the empirical evaluation of design decisions in program comprehension experiments, we conducted a 2-phases study consisting of systematic literature searches, snowballing, and thematic synthesis. We found out (1) which validity threat categories are most often discussed in primary studies of code comprehension, and we collected evidence to build (2) the evidence profiles for the three most commonly reported threats to validity. We discovered that few mentions of validity threats in primary studies (31 of 409) included a reference to supporting evidence. For the three most commonly mentioned threats, namely the influence of programming experience, program length, and the selected comprehension measures, almost all cited studies (17 of 18) did not meet our criteria for evidence. We show that for many threats to validity that are currently assumed to be influential across all studies, their actual impact may depend on the design and context of each specific study. Researchers should discuss threats to validity within the context of their particular study and support their discussions with evidence. The present paper can be one resource for evidence, and we call for more meta-studies of this type to be conducted, which will then inform design decisions in primary studies. Further, although we have applied our methodology in the context of program comprehension, our approach can also be used in other SE research areas to enable evidence-based experiment design decisions and meaningful discussions of threats to validity.

cs.SE

How Are Communication Channels on GitHub Presented to Their Intended Audience? -- A Thematic Analysis

Communication is essential in software development, and even more in distributed settings. Communication activities need to be organized and coordinated to defend against the threat of productivity losses, increases in cognitive load, and stress among team members. With a plethora of communication channels that were identified by previous research in open-source projects, there is a need to explore organizational issues in how these communication channels are introduced, explained, and motivated for use among all project members. In this study, we wanted to understand which communication channels are used in GitHub projects and how they are presented to the GitHub project audience. We employed thematic analysis to analyze 151 artifacts in 90 GitHub projects. Our results revealed 32 unique communications channels that can be divided into nine different types. Projects mostly provide channels of different types, but for some types (e.g., chat) it is common to provide several channels. Maintainers are aware that channels have different properties and help the developers to decide which channel should be used in which case. However, this is not true for all projects, and often we have not found any explicit reasons why maintainers chose to provide one channel over another. Different channels can be used for different purposes and have different affordances, so maintainers have to decide wisely which channels they want to provide and make clear which channel should be used in which case. Otherwise, developers might feel overwhelmed of too many channels and information can get fragmented over multiple channels.

cs.SE

Text and Team: What Article Metadata Characteristics Drive Citations in Software Engineering?

Context: Citations are a key measure of scientific performance in most fields, including software engineering. However, there is limited research that studies which characteristics of articles' metadata (title, abstract, keywords, and author list) are driving citations in this field. Objective: In this study, we propose a simple theoretical model for how citations come to be with respect to article metadata, we hypothesize theoretical linkages between metadata characteristics and citations of articles, and we empirically test these hypotheses. Method: We use multiple regression analyses to examine a data set comprising the titles, abstracts, keywords, and authors of 16,131 software engineering articles published between 1990 and 2020 in 20 highly influential software engineering venues. Results: We find that number of authors, number of keywords, number of question marks and dividers in the title, number of acronyms, abstract length, abstract propositional idea density, and corresponding authors in the core Anglosphere are significantly related to citations. Conclusion: Various characteristics of articles' metadata are linked to the frequency with which the corresponding articles are cited. These results partially confirm and partially go counter to prior findings in software engineering and other disciplines.

cs.SE

Anchoring Code Understandability Evaluations Through Task Descriptions

In code comprehension experiments, participants are usually told at the beginning what kind of code comprehension task to expect. Describing experiment scenarios and experimental tasks will influence participants in ways that are sometimes hard to predict and control. In particular, describing or even mentioning the difficulty of a code comprehension task might anchor participants and their perception of the task itself. In this study, we investigated in a randomized, controlled experiment with 256 participants (50 software professionals and 206 computer science students) whether a hint about the difficulty of the code to be understood in a task description anchors participants in their own code comprehensibility ratings. Subjective code evaluations are a commonly used measure for how well a developer in a code comprehension study understood code. Accordingly, it is important to understand how robust these measures are to cognitive biases such as the anchoring effect. Our results show that participants are significantly influenced by the initial scenario description in their assessment of code comprehensibility. An initial hint of hard to understand code leads participants to assess the code as harder to understand than participants who received no hint or a hint of easy to understand code. This affects students and professionals alike. We discuss examples of design decisions and contextual factors in the conduct of code comprehension experiments that can induce an anchoring effect, and recommend the use of more robust comprehension measures in code comprehension studies to enhance the validity of results.

cs.SE