arXiv subjects

Igor Steinmacher

Publications and source records attributed to Igor Steinmacher.

At least 37 records · Page 2Linked to original sources

arXiv2025

SkillScope: A Tool to Predict Fine-Grained Skills Needed to Solve Issues on GitHub

New contributors often struggle to find tasks that they can tackle when onboarding onto a new Open Source Software (OSS) project. One reason for this difficulty is that issue trackers lack explanations about the knowledge or skills needed to complete a given task successfully. These explanations can be complex and time-consuming to produce. Past research has partially addressed this problem by labeling issues with issue types, issue difficulty level, and issue skills. However, current approaches are limited to a small set of labels and lack in-depth details about their semantics, which may not sufficiently help contributors identify suitable issues. To surmount this limitation, this paper explores large language models (LLMs) and Random Forest (RF) to predict the multilevel skills required to solve the open issues. We introduce a novel tool, SkillScope, which retrieves current issues from Java projects hosted on GitHub and predicts the multilevel programming skills required to resolve these issues. In a case study, we demonstrate that SkillScope could predict 217 multilevel skills for tasks with 91% precision, 88% recall, and 89% F-measure on average. Practitioners can use this tool to better delegate or choose tasks to solve in OSS projects.

cs.SE↗

arXiv2025

The Multifaceted Nature of Mentoring in OSS: Strategies, Qualities, and Ideal Outcomes

Mentorship in open source software (OSS) is a vital, multifaceted process that includes onboarding newcomers, fostering skill development, and enhancing community building. This study examines task-focused mentoring strategies that help mentees complete their tasks and the ideal personal qualities and outcomes of good mentorship in OSS communities. We conducted two surveys to gather contributor perceptions: the first survey, with 70 mentors, mapped 17 mentoring challenges to 21 strategies that help support mentees. The second survey, with 85 contributors, assessed the importance of personal qualities and ideal mentorship outcomes. Our findings not only provide actionable strategies to help mentees overcome challenges and become successful contributors but also guide current and future mentors and OSS communities in understanding the personal qualities that are the cornerstone of good mentorship and the outcomes that mentor-mentee pairs should aspire to achieve.

cs.HC↗

arXiv2024

Insights from the Frontline: GenAI Utilization Among Software Engineering Students

Generative AI (genAI) tools (e.g., ChatGPT, Copilot) have become ubiquitous in software engineering (SE). As SE educators, it behooves us to understand the consequences of genAI usage among SE students and to create a holistic view of where these tools can be successfully used. Through 16 reflective interviews with SE students, we explored their academic experiences of using genAI tools to complement SE learning and implementations. We uncover the contexts where these tools are helpful and where they pose challenges, along with examining why these challenges arise and how they impact students. We validated our findings through member checking and triangulation with instructors. Our findings provide practical considerations of where and why genAI should (not) be used in the context of supporting SE students.

cs.HC↗

arXiv2024

What Guides Our Choices? Modeling Developers' Trust and Behavioral Intentions Towards GenAI

Generative AI (genAI) tools, such as ChatGPT or Copilot, are advertised to improve developer productivity and are being integrated into software development. However, misaligned trust, skepticism, and usability concerns can impede the adoption of such tools. Research also indicates that AI can be exclusionary, failing to support diverse users adequately. One such aspect of diversity is cognitive diversity -- variations in users' cognitive styles -- that leads to divergence in perspectives and interaction styles. When an individual's cognitive style is unsupported, it creates barriers to technology adoption. Therefore, to understand how to effectively integrate genAI tools into software development, it is first important to model what factors affect developers' trust and intentions to adopt genAI tools in practice? We developed a theoretically grounded statistical model to (1) identify factors that influence developers' trust in genAI tools and (2) examine the relationship between developers' trust, cognitive styles, and their intentions to use these tools in their work. We surveyed software developers (N=238) at two major global tech organizations: GitHub Inc. and Microsoft; and employed Partial Least Squares-Structural Equation Modeling (PLS-SEM) to evaluate our model. Our findings reveal that genAI's system/output quality, functional value, and goal maintenance significantly influence developers' trust in these tools. Furthermore, developers' trust and cognitive styles influence their intentions to use these tools in their work. We offer practical suggestions for designing genAI tools for effective use and inclusive user experience.

cs.HC↗

arXiv2024

Investigating the Impact of Interpersonal Challenges on Feeling Welcome in OSS

The sustainability of open source software (OSS) projects hinges on contributor retention. Interpersonal challenges can inhibit a feeling of welcomeness among contributors, particularly from underrepresented groups, which impacts their decision to continue with the project. How much this impact is, varies among individuals, underlining the importance of a thorough understanding of their effects. Here, we investigate the effects of interpersonal challenges on the sense of welcomeness among diverse populations within OSS, through the diversity lenses of gender, race, and (dis)ability. We analyzed the large-scale Linux Foundation Diversity and Inclusion survey (n = 706) to model a theoretical framework linking interpersonal challenges with the sense of welcomeness through Structural Equation Models Partial Least Squares (PLS-SEM). We then examine the model to identify the impact of these challenges on different demographics through Multi-Group Analysis (MGA). Finally, we conducted a regression analysis to investigate how differently people from different demographics experience different types of interpersonal challenges. Our findings confirm the negative association between interpersonal challenges and the feeling of welcomeness in OSS, with this relationship being more pronounced among gender minorities and people with disabilities. We found that different challenges have unique impacts on how people feel welcomed, with variations across gender, race, and disability groups. We also provide evidence that people from gender minorities and with disabilities are more likely to experience interpersonal challenges than their counterparts, especially when we analyze stalking, sexual harassment, and doxxing. Our insights benefit OSS communities, informing potential strategies to improve the landscape of interpersonal relationships, ultimately fostering more inclusive and welcoming communities.

cs.SE↗

arXiv2024

Software Solutions for Newcomers' Onboarding in Software Projects: A Systematic Literature Review

[Context] Newcomers joining an unfamiliar software project face numerous barriers; therefore, effective onboarding is essential to help them engage with the team and develop the behaviors, attitudes, and skills needed to excel in their roles. However, onboarding can be a lengthy, costly, and error-prone process. Software solutions can help mitigate these barriers and streamline the process without overloading senior members. [Objective] This study aims to identify the state-of-the-art software solutions for onboarding newcomers. [Method] We conducted a systematic literature review (SLR) to answer six research questions. [Results] We analyzed 32 studies about software solutions for onboarding newcomers and yielded several key findings: (1) a range of strategies exists, with recommendation systems being the most prevalent; (2) most solutions are web-based; (3) solutions target a variety of onboarding aspects, with a focus on process; (4) many onboarding barriers remain unaddressed by existing solutions; (5) laboratory experiments are the most commonly used method for evaluating these solutions; and (6) diversity and inclusion aspects primarily address experience level. [Conclusion] We shed light on current technological support and identify research opportunities to develop more inclusive software solutions for onboarding. These insights may also guide practitioners in refining existing platforms and onboarding programs to promote smoother integration of newcomers into software projects.

cs.SE↗

arXiv2024

Game Elements to Engage Students Learning the Open Source Software Contribution Process

Contributing to OSS projects can help students to enhance their skills and expand their professional networks. However, novice contributors often feel discouraged due to various barriers. Gamification techniques hold the potential to foster engagement and facilitate the learning process. Nevertheless, it is unknown which game elements are effective in this context. This study explores students' perceptions of gamification elements to inform the design of a gamified learning environment. We surveyed 115 students and segmented the analysis from three perspectives: (1) cognitive styles, (2) gender, and (3) ethnicity (Hispanic/LatinX and Non-Hispanic/LatinX). The results showed that Quest, Point, Stats, and Badge are favored elements, while competition and pressure-related are less preferred. Across cognitive styles (persona), gender, and ethnicity, we could not observe any statistical differences, except for Tim's GenderMag persona, which demonstrated a higher preference for storytelling. Conversely, Hispanic/LatinX participants showed a preference for the Choice element. These results can guide tool builders in designing effective gamified learning environments focused on the OSS contributions process.

cs.SE↗

arXiv2024

Anticipating User Needs: Insights from Design Fiction on Conversational Agents for Computational Thinking

Computational thinking, and by extension, computer programming, is notoriously challenging to learn. Conversational agents and generative artificial intelligence (genAI) have the potential to facilitate this learning process by offering personalized guidance, interactive learning experiences, and code generation. However, current genAI-based chatbots focus on professional developers and may not adequately consider educational needs. Involving educators in conceiving educational tools is critical for ensuring usefulness and usability. We enlisted nine instructors to engage in design fiction sessions in which we elicited abilities such a conversational agent supported by genAI should display. Participants envisioned a conversational agent that guides students stepwise through exercises, tuning its method of guidance with an awareness of the educational background, skills and deficits, and learning preferences. The insights obtained in this paper can guide future implementations of tutoring conversational agents oriented toward teaching computational thinking and computer programming.

cs.HC↗

arXiv2024

Gamification in Software Engineering Education: a Tertiary Study

As the significance of Software Engineering (SE) professionals continues to grow in the industry, the adoption of gamification techniques for training purposes has gained traction due to its potential to enhance class appeal through game-derived elements. This paper presents a tertiary study investigating the application of gamification in Software Engineering (SE) education. The study was conducted in response to recent systematic literature reviews and mappings on the topic. The findings reveal that the areas of SE most frequently gamified are Software Testing and Software Quality, with competition and cooperation being the most commonly utilized gamification elements. Additionally, the majority of studies focus on structural gamification, where game elements are employed to modify the learning environment without altering the content. The results demonstrate the potential of gamification to improve students' engagement and motivation throughout the SE learning process, while also impacting other aspects such as performance improvement, skill development, and fostering good SE practices. However, caution is advised as unplanned and incorrectly applied gamification measures may lead to significant declines in performance and motivation. (English Version of the paper in Portuguese available here: HTTP://doi.org/10.1145/3613372.3614193

cs.SE↗

arXiv2024

Towards the First Code Contribution: Processes and Information Needs

Newcomers to a software project must overcome many barriers before they can successfully place their first code contribution, and they often struggle to find information that is relevant to them. In this work, we argue that much of the information needed by newcomers already exists, albeit scattered among many different sources, and that many barriers can be addressed by automatically identifying, extracting, generating, summarizing, and presenting documentation that is specifically aimed and customized for newcomers. To gain a detailed understanding of the processes followed by newcomers and their information needs before making their first code contribution, we conducted an empirical study. Based on a survey with about 100 practitioners, grounded theory analysis, and validation interviews, we contribute a 16-step model for the processes followed by newcomers to a software project and we identify relevant information, along with individual and project characteristics that influence the relevancy of information types and sources. Our findings form an essential step towards automated tool support that provides relevant information to project newcomers in each step of their contribution processes.

cs.SE↗

arXiv2024

Applying Large Language Models API to Issue Classification Problem

Effective prioritization of issue reports is crucial in software engineering to optimize resource allocation and address critical problems promptly. However, the manual classification of issue reports for prioritization is laborious and lacks scalability. Alternatively, many open source software (OSS) projects employ automated processes for this task, albeit relying on substantial datasets for adequate training. This research seeks to devise an automated approach that ensures reliability in issue prioritization, even when trained on smaller datasets. Our proposed methodology harnesses the power of Generative Pre-trained Transformers (GPT), recognizing their potential to efficiently handle this task. By leveraging the capabilities of such models, we aim to develop a robust system for prioritizing issue reports accurately, mitigating the necessity for extensive training data while maintaining reliability. In our research, we have developed a reliable GPT-based approach to accurately label and prioritize issue reports with a reduced training dataset. By reducing reliance on massive data requirements and focusing on few-shot fine-tuning, our methodology offers a more accessible and efficient solution for issue prioritization in software engineering. Our model predicted issue types in individual projects up to 93.2% in precision, 95% in recall, and 89.3% in F1-score.

cs.SE↗

arXiv2023

How Far Are We? The Triumphs and Trials of Generative AI in Learning Software Engineering

Conversational Generative AI (convo-genAI) is revolutionizing Software Engineering (SE) as engineers and academics embrace this technology in their work. However, there is a gap in understanding the current potential and pitfalls of this technology, specifically in supporting students in SE tasks. In this work, we evaluate through a between-subjects study (N=22) the effectiveness of ChatGPT, a convo-genAI platform, in assisting students in SE tasks. Our study did not find statistical differences in participants' productivity or self-efficacy when using ChatGPT as compared to traditional resources, but we found significantly increased frustration levels. Our study also revealed 5 distinct faults arising from violations of Human-AI interaction guidelines, which led to 7 different (negative) consequences on participants.

cs.SE↗

arXiv2023

Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomes

In the rapidly advancing field of artificial intelligence, software development has emerged as a key area of innovation. Despite the plethora of general-purpose AI assistants available, their effectiveness diminishes in complex, domain-specific scenarios. Noting this limitation, both the academic community and industry players are relying on contextualized coding AI assistants. These assistants surpass general-purpose AI tools by integrating proprietary, domain-specific knowledge, offering precise and relevant solutions. Our study focuses on the initial experiences of 62 participants who used a contextualized coding AI assistant -- named StackSpot AI -- in a controlled setting. According to the participants, the assistants' use resulted in significant time savings, easier access to documentation, and the generation of accurate codes for internal APIs. However, challenges associated with the knowledge sources necessary to make the coding assistant access more contextual information as well as variable responses and limitations in handling complex codes were observed. The study's findings, detailing both the benefits and challenges of contextualized AI assistants, underscore their potential to revolutionize software development practices, while also highlighting areas for further refinement.

cs.SE↗

arXiv2023

Can AI Serve as a Substitute for Human Subjects in Software Engineering Research?

Research within sociotechnical domains, such as Software Engineering, fundamentally requires a thorough consideration of the human perspective. However, traditional qualitative data collection methods suffer from challenges related to scale, labor intensity, and the increasing difficulty of participant recruitment. This vision paper proposes a novel approach to qualitative data collection in software engineering research by harnessing the capabilities of artificial intelligence (AI), especially large language models (LLMs) like ChatGPT. We explore the potential of AI-generated synthetic text as an alternative source of qualitative data, by discussing how LLMs can replicate human responses and behaviors in research settings. We examine the application of AI in automating data collection across various methodologies, including persona-based prompting for interviews, multi-persona dialogue for focus groups, and mega-persona responses for surveys. Additionally, we discuss the prospective development of new foundation models aimed at emulating human behavior in observational studies and user evaluations. By simulating human interaction and feedback, these AI models could offer scalable and efficient means of data generation, while providing insights into human attitudes, experiences, and performance. We discuss several open problems and research opportunities to implement this vision and conclude that while AI could augment aspects of data gathering in software engineering research, it cannot replace the nuanced, empathetic understanding inherent in human subjects in some cases, and an integrated approach where both AI and human-generated data coexist will likely yield the most effective outcomes.

cs.SE↗

arXiv2023

Barriers and Self-Efficacy: A Large-Scale Study on the Impact of OSS Courses on Student Perceptions

Open source software (OSS) development offers a unique opportunity for students in Software Engineering to experience and participate in large-scale software development, however, the impact of such courses on students' self-efficacy and the challenges faced by students are not well understood. This paper aims to address this gap by analyzing data from multiple instances of OSS development courses at universities in different countries and reporting on how students' self-efficacy changed as a result of taking the course, as well as the barriers and challenges faced by students.

cs.SE↗

arXiv2023

Tag that issue: Applying API-domain labels in issue tracking systems

Labeling issues with the skills required to complete them can help contributors to choose tasks in Open Source Software projects. However, manually labeling issues is time-consuming and error-prone, and current automated approaches are mostly limited to classifying issues as bugs/non-bugs. We investigate the feasibility and relevance of automatically labeling issues with what we call "API-domains," which are high-level categories of APIs. Therefore, we posit that the APIs used in the source code affected by an issue can be a proxy for the type of skills (e.g., DB, security, UI) needed to work on the issue. We ran a user study (n=74) to assess API-domain labels' relevancy to potential contributors, leveraged the issues' descriptions and the project history to build prediction models, and validated the predictions with contributors (n=20) of the projects. Our results show that (i) newcomers to the project consider API-domain labels useful in choosing tasks, (ii) labels can be predicted with a precision of 84% and a recall of 78.6% on average, (iii) the results of the predictions reached up to 71.3% in precision and 52.5% in recall when training with a project and testing in another (transfer learning), and (iv) project contributors consider most of the predictions helpful in identifying needed skills. These findings suggest our approach can be applied in practice to automatically label issues, assisting developers in finding tasks that better match their skills.

cs.SE↗

arXiv2023

GiveMeLabeledIssues: An Open Source Issue Recommendation System

Developers often struggle to navigate an Open Source Software (OSS) project's issue-tracking system and find a suitable task. Proper issue labeling can aid task selection, but current tools are limited to classifying the issues according to their type (e.g., bug, question, good first issue, feature, etc.). In contrast, this paper presents a tool (GiveMeLabeledIssues) that mines project repositories and labels issues based on the skills required to solve them. We leverage the domain of the APIs involved in the solution (e.g., User Interface (UI), Test, Databases (DB), etc.) as a proxy for the required skills. GiveMeLabeledIssues facilitates matching developers' skills to tasks, reducing the burden on project maintainers. The tool obtained a precision of 83.9% when predicting the API domains involved in the issues. The replication package contains instructions on executing the tool and including new projects. A demo video is available at https://www.youtube.com/watch?v=ic2quUue7i8

cs.SE↗

arXiv2023

Rules of Engagement: Why and How Companies Participate in OSS

Company engagement in open source (OSS) is now the new norm. From large technology companies to startups, companies are participating in the OSS ecosystem by open-sourcing their technology, sponsoring projects through funding or paid developer time. However, our understanding of the OSS ecosystem is rooted in the 'old world' model where individual contributors sustain OSS projects. In this work, we create a more comprehensive understanding of the hybrid OSS landscape by investigating what motivates companies to contribute and how they contribute to OSS. We conducted interviews with 20 participants who have different roles (e.g., CEO, OSPO Lead, Ecosystem Strategist) at 17 different companies of different sizes from large companies (e.g. Microsoft, RedHat, Google, Spotify) to startups. Data from semi-structured interviews reveal that company motivations can be categorized into four levels (Founders' Vision, Reputation, Business Advantage, and Reciprocity) and companies participate through different mechanisms (e.g., Developers' Time, Mentoring Time, Advocacy & Promotion Time), each of which tie to the different types of motivations. We hope our findings nudge more companies to participate in the OSS ecosystem, helping make it robust, diverse, and sustainable.

cs.CY↗