Searcharxiv⌕ Search

arXiv subjects

Nicole Novielli

Publications and source records attributed to Nicole Novielli.

At least 19 recordsLinked to original sources
arXiv2026

Using Biometrics to Understand AI-Assisted Coding Performance and its Perception

AI-based code assistants are transforming software development, yet we lack empirical evidence on how they affect developers' cognitive processes. We present a multisite study investigating the neurophysiological correlates of AI-assisted programming through a within-subjects crossover design. We recruited participants at two universities (Bari, Italy, and Copenhagen, Denmark) and collected electroencephalography, eye-tracking, electrodermal activity, and heart rate variability data alongside a rubric-based performance score and self-reported workload across six dimensions using the NASA Task Load Index (NASA-TLX). We tested four hypotheses addressing physiological differences between AI-assisted and non-assisted conditions, the moderating role of developer experience, the association between physiology and performance, and the alignment between subjective perceptions and objective measures. Under AI assistance, the EEG $θ/α$ ratio was lower during the first task and the gaze blink rate was higher during the second, both consistent with reduced cognitive engagement when developers offload generative effort to the model. This pattern did not differ between undergraduate and graduate students. Electrodermal activity correlated with performance under the non-AI condition but not under AI. Among the six NASA-TLX dimensions of self-reported workload, only Physical demand was associated with performance under the non-AI condition but not under AI. These findings suggest that AI-assisted programming is not a faster version of solo coding but a cognitively distinct activity, with implications for the design of AI assistants and for biometric monitoring in AI-augmented development.

cs.HC↗
arXiv2026

Forecasting Developer Environments with GenAI: A Research Perspective

Generative Artificial Intelligence (GenAI) models are achieving remarkable performance in various tasks, including code generation, testing, code review, and program repair. The ability to increase the level of abstraction away from writing code has the potential to change the Human-AI interaction within the integrated development environment (IDE). To explore the impact of GenAI on IDEs, 33 experts from the Software Engineering, Artificial Intelligence, and Human-Computer Interaction domains gathered to discuss challenges and opportunities at Shonan Meeting 222, a four-day intensive research meeting. Four themes emerged as areas of interest for researchers and practitioners.

cs.SE↗
arXiv2025

Exploring Engagement in Hybrid Meetings

Background. The widespread adoption of hybrid work following the COVID-19 pandemic has fundamentally transformed software development practices, introducing new challenges in communication and collaboration as organizations transition from traditional office-based structures to flexible working arrangements. This shift has established a new organizational norm where even traditionally office-first companies now embrace hybrid team structures. While remote participation in meetings has become commonplace in this new environment, it may lead to isolation, alienation, and decreased engagement among remote team members. Aims. This study aims to identify and characterize engagement patterns in hybrid meetings through objective measurements, focusing on the differences between co-located and remote participants. Method. We studied professionals from three software companies over several weeks, employing a multimodal approach to measure engagement. Data were collected through self-reported questionnaires and physiological measurements using biometric devices during hybrid meetings to understand engagement dynamics. Results. The regression analyses revealed comparable engagement levels between onsite and remote participants, though remote participants show lower engagement in long meetings regardless of participation mode. Active roles positively correlate with higher engagement, while larger meetings and afternoon sessions are associated with lower engagement. Conclusions. Our results offer insights into factors associated with engagement and disengagement in hybrid meetings, as well as potential meeting improvement recommendations. These insights are potentially relevant not only for software teams but also for knowledge-intensive organizations across various sectors facing similar hybrid collaboration challenges.

cs.SE↗
arXiv2025

Blended PC Peer Review Model: Process and Reflection

The academic peer review system is under increasing pressure due to a growing volume of submissions and a limited pool of available reviewers, resulting in delayed decisions and an uneven distribution of reviewing responsibilities. Building upon the International Conference on Mining Software Repositories (MSR) community's earlier experience with a Shadow PC (2021 and 2022) and Junior PC (2023 and 2024), MSR 2025 experimented with a Blended Program Committee (PC) peer review model for its Technical Track. This new model pairs up one Junior PC member with two regular PC members as part of the core review team of a given paper, instead of adding them as an extra reviewer. This paper presents the rationale, implementation, and reflections on the model, including empirical insights from a post-review author survey evaluating the quality and usefulness of reviews. Our findings highlight the potential of a Blended PC to alleviate reviewer shortages, foster inclusivity, and sustain a high-quality peer review process. We offer lessons learned and recommendations to guide future adoption and refinement of the model.

cs.SE↗
arXiv2025

A Multimodal Approach Combining Biometrics and Self-Report Instruments for Monitoring Stress in Programming: Methodological Insights

The study of well-being, stress and other human factors has traditionally relied on self-report instruments to assess key variables. However, concerns about potential biases in these instruments, even when thoroughly validated and standardised, have driven growing interest in alternatives in combining these measures with more objective methods, such as physiological measures. We aimed to (i) compare psychometric stress measures and biometric indicators and (ii) identify stress-related patterns in biometric data during software engineering tasks. We conducted an experiment where participants completed a pre-survey, then programmed two tasks wearing biometric sensors, answered brief post-surveys for each, and finally went through a short exit interview. Our results showed diverse outcomes; we found no stress in the psychometric instruments. Participants in the interviews reported a mix of feeling no stress and experiencing time pressure. Finally, the biometrics showed a significant difference only in EDA phasic peaks. We conclude that our chosen way of inducing stress by imposing a stricter time limit was insufficient. We offer methodological insights for future studies working with stress, biometrics, and psychometric instruments.

cs.SE↗
arXiv2025

Negativity in Self-Admitted Technical Debt: How Sentiment Influences Prioritization

Self-Admitted Technical Debt, or SATD, is a self-admission of technical debt present in a software system. To effectively manage SATD, developers need to estimate its priority and assess the effort required to fix the described technical debt. About a quarter of descriptions of SATD in software systems express some form of negativity or negative emotions when describing technical debt. In this paper, we report on an experiment conducted with 59 respondents to study whether negativity expressed in the description of SATD \textbf{actually} affects the prioritization of SATD. The respondents are a mix of professional developers and students, and in the experiment, we asked participants to prioritize four vignettes: two expressing negativity and two expressing neutral sentiment. To ensure realism, vignettes were based on existing SATD. We find that negativity causes between one-third and half of developers to prioritize SATD, in which negativity is expressed as having more priority. Developers affected by negativity when prioritizing SATD are twice as likely to increase their estimation of urgency and 1.5 times as likely to increase their estimation of importance and effort for SATD compared to the likelihood of decreasing these prioritization scores. Our findings show how developers actively use negativity in SATD to determine how urgently a particular instance of TD should be addressed. However, our study also describes a gap in the actions and belief of developers. Even if 33% to 50% use negativity to prioritize SATD, 67% of developers believe that using negativity as a proxy for priority is unacceptable. Therefore, we would not recommend using negativity as a proxy for priority. However, we also recognize that developers might unavoidably express negativity when describing technical debt.

cs.SE↗
arXiv2024

Using Voice and Biofeedback to Predict User Engagement during Product Feedback Interviews

Capturing users' engagement is crucial for gathering feedback about the features of a software product. In a market-driven context, current approaches to collect and analyze users' feedback are based on techniques leveraging information extracted from product reviews and social media. These approaches are hardly applicable in bespoke software development, or in contexts in which one needs to gather information from specific users. In such cases, companies need to resort to face-to-face interviews to get feedback on their products. In this paper, we propose to utilize biometric data, in terms of physiological and voice features, to complement interviews with information about the engagement of the user on the discussed product-relevant topics. We evaluate our approach by interviewing users while gathering their physiological data (i.e., biofeedback) using an Empatica E4 wristband, and capturing their voice through the default audio-recorder of a common laptop. Our results show that we can predict users' engagement by training supervised machine learning algorithms on biometric data (F1=0.72), and that voice features alone are sufficiently effective (F1=0.71). Our work contributes with one the first studies in requirements engineering in which biometrics are used to identify emotions. This is also the first study in software engineering that considers voice analysis. The usage of voice features could be particularly helpful for emotion-aware requirements elicitation in remote communication, either performed by human analysts or voice-based chatbots, and can also be exploited to support the analysis of meetings in software engineering research.

cs.SE↗
arXiv2024

A Disruptive Research Playbook for Studying Disruptive Innovations

As researchers, we are now witnessing a fundamental change in our technologically-enabled world due to the advent and diffusion of highly disruptive technologies such as generative AI, Augmented Reality (AR) and Virtual Reality (VR). In particular, software engineering has been profoundly affected by the transformative power of disruptive innovations for decades, with a significant impact of technical advancements on social dynamics due to its the socio-technical nature. In this paper, we reflect on the importance of formulating and addressing research in software engineering through a socio-technical lens, thus ensuring a holistic understanding of the complex phenomena in this field. We propose a research playbook with the goal of providing a guide to formulate compelling and socially relevant research questions and to identify the appropriate research strategies for empirical investigations, with an eye on the long-term implications of technologies or their use. We showcase how to apply the research playbook. Firstly, we show how it can be used retrospectively to reflect on a prior disruptive technology, Stack Overflow, and its impact on software development. Secondly, we show it can be used to question the impact of two current disruptive technologies: AI and AR/VR. Finally, we introduce a specialized GPT model to support the researcher in framing future investigations. We conclude by discussing the broader implications of adopting the playbook for both researchers and practitioners in software engineering and beyond.

cs.SE↗
arXiv2021

GitHub Discussions: An Exploratory Study of Early Adoption

Discussions is a new feature of GitHub for asking questions or discussing topics outside of specific Issues or Pull Requests. Before being available to all projects in December 2020, it had been tested on selected open source software projects. To understand how developers use this novel feature, how they perceive it, and how it impacts the development processes, we conducted a mixed-methods study based on early adopters of GitHub discussions from January until July 2020. We found that: (1) errors, unexpected behavior, and code reviews are prevalent discussion categories; (2) there is a positive relationship between project member involvement and discussion frequency; (3) developers consider GitHub Discussions useful but face the problem of topic duplication between Discussions and Issues; (4) Discussions play a crucial role in advancing the development of projects; and (5) positive sentiment in Discussions is more frequent than in Stack Overflow posts. Our findings are a first step towards data-informed guidance for using GitHub Discussions, opening up avenues for future work on this novel communication channel.

cs.SE↗
arXiv2021

Recognizing Developers' Emotions while Programming

Developers experience a wide range of emotions during programming tasks, which may have an impact on job performance. In this paper, we present an empirical study aimed at (i) investigating the link between emotion and progress, (ii) understanding the triggers for developers' emotions and the strategies to deal with negative ones, (iii) identifying the minimal set of non-invasive biometric sensors for emotion recognition during programming task. Results confirm previous findings about the relation between emotions and perceived productivity. Furthermore, we show that developers' emotions can be reliably recognized using only a wristband capturing the electrodermal activity and heart-related metrics.

cs.SE↗
arXiv2021

EMTk -- The Emotion Mining Toolkit

The Emotion Mining Toolkit (EMTk) is a suite of modules and datasets offering a comprehensive solution for mining sentiment and emotions from technical text contributed by developers on communication channels. The toolkit is written in Java, Python, and R, and is released under the MIT open source license. In this paper, we describe its architecture and the benchmark against the previous, standalone versions of our sentiment analysis tools. Results show large improvements in terms of speed.

cs.SE↗
arXiv2021

Assessment of Off-the-Shelf SE-specific Sentiment Analysis Tools: An Extended Replication Study

Sentiment analysis methods have become popular for investigating human communication, including discussions related to software projects. Since general-purpose sentiment analysis tools do not fit well with the information exchanged by software developers, new tools, specific for software engineering (SE), have been developed. We investigate to what extent SE-specific tools for sentiment analysis mitigate the threats to conclusion validity of empirical studies in software engineering, highlighted by previous research. First, we replicate two studies addressing the role of sentiment in security discussions on GitHub and in question-writing on Stack Overflow. Then, we extend the previous studies by assessing to what extent the tools agree with each other and with the manual annotation on a gold standard of 600 documents. We find that different SE-specific sentiment analysis tools might lead to contradictory results at a fine-grain level, when used 'off-the-shelf'. Conversely, platform-specific tuning or retraining might be needed to take into account differences in platform conventions, jargon, or document lengths.

cs.SE↗
arXiv2020

Pandemic Programming: How COVID-19 affects software developers and how their organizations can help

Context. As a novel coronavirus swept the world in early 2020, thousands of software developers began working from home. Many did so on short notice, under difficult and stressful conditions. Objective. This study investigates the effects of the pandemic on developers' wellbeing and productivity. Method. A questionnaire survey was created mainly from existing, validated scales and translated into 12 languages. The data was analyzed using non-parametric inferential statistics and structural equation modeling. Results. The questionnaire received 2225 usable responses from 53 countries. Factor analysis supported the validity of the scales and the structural model achieved a good fit (CFI = 0.961, RMSEA = 0.051, SRMR = 0.067). Confirmatory results include: (1) the pandemic has had a negative effect on developers' wellbeing and productivity; (2) productivity and wellbeing are closely related; (3) disaster preparedness, fear related to the pandemic and home office ergonomics all affect wellbeing or productivity. Exploratory analysis suggests that: (1) women, parents and people with disabilities may be disproportionately affected; (2) different people need different kinds of support. Conclusions. To improve employee productivity, software companies should focus on maximizing employee wellbeing and improving the ergonomics of employees' home offices. Women, parents and disabled persons may require extra support.

cs.SE↗
arXiv2020

Can We Use SE-specific Sentiment Analysis Tools in a Cross-Platform Setting?

In this paper, we address the problem of using sentiment analysis tools 'off-the-shelf,' that is when a gold standard is not available for retraining. We evaluate the performance of four SE-specific tools in a cross-platform setting, i.e., on a test set collected from data sources different from the one used for training. We find that (i) the lexicon-based tools outperform the supervised approaches retrained in a cross-platform setting and (ii) retraining can be beneficial in within-platform settings in the presence of robust gold standard datasets, even using a minimal training set. Based on our empirical findings, we derive guidelines for reliable use of sentiment analysis tools in software engineering.

cs.SE↗
arXiv2020

ArguLens: Anatomy of Community Opinions On Usability Issues Using Argumentation Models

In open-source software (OSS), the design of usability is often influenced by the discussions among community members on platforms such as issue tracking systems (ITSs). However, digesting the rich information embedded in issue discussions can be a major challenge due to the vast number and diversity of the comments. We propose and evaluate ArguLens, a conceptual framework and automated technique leveraging an argumentation model to support effective understanding and consolidation of community opinions in ITSs. Through content analysis, we anatomized highly discussed usability issues from a large, active OSS project, into their argumentation components and standpoints. We then experimented with supervised machine learning techniques for automated argument extraction. Finally, through a study with experienced ITS users, we show that the information provided by ArguLens supported the digestion of usability-related opinions and facilitated the review of lengthy issues. ArguLens provides the direction of designing valuable tools for high-level reasoning and effective discussion about usability.

cs.HC↗
arXiv2019

A Replication Study on Code Comprehension and Expertise using Lightweight Biometric Sensors

Code comprehension has been recently investigated from physiological and cognitive perspectives through the use of medical imaging. Floyd et al (i.e., the original study) used fMRI to classify the type of comprehension tasks performed by developers and relate such results to their expertise. We replicate the original study using lightweight biometrics sensors which participants (28 undergrads in computer science) wore when performing comprehension tasks on source code and natural language prose. We developed machine learning models to automatically identify what kind of tasks developers are working on leveraging their brain-, heart-, and skin-related signals. The best improvement over the original study performance is achieved using solely the heart signal obtained through a single device (BAC 87% vs. 79.1%). Differently from the original study, we were not able to observe a correlation between the participants' expertise and the classifier performance (tau = 0.16, p = 0.31). Our findings show that lightweight biometric sensors can be used to accurately recognize comprehension tasks opening interesting scenarios for research and practice.

cs.SE↗
arXiv2019

An empirical assessment of best-answer prediction models in technical Q&A sites

Technical Q&A sites have become essential for software engineers as they constantly seek help from other experts to solve their work problems. Despite their success, many questions remain unresolved, sometimes because the asker does not acknowledge any helpful answer. In these cases, an information seeker can only browse all the answers within a question thread to assess their quality as potential solutions. We approach this time-consuming problem as a binary-classification task where a best-answer prediction model is built to identify the accepted answer among those within a resolved question thread, and the candidate solutions to those questions that have received answers but are still unresolved. In this paper, we report on a study aimed at assessing 26 best-answer prediction models in two steps. First, we study how models perform when predicting best answers in Stack Overflow, the most popular Q&A site for software engineers. Then, we assess performance in a cross-platform setting where the prediction models are trained on Stack Overflow and tested on other technical Q&A sites. Our findings show that the choice of the classifier and automated parameter tuning have a large impact on the prediction of the best answer. We also demonstrate that our approach to the best-answer prediction problem is generalizable across technical Q&A sites. Finally, we provide practical recommendations to Q&A platform designers to curate and preserve the crowdsourced knowledge shared through these sites.

cs.SE↗