arXiv subjects

Margaret-Anne Storey

Publications and source records attributed to Margaret-Anne Storey.

At least 19 recordsLinked to original sources

arXiv2026

ChatGPT: Friend or Foe When Comprehending and Changing Unfamiliar Code

A rapidly growing body of research is examining how LLMs influence developers when they code. To date, this research has tended to focus on productivity and code quality outcomes, rather than the underlying cognitive processes involved in programming. To address this gap, we report on the results of an exploratory laboratory study of ten advanced student developers (five with support from AI and five without) who had to make a non-trivial extension to a sizable software system. Leveraging Polya's four problem-solving phases and 25 inductively-generated codes detailing distinct problem-solving behaviors as the primary lenses, we examined: (1) how AI impacted the problem-solving approach the developers used to solve the programming task, and (2) how AI impacted their progress when they became stuck. For the analysis, we triangulated data across multiple sources (e.g., think-aloud, code changes, web searches, and LLM prompts). Unexpectedly, while developers in the AI group repeatedly turned to the AI tool to offload certain aspects of the process, all detailed problem-solving behaviors appeared in both groups. We also found that nine out of ten participants found themselves stuck in their work, but with key differences in how they became stuck and unstuck. We highlight seven distinct causes for being stuck and highlight how AI in some cases helped and in other cases hindered becoming unstuck.

cs.SE↗

arXiv2026

Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscape

Context: Software engineering (SE) researchers increasingly study Generative AI (GenAI) while also incorporating it into their own research practices. Despite rapid adoption, there is limited empirical evidence on how GenAI is used in SE research and its implications for research practices and governance. Aims: We conduct a large-scale survey of 457 SE researchers publishing in top venues between 2023 and 2025. Method: Using quantitative and qualitative analyses, we examine who uses GenAI and why, where it is used across research activities, and how researchers perceive its benefits, opportunities, challenges, risks, and governance. Results: GenAI use is widespread, with many researchers reporting pressure to adopt and align their work with it. Usage is concentrated in writing and early-stage activities, while methodological and analytical tasks remain largely human-driven. Although productivity gains are widely perceived, concerns about trust, correctness, and regulatory uncertainty persist. Researchers highlight risks such as inaccuracies and bias, emphasize mitigation through human oversight and verification, and call for clearer governance, including guidance on responsible use and peer review. Conclusion: We provide a fine-grained, SE-specific characterization of GenAI use across research activities, along with taxonomies of GenAI use cases for research and peer review, opportunities, risks, mitigation strategies, and governance needs. These findings establish an empirical baseline for the responsible integration of GenAI into academic practice.

cs.SE↗

arXiv2026

From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI

Generative AI is accelerating software development, but may quietly shift where the most significant risks lie. As AI generates code faster than teams can understand it, two under appreciated forms of debt accumulate: cognitive debt, the erosion of shared understanding across a team, and intent debt, the absence of externalized rationale that developers and AI agents need to work safely with code. This article proposes a Triple Debt Model for reasoning about software health, built around three interacting debt types: technical debt in code, cognitive debt in people, and intent debt in externalized knowledge. Cognitive debt is a team-level, project-level property reflecting the erosion of shared understanding across a software system over time, leading to increasingly inadequate shared mental models for reasoning about and safely changing the system. Intent debt refers to the absence or erosion of explicit rationale, goals, and constraints that guide how humans and agents evolve the system. We discuss how generative AI changes the relative importance of these debt types, how each can be diagnosed and mitigated, and surface points of debate for practitioners.

cs.SE↗

arXiv2026

Theory of Troubleshooting: The Developer's Cognitive Experience of Overcoming Confusion

This paper introduces a Theory of Troubleshooting that is rooted in cognitive science. This theory helps software developers explain the challenges they face and the project risks that emerge as troubleshooting becomes difficult. We define troubleshooting as the cognitive problem-solving process of identifying, understanding, and constructing a mental model of the cause of an unexpected system behavior, and consider the cognitive process of troubleshooting to be an integral part of the activity of debugging. Troubleshooting is a particularly intense and draining aspect of software work, placing sustained demands on attention, working memory, and mental modeling. By surfacing and naming the confusion experience inherent in troubleshooting in terms of neurological and attentional dynamics, our theory explains how prolonged troubleshooting can deplete cognitive resources and lead to cognitive fatigue. In the study presented in this paper, we interview 27 professional developers about their troubleshooting experiences, and follow a Constructivist Grounded Theory approach to construct a theory grounded in empirical data. Our theory contributes to research on Developer Experience by providing a cognitive foundation for understanding troubleshooting difficulty, fatigue, and sustainability risk--and offers practical implications for both research and industry.

cs.SE↗

arXiv2026

Towards a Cognitive-Support Tool for Threat Hunters

Cybersecurity increasingly relies on threat hunters to proactively identify adversarial activity, yet the cognitive work underlying threat hunting remains underexplored or insufficiently supported by existing tools. Building on prior studies that examined how threat hunters construct and share mental models during investigations, we derived a set of design propositions to support their cognitive and collaborative work. In this paper, we present the Threat Hunter Board, a prototype tool that operationalizes these design propositions by enabling threat hunters to externalize reasoning, organize investigative leads, and maintain continuity across sessions. Using a design science paradigm, we describe the solution design rationale and artifact development. In addition, we propose six design heuristics that form a solution-evaluation framework for assessing cognitive support in threat hunting tools. An initial evaluation using a cognitive walkthrough provides early evidence of feasibility, while future work will focus on user-based validation with professional threat hunters.

cs.CR↗

arXiv2025

Generative AI and Empirical Software Engineering: A Paradigm Shift

The adoption of large language models (LLMs) and autonomous agents in software engineering marks an enduring paradigm shift. These systems create new opportunities for tool design, workflow orchestration, and empirical observation, while fundamentally reshaping the roles of developers and the artifacts they produce. Although traditional empirical methods remain central to software engineering research, the rapid evolution of AI introduces new data modalities, alters causal assumptions, and challenges foundational constructs such as "developer", "artifact", and "interaction". As humans and AI agents increasingly co-create, the boundaries between social and technical actors blur, and the reproducibility of findings becomes contingent on model updates and prompt contexts. This vision paper examines how the integration of LLMs into software engineering disrupts established research paradigms. We discuss how it transforms the phenomena we study, the methods and theories we rely on, the data we analyze, and the threats to validity that arise in dynamic AI-mediated environments. Our aim is to help the empirical software engineering community adapt its questions, instruments, and validation standards to a future in which AI systems are not merely tools, but active collaborators shaping software engineering and its study.

cs.SE↗

arXiv2025

Product Manager Practices for Delegating Work to Generative AI: "Accountability must not be delegated to non-human actors"

Generative AI (GenAI) is changing the nature of knowledge work, particularly for Product Managers (PMs) in software development teams. While much software engineering research has focused on developers' interactions with GenAI, there is less understanding of how the work of PMs is evolving due to GenAI. To address this gap, we conducted a mixed-methods study at Microsoft, a large, multinational software company: surveying 885 PMs, analyzing telemetry data for a subset of PMs (N=731), and interviewing a subset of 15 PMs. We contribute: (1) PMs' current GenAI adoption rates, uses cases, and perceived benefits and barriers and; (2) a framework capturing how PMs assess which tasks to delegate to GenAI; (3) PMs adaptation practices for integrating GenAI into their roles and perceptions of how their role is evolving. We end by discussing implications on the broader GenAI workflow adoption process and software development roles.

cs.SE↗

arXiv2025

Good Vibrations? A Qualitative Study of Co-Creation, Communication, Flow, and Trust in Vibe Coding

Vibe coding, a term coined by Andrej Karpathy in February 2025, has quickly become a compelling and controversial natural language programming paradigm in AI-assisted software development. Centered on iterative co-design with an AI assistant, vibe coding emphasizes flow and experimentation over strict upfront specification. While initial studies have begun to explore this paradigm, most focus on analyzing code artifacts or proposing theories with limited empirical backing. There remains a need for a grounded understanding of vibe coding as it is perceived and experienced by developers. We present the first systematic qualitative investigation of vibe coding perceptions and practice. Drawing on over 190,000 words from semi-structured interviews, Reddit threads, and LinkedIn posts, we characterize what vibe coding is, why and how developers use it, where it breaks down, and which emerging practices aim to support it. We propose a qualitatively grounded theory of vibe coding centered on conversational interaction with AI, co-creation, and developer flow and joy. We find that AI trust regulates movement along a continuum from delegation to co-creation and supports the developer experience by sustaining flow. We surface recurring pain points and risks in areas including specification, reliability, debugging, latency, code review burden, and collaboration. We also present best practices that have been discovered and shared to mitigate these challenges. We conclude with implications for the future of AI dev tools and directions for researchers investigating vibe coding.

cs.SE↗

arXiv2025

From Passive to Participatory: How Liberating Structures Can Revolutionize Our Conferences

Our conferences face a growing crisis: an overwhelming flood of submissions, increased reviewing burdens, and diminished opportunities for meaningful engagement. With AI making paper generation easier than ever, we must ask whether the current model fosters real innovation or simply incentivizes more publications. This article advocates for a shift from passive paper presentations to interactive, participatory formats. We propose Liberating Structures, facilitation techniques that promote collaboration and deeper intellectual exchange. By restructuring conferences into two tracks, one for generating new ideas and another for discussing established work, we can prioritize quality over quantity and reinvigorate academic gatherings. Embracing this change will ensure conferences remain spaces for real insight, creativity, and impactful collaboration in the AI era.

cs.CY↗

arXiv2025

Fuzzy to Clear: Elucidating the Threat Hunter Cognitive Process and Cognitive Support Needs

With security threats increasing in frequency and severity, it is critical that we consider the important role of threat hunters. These highly-trained security professionals learn to see, identify, and intercept security threats. Many recent works and existing tools in cybersecurity are focused on automating the threat hunting process, often overlooking the critical human element. Our study shifts this paradigm by emphasizing a human-centered approach to understanding the lived experiences of threat hunters. By observing threat hunters during hunting sessions and analyzing the rich insights they provide, we seek to advance the understanding of their cognitive processes and the tool support they need. Through an in-depth observational study of threat hunters, we introduce a model of how they build and refine their mental models during threat hunting sessions. We also present 23 themes that provide a foundation to better understand threat hunter needs and suggest five actionable design propositions to enhance the tools that support them. Through these contributions, our work enriches the theoretical understanding of threat hunting and provides practical insights for designing more effective, human-centered cybersecurity tools.

cs.CR↗

arXiv2025

"Maybe We Need Some More Examples:" Individual and Team Drivers of Developer GenAI Tool Use

Despite the widespread availability of generative AI tools in software engineering, developer adoption remains uneven. This unevenness is problematic because it hampers productivity efforts, frustrates management's expectations, and creates uncertainty around the future roles of developers. Through paired interviews with 54 developers across 27 teams -- one frequent and one infrequent user per team -- we demonstrate that differences in usage result primarily from how developers perceive the tool (as a collaborator vs. feature), their engagement approach (experimental vs. conservative), and how they respond when encountering challenges (with adaptive persistence vs. quick abandonment). Our findings imply that widespread organizational expectations for rapid productivity gains without sufficient investment in learning support creates a "Productivity Pressure Paradox," undermining the very productivity benefits that motivate adoption.

cs.SE↗

arXiv2025

Get on the Train or be Left on the Station: Using LLMs for Software Engineering Research

The adoption of Large Language Models (LLMs) is not only transforming software engineering (SE) practice but is also poised to fundamentally disrupt how research is conducted in the field. While perspectives on this transformation range from viewing LLMs as mere productivity tools to considering them revolutionary forces, we argue that the SE research community must proactively engage with and shape the integration of LLMs into research practices, emphasizing human agency in this transformation. As LLMs rapidly become integral to SE research - both as tools that support investigations and as subjects of study - a human-centric perspective is essential. Ensuring human oversight and interpretability is necessary for upholding scientific rigor, fostering ethical responsibility, and driving advancements in the field. Drawing from discussions at the 2nd Copenhagen Symposium on Human-Centered AI in SE, this position paper employs McLuhan's Tetrad of Media Laws to analyze the impact of LLMs on SE research. Through this theoretical lens, we examine how LLMs enhance research capabilities through accelerated ideation and automated processes, make some traditional research practices obsolete, retrieve valuable aspects of historical research approaches, and risk reversal effects when taken to extremes. Our analysis reveals opportunities for innovation and potential pitfalls that require careful consideration. We conclude with a call to action for the SE research community to proactively harness the benefits of LLMs while developing frameworks and guidelines to mitigate their risks, to ensure continued rigor and impact of research in an AI-augmented future.

cs.SE↗

arXiv2025

Who "Controls" Where Work Shall be Done? State-of-Practice in Post-Pandemic Remote Work Regulation

The COVID-19 pandemic has permanently altered workplace structures, making remote work a widespread practice. While many employees advocate for flexibility, many employers reconsider their attitude toward remote work and opt for structured return-to-office mandates. Media headlines repeatedly emphasize that the corporate world is returning to full-time office work. This study examines how companies employing software engineers and supporting roles regulate work location, whether corporate policies have evolved in the last five years, and, if so, how, and why. We collected data on remote work regulation from corporate HR and/or management representatives from 68 corporate entities that vary in size, location, and orientation towards remote or office work. Our findings reveal that although many companies prioritize office-centred working (50%), most companies in our sample permit hybrid working to varying degrees (85%). Remote work regulation does not reveal any particular new "best practice" as policies differ greatly, but the single most popular arrangement was the three in-office days per week. More than half of the companies (51%) encourage or mandate office days, and more than quarter (28%) have changed regulations, gradually increasing the mandatory office presence or implementing differentiated conditions. Although no companies have increased flexibility, only four companies are returning to full-time office work. Our key recommendation for office-oriented companies is to consider a trust-based alternative to strict office presence mandates, while for companies oriented toward remote working, we warn about the points of no (or hard) return. Finally, the current state of policies is clearly not final, as companies continue to experiment and adjust their work regulation.

cs.SE↗

arXiv2025

Code Review Comprehension: Reviewing Strategies Seen Through Code Comprehension Theories

Despite the popularity and importance of modern code review, the understanding of the cognitive processes that enable reviewers to analyze code and provide meaningful feedback is lacking. To address this gap, we observed and interviewed ten experienced reviewers while they performed 25 code reviews from their review queue. Since comprehending code changes is essential to perform code review and the primary challenge for reviewers, we focused our analysis on this cognitive process. Using Letovsky's model of code comprehension, we performed a theory-driven thematic analysis to investigate how reviewers apply code comprehension to navigate changes and provide feedback. Our findings confirm that code comprehension is fundamental to code review. We extend Letovsky's model to propose the Code Review Comprehension Model and demonstrate that code review, like code comprehension, relies on opportunistic strategies. These strategies typically begin with a context-building phase, followed by code inspection involving code reading, testing, and discussion management. To interpret and evaluate the proposed change, reviewers construct a mental model of the change as an extension of their understanding of the overall software system and contrast mental representations of expected and ideal solutions against the actual implementation. Based on our findings, we discuss how review tools and practices can better support reviewers in employing their strategies and in forming understanding. Data and material: https://doi.org/10.5281/zenodo.14748996

cs.SE↗

arXiv2025

Guiding Principles for Using Mixed Methods Research in Software Engineering

Mixed methods research is often used in software engineering, but researchers outside of the social or human sciences often lack experience when using these designs. This paper provides guiding principles and advice on how to design mixed method research, and to encourage the intentional, rigorous, and innovative application of mixed methods in software engineering. It also presents key properties of core mixed method research designs. Through a number of fictitious but recognizable software engineering research scenarios, we showcase how to choose suitable designs and consider the inevitable trade-offs any design choice leads to. We describe several antipatterns that illustrate what to avoid in mixed method research, and when mixed method research should be considered over other approaches.

cs.SE↗

arXiv2025

Exploring Retrospective Meeting Practices and the Use of Data in Agile Teams

Retrospectives are vital for software development teams to continuously enhance their processes and teamwork. Despite the increasing availability of objective data generated throughout the project and software development processes, many teams do not fully utilize this information in retrospective meetings. Instead, they often rely on subjective data, anecdotal insights and their memory. While some literature underscores the value of data-driven retrospectives, little attention has been given to the role data can play and the challenges of effectively incorporating objective project data into these meetings. To address this gap, we conducted a survey with 19 practitioners on retrospective meeting practices and how their teams gather and use subjective and objective data in their retrospectives. Our findings confirm that although teams routinely collect project data, they seldom employ it systematically during retrospectives. Furthermore, this study provides insights into retrospective practices by exploring barriers to project data utilization, including psychological safety concerns and the disconnect between data collection and meaningful integration of data into retrospective meetings. We close by considering preliminary insights that may help to mitigate these concerns and how future research might build on our paper findings to support the integration of project data into retrospective meetings, fostering a balance between human-centric reflections and data-driven insights.

cs.SE↗

arXiv2024

Identifying Factors Contributing to Bad Days for Software Developers: A Mixed Methods Study

Software development is a dynamic activity that requires engineers to work effectively with tools, processes, and collaborative teams. As a result, the presence of friction can significantly hinder productivity, increase frustration, and contribute to low morale among developers. By contrast, higher satisfaction levels are positively correlated with higher levels of perceived productivity. Hence, understanding the factors that cause bad experiences for developers is critical for fostering a positive and productive engineering environment. In this research, we employed a mixed-method approach, including interviews, surveys, diary studies, and analysis of developer telemetry data to uncover and triangulate common factors that cause "bad days" for developers. The interviews involved 22 developers across different levels and roles. The survey captured the perception of 214 developers about factors that cause them to have "bad days," their frequency, and their impact on job satisfaction. The daily diary study engaged 79 developers for 30 days to document factors that caused "bad days" in the moment. We examined the telemetry signals of 131 consenting participants to validate the impact of bad developer experience using system data. Findings from our research revealed factors that cause "bad days" for developers and significantly impact their work and well-being. We discuss the implications of these findings and suggest future work.

cs.SE↗

arXiv2024

A Disruptive Research Playbook for Studying Disruptive Innovations

As researchers, we are now witnessing a fundamental change in our technologically-enabled world due to the advent and diffusion of highly disruptive technologies such as generative AI, Augmented Reality (AR) and Virtual Reality (VR). In particular, software engineering has been profoundly affected by the transformative power of disruptive innovations for decades, with a significant impact of technical advancements on social dynamics due to its the socio-technical nature. In this paper, we reflect on the importance of formulating and addressing research in software engineering through a socio-technical lens, thus ensuring a holistic understanding of the complex phenomena in this field. We propose a research playbook with the goal of providing a guide to formulate compelling and socially relevant research questions and to identify the appropriate research strategies for empirical investigations, with an eye on the long-term implications of technologies or their use. We showcase how to apply the research playbook. Firstly, we show how it can be used retrospectively to reflect on a prior disruptive technology, Stack Overflow, and its impact on software development. Secondly, we show it can be used to question the impact of two current disruptive technologies: AI and AR/VR. Finally, we introduce a specialized GPT model to support the researcher in framing future investigations. We conclude by discussing the broader implications of adopting the playbook for both researchers and practitioners in software engineering and beyond.

cs.SE↗