arXiv subjects

Stefano Lambiase

Publications and source records attributed to Stefano Lambiase.

13 recordsLinked to original sources

arXiv2026

From Challenge to Change: Design Principles for AI Transformations

The rapid rise of Artificial Intelligence (AI) is reshaping Software Engineering (SE), creating new opportunities while introducing human-centered challenges. Although prior work notes behavioral and other non-technical factors in AI integration, most studies still emphasize technical concerns and offer limited insight into how teams adapt to and trust AI. This paper proposes a Behavioral Software Engineering (BSE)-informed, human-centric framework to support SE organizations during early AI adoption. Using a mixed-methods approach, we built and refined the framework through a literature review of organizational change models and thematic analysis of interview data, producing concrete, actionable steps. The framework comprises nine dimensions: AI Strategy Design, AI Strategy Evaluation, Collaboration, Communication, Governance and Ethics, Leadership, Organizational Culture, Organizational Dynamics, and Up-skilling, each supported by design principles and actions. To gather preliminary practitioner input, we conducted a survey (N=105) and two expert workshops (N=4). Survey results show that Up-skilling (15.2%) and AI Strategy Design (15.1%) received the highest $100-method allocations, underscoring their perceived importance in early AI initiatives. Findings indicate that organizations currently prioritize procedural elements such as strategy design, while human-centered guardrails remain less developed. Workshop feedback reinforced these patterns and emphasized the need to ground the framework in real-world practice. By identifying key behavioral dimensions and offering actionable guidance, this work provides a pragmatic roadmap for navigating the socio-technical complexity of early AI adoption and highlights future research directions for human-centric AI in SE.

cs.SE↗

arXiv2026

The Language of Security: How Prompt Syntax Shapes Secure Code Generation in Open LLMs

Large Language Models (LLMs) are increasingly used for source code generation despite their outputs often exhibiting security vulnerabilities. Prior work shows that prompt engineering can mitigate such risks, yet (1) they focused on high-level prompting strategies, neglecting recent evidence that fine-grained syntactic variations can substantially alter model behavior; and (2) predominantly evaluate proprietary LLMs, limiting the applicability of their findings in industrial settings where self-hosted, open models are preferred for privacy, compliance, and deployment control. In this paper, we study how fine-grained syntactic constituents of prompts influence the security of open LLM-generated code. Using a parser-driven approach, we systematically generate syntactic variants of security-relevant code generation prompts and evaluate their impact on code security across multiple open LLMs and programming languages. Our results show that specific syntactic elements, such as constraints, guards, conditions, and concept bindings, and their position within the prompt consistently affect the likelihood of generating insecure code. These findings identify prompt syntax as a concrete security control surface and provide actionable guidance for reducing vulnerability risk in LLM-assisted development.

cs.CR↗

arXiv2026

TARS: A Theory-of-Mind Agent for Personalized In-IDE Code Comprehension

Code comprehension is one of the most time-consuming tasks in software engineering, yet most LLM-based assistants produce explanations that ignore who is asking and force developers into a disruptive copy-paste workflow. We present TARS, an LLM-powered agent integrated into Visual Studio Code that supports program comprehension through autonomous explanations anchored directly to the code under analysis. Built around a lightweight Theory of Mind paradigm, TARS profiles a developer's expertise, role, and stylistic preferences, then adapts the depth and tone of its explanations accordingly, grounding them in project documentation via Retrieval-Augmented Generation. To evaluate TARS, we conducted a controlled experiment with 18 participants on non-trivial Java snippets. Participants using TARS completed tasks 26\% faster, reported lower cognitive load, and found the explanations meaningfully adapted to their profiles.

cs.SE↗

arXiv2026

Guidelines for Empirical Studies in Software Engineering involving Large Language Models

Large Language Models (LLMs) are widely used in software engineering (SE) research and practice, yet their non-determinism, opaque training data, and rapidly evolving models threaten the reproducibility and replicability of empirical studies. We address this challenge through a collaborative effort of 22 researchers, presenting a taxonomy of seven study types that organizes how LLMs are used in SE research, together with eight guidelines for designing and reporting such studies. Each guideline distinguishes requirements (must) from recommendations (should) and is contextualized by the study types it applies to. Our guidelines recommend that researchers: (1) declare LLM usage and role; (2) report model versions, configurations, and customizations; (3) document the system and prompt design beyond the model; (4) report session traces, i.e., interaction logs and runtime traces; (5) use suitable baselines, benchmarks, and metrics; (6) include an open LLM as a baseline; (7) validate LLM outputs against human judgment; and (8) articulate limitations and mitigations. We complement the guidelines with an applicability matrix mapping guidelines to study types and a reporting checklist for authors and reviewers. We maintain the study types and guidelines online as a living resource for the community to use and shape (llm-guidelines$.$org).

cs.SE↗

arXiv2026

Exploring Individual Factors in the Adoption of LLMs for Specific Software Engineering Purposes

Context: The advent of Large Language Models (LLMs) is transforming software development, significantly enhancing software engineering (SE) processes. Research has explored their role within development teams, focusing on the specific purposes for which LLMs are used within SE tasks, such as artifact generation, decision-making support, and information retrieval. Despite the growing body of work on LLMs in SE, most studies have centered on broad adoption trends, neglecting the nuanced relationship between individual cognitive and behavioral factors and their impact on purpose-specific adoption. While factors such as perceived effort and performance expectancy have been explored at a general level, their influence on distinct SE purposes remains underexamined. This gap hinders the development of tailored LLM-based systems (e.g., Generative AI Agents) that align with engineers' specific needs and limits the ability of team leaders to devise effective strategies for fostering LLM adoption in targeted workflows. Objectives: For the reasons mentioned above, this study aims to study the individual factors that drive the choice to use LLMs for distinct SE purposes. Methods: To achieve the above-mentioned objective, we surveyed 188 software engineers to test the relationship between individual attributes related to technology adoption and LLM adoption across five key purposes, using structural equation modeling (SEM). The Unified Theory of Acceptance and Use of Technology (UTAUT2) was applied to characterize individual adoption behaviors. Results: The findings reveal that purpose-specific adoption is influenced by distinct factors, some of which negatively impact adoption when considered in isolation, underscoring the complexity of LLM integration in SE.

cs.SE↗

arXiv2026

Socio-Technical Well-Being of Quantum Software Communities: An Overview on Community Smells

Quantum computing has gained significant attention due to its potential to solve computational problems beyond the capabilities of classical computers. With major corporations and academic institutions investing in quantum hardware and software, there has been a rise in the development of quantum-enabled systems, particularly within open-source communities. However, despite the promising nature of quantum technologies, these communities face critical socio-technical challenges, including the emergence of socio-technical anti-patterns known as community smells. These anti-patterns, prevalent in open-source environments, have the potential to negatively impact both product quality and community health by introducing technical debt and amplifying architectural and code smells. Despite the importance of these socio-technical factors, there remains a scarcity of research investigating their influence within quantum open-source communities. This work aims to address this gap by providing a first step in analyzing the socio-technical well-being of quantum communities through a cross-sectional study. By understanding the socio-technical dynamics at play, it is expected that foundational knowledge can be established to mitigate the risks associated with community smells and ensure the long-term sustainability of open-source quantum initiatives.

cs.SE↗

arXiv2025

Sustainability of Machine Learning-Enabled Systems: The Machine Learning Practitioner's Perspective

Software sustainability is a key multifaceted non-functional requirement that encompasses environmental, social, and economic concerns, yet its integration into the development of Machine Learning (ML)-enabled systems remains an open challenge. While previous research has explored high-level sustainability principles and policy recommendations, limited empirical evidence exists on how sustainability is practically managed in ML workflows. Existing studies predominantly focus on environmental sustainability, e.g., carbon footprint reduction, while missing the broader spectrum of sustainability dimensions and the challenges practitioners face in real-world settings. To address this gap, we conduct an empirical study to characterize sustainability in ML-enabled systems from a practitioner's perspective. We investigate (1) how ML engineers perceive and describe sustainability, (2) the software engineering practices they adopt to support it, and (3) the key challenges hindering its adoption. We first perform a qualitative analysis based on interviews with eight experienced ML engineers, followed by a large-scale quantitative survey with 203 ML practitioners. Our key findings reveal a significant disconnection between sustainability awareness and its systematic implementation, highlighting the need for more structured guidelines, measurement frameworks, and regulatory support.

cs.SE↗

arXiv2025

How Do Communities of ML-Enabled Systems Smell? A Cross-Sectional Study on the Prevalence of Community Smells

Effective software development relies on managing both collaboration and technology, but sociotechnical challenges can harm team dynamics and increase technical debt. Although teams working on ML enabled systems are interdisciplinary, research has largely focused on technical issues, leaving their socio-technical dynamics underexplored. This study aims to address this gap by examining the prevalence, evolution, and interrelations of community smells, in open-source ML projects. We conducted an empirical study on 188 repositories from the NICHE dataset using the CADOCS tool to identify and analyze community smells. Our analysis focused on their prevalence, interrelations, and temporal variations. We found that certain smells, such as Prima Donna Effects and Sharing Villainy, are more prevalent and fluctuate over time compared to others like Radio Silence or Organizational Skirmish. These insights might provide valuable support for ML project managers in addressing socio-technical issues and improving team coordination.

cs.SE↗

arXiv2025

Do Prompt Patterns Affect Code Quality? A First Empirical Assessment of ChatGPT-Generated Code

Large Language Models (LLMs) have rapidly transformed software development, especially in code generation. However, their inconsistent performance, prone to hallucinations and quality issues, complicates program comprehension and hinders maintainability. Research indicates that prompt engineering-the practice of designing inputs to direct LLMs toward generating relevant outputs-may help address these challenges. In this regard, researchers have introduced prompt patterns, structured templates intended to guide users in formulating their requests. However, the influence of prompt patterns on code quality has yet to be thoroughly investigated. An improved understanding of this relationship would be essential to advancing our collective knowledge on how to effectively use LLMs for code generation, thereby enhancing their understandability in contemporary software development. This paper empirically investigates the impact of prompt patterns on code quality, specifically maintainability, security, and reliability, using the Dev-GPT dataset. Results show that Zero-Shot prompting is most common, followed by Zero-Shot with Chain-of-Thought and Few-Shot. Analysis of 7583 code files across quality metrics revealed minimal issues, with Kruskal-Wallis tests indicating no significant differences among patterns, suggesting that prompt structure may not substantially impact these quality metrics in ChatGPT-assisted code generation.

cs.SE↗

arXiv2025

Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review

Bots are software systems designed to support users by automating a specific process, task, or activity. When such systems implement a conversational component to interact with the users, they are also known as conversational agents. Bots, particularly in their conversation-oriented version and AI-powered, have seen their adoption increase over time for software development and engineering purposes. Despite their exciting potential, ulteriorly enhanced by the advent of Generative AI and Large Language Models, bots still need to be improved to develop and integrate into the development cycle since practitioners report that bots add additional challenges that may worsen rather than improve. In this work, we aim to provide a taxonomy for characterizing bots, as well as a series of challenges for their adoption for Software Engineering associated with potential mitigation strategies. To reach our objectives, we conducted a multivocal literature review, reviewing both research and practitioner's literature. Through such an approach, we hope to contribute to both researchers and practitioners by providing first, a series of future research routes to follow, second, a list of strategies to adopt for improving the use of bots for software engineering purposes, and third, enforce a technology and knowledge transfer from the research field to the practitioners one, that is one of the primary goal of multivocal literature reviews.

cs.SE↗

arXiv2024

On the Impact of 3D Visualization of Repository Metrics in Software Engineering Education

Context: Software development is a complex socio-technical process requiring a deep understanding of various aspects. In order to support practitioners in understanding such a complex activity, repository process metrics, like number of pull requests and issues, emerged as crucial for evaluating CI/CD workflows and guiding informed decision-making. The research community proposed different ways to visualize these metrics to increase their impact on developers' process comprehension: VR is a promising one. Nevertheless, despite such promising results, the role of VR, especially in educational settings, has received limited research attention. Objective: This study aims to address this gap by exploring how VR-based repository metrics visualization can support the teaching of process comprehension. Method: The registered report proposes the execution of a controlled experiment where VR and non-VR approaches will be compared, with the final aim to assess whether repository metrics in VR's impact on learning experience and software process comprehension. By immersing students in an intuitive environment, this research hypothesizes that VR can foster essential analytical skills, thus preparing software engineering students more effectively for industry requirements and equipping them to navigate complex software development tasks with enhanced comprehension and critical thinking abilities.

cs.CY↗

arXiv2024

From Expectation to Habit: Why Do Software Practitioners Adopt Fairness Toolkits?

As the adoption of machine learning (ML) systems continues to grow across industries, concerns about fairness and bias in these systems have taken center stage. Fairness toolkits, designed to mitigate bias in ML models, serve as critical tools for addressing these ethical concerns. However, their adoption in the context of software development remains underexplored, especially regarding the cognitive and behavioral factors driving their usage. As a deeper understanding of these factors could be pivotal in refining tool designs and promoting broader adoption, this study investigates the factors influencing the adoption of fairness toolkits from an individual perspective. Guided by the Unified Theory of Acceptance and Use of Technology (UTAUT2), we examined the factors shaping the intention to adopt and actual use of fairness toolkits. Specifically, we employed Partial Least Squares Structural Equation Modeling (PLS-SEM) to analyze data from a survey study involving practitioners in the software industry. Our findings reveal that performance expectancy and habit are the primary drivers of fairness toolkit adoption. These insights suggest that by emphasizing the effectiveness of these tools in mitigating bias and fostering habitual use, organizations can encourage wider adoption. Practical recommendations include improving toolkit usability, integrating bias mitigation processes into routine development workflows, and providing ongoing support to ensure professionals see clear benefits from regular use.

cs.SE↗

arXiv2024

Investigating the Role of Cultural Values in Adopting Large Language Models for Software Engineering

As a socio-technical activity, software development involves the close interconnection of people and technology. The integration of Large Language Models (LLMs) into this process exemplifies the socio-technical nature of software development. Although LLMs influence the development process, software development remains fundamentally human-centric, necessitating an investigation of the human factors in this adoption. Thus, with this study we explore the factors influencing the adoption of LLMs in software development, focusing on the role of professionals' cultural values. Guided by the Unified Theory of Acceptance and Use of Technology (UTAUT2) and Hofstede's cultural dimensions, we hypothesized that cultural values moderate the relationships within the UTAUT2 framework. Using Partial Least Squares-Structural Equation Modelling and data from 188 software engineers, we found that habit and performance expectancy are the primary drivers of LLM adoption, while cultural values do not significantly moderate this process. These findings suggest that, by highlighting how LLMs can boost performance and efficiency, organizations can encourage their use, no matter the cultural differences. Practical steps include offering training programs to demonstrate LLM benefits, creating a supportive environment for regular use, and continuously tracking and sharing performance improvements from using LLMs.

cs.SE↗