arXiv subjects

Rick Kazman

Publications and source records attributed to Rick Kazman.

At least 19 recordsLinked to original sources

arXiv2026

Generative AI for trustworthy systems - Towards a health check model

The adoption of generative AI in software-intensive systems is proceeding rapidly, but current analytical instruments - principally unidimensional maturity models - compress important configurational variation into a single progressive axis. Drawing on an inductive interview study of eighteen senior practitioners across telecommunications, automotive, defence, aviation, banking, energy, government, and enterprise software services, this paper presents the Trustworthy Autonomy Health Check Model: a structured, multidimensional instrument for characterizing how organizations establish trust in GenAI-assisted software engineering. The model organizes eight empirically grounded dimensions into a system layer (Scope of Agent Authority, Assurance Mechanisms, Data Trustworthiness, Architectural Containment, Traceability & Comprehensibility) and an organizational layer (Governance, Human Oversight Posture, Workforce Capability Sustainability), each on a five-level ordinal scale. A cross-cutting overlay of four trust paradigms - operational, engineering, statistical, and containment-based - captures how trust is established, complementing the dimensions capturing what must be trustworthy. The model is diagnostic rather than prescriptive: it supports cross-organizational comparison, surfaces configurational trade-offs, and locates an organization in a shared space without imposing a single progression path. Higher levels are not inherently better; the goal is alignment across dimensions appropriate to the organization's domain and chosen trust paradigm. This is formalized as the alignment hypothesis: effective trustworthiness is constrained by the fit between the scope of agent authority and the dimensions enabling it, with misalignment producing pathological risk or unnecessary friction. We discuss how practitioners can apply the model and outline directions for empirical validation.

cs.SE↗

arXiv2026

From AI Technical Debt to Agentic Technical Debt: A Systematic Mapping of Root Causes and Manifestations in Agentic AI Systems

The emergence of Agentic AI systems, characterized by autonomous reasoning, multi-agent collaboration, tool orchestration, adaptive decision-making, and persistent memory, represents a fundamental shift from traditional AI pipelines to dynamic software ecosystems. While AI Technical Debt (AITD) has been widely studied in machine learning and software engineering, existing models assume static, component-level architectures and fail to capture the dynamic and emergent behaviors of agentic environments. To address this gap, this paper introduces Agentic Technical Debt (AgTD), defined as technical debt that emerges, accumulates, propagates, and amplifies due to the autonomous and collaborative nature of Agentic AI systems. Building on our prior systematic scoping review of 31 AITDs across seven root-cause categories, we employ a theory-informed transformation methodology to reinterpret these debts in Agentic AI through direct transformation, contextual transformation, and manifestation expansion. We present the first systematic mapping of established AITDs to their agentic manifestations, showing how conventional debts evolve into system-level liabilities, including memory inconsistencies, orchestration fragility, cascading failures, and unsafe autonomous decision-making. Our findings show that technical debt extends beyond software artifacts to encompass agent behaviors, coordination mechanisms, and interactions among agents, tools, and execution environments. We further examine its implications for AI Trust, Risk, and Security Management (AI TRiSM), highlighting impacts on trustworthiness, governance, security, operational resilience, and Sustainability Technical Debt. Overall, this work establishes AgTD as a foundational software engineering construct and provides a transformation framework, taxonomy, and research agenda for managing technical debt in autonomous multi-agent AI systems.

cs.AI↗

arXiv2026

On AI Safety and Security Technical Debt in Engineering AI-Enabled Systems

Artificial intelligence (AI) systems are increasingly deployed in high-stakes domains such as healthcare, autonomous driving, finance, and education. While these systems offer powerful data-driven and adaptive capabilities, their complexity, rapid evolution, and dependence on dynamic data pipelines introduce new forms of engineering liability collectively referred to as AI Technical Debts (AITDs). AITDs arise from root causes spanning data governance, model implementation, algorithm design, architectural decisions, operational processes, documentation practices, and testing adequacy. Unlike conventional technical debt, many AITDs are latent and propagate across tightly coupled AI pipelines, leading to maintenance challenges, reliability degradation, and heightened safety or security risks. Guided by the principles of AI Trust, Risk, and Security Management (AI TRiSM), this study reinterprets technical debt through the interconnected dimensions of trustworthiness, focusing on AI safety and security technical debts. We conduct a systematic review of 60 primary studies and identify 31 distinct types of AITD, which are organized into a root-cause-oriented taxonomy comprising seven classes. The analysis examines how these debts map to 18 trust-related concerns, including 6 safety hazards and 12 security vulnerabilities. To support mitigation, the review synthesizes 34 actionable guidelines (8 safety and 26 security) targeting the prevention, detection, and reduction of AITDs across the AI lifecycle. Building on these findings, we introduce AITD-MAP, an integrated framework that connects the AITD taxonomy, quality and risk impacts, and mitigation strategies into a unified structure for risk-aware AI engineering. The framework aims to assist AI software engineers in making AI safety and security technical debts visible, understanding their root causes, and mitigating their presence.

cs.SE↗

arXiv2026

Teaching Business Process Modeling to Leverage Soft Skills of Computing Students

Background and Context: Organizations have highly depended on software systems to realize their business processes and achieve strategic business goals and competitive advantage; hence, computing professionals who can understand these processes could more effectively build such systems. At the same time, while Artificial Intelligence (AI) has increased the productivity in developing these and other systems, professionals should be better prepared due to the competitiveness of the job market; thus, soft skills could be an important factor to ensure their position in this market. Objective: This paper aims to analyze and present the impact of teaching business process modeling to leverage key soft skills of future computing professionals. Method and Findings: Our method involved an industry-driven, project-based approach with 53 computing students who actively interacted with stakeholders and modeled the organization's business process. We systematically collected the students' feedback, including the soft skills practiced and demonstrating that professionalism, initiative, motivation, communication, and teamwork can be leveraged. Implications: The main implication of this work for the Computer Science Education research community is to provide evidence that teaching business process modeling subjects is particularly beneficial for future professionals, as it could contribute to preparing them regarding their soft skills for the era of AI-assisted software development.

cs.SE↗

arXiv2026

AI Writes Code, Humans Pay the Debt. An Empirical Study on the Sustainability and Evolution of Agent-Generated Code

Context. The increasing adoption of Generative AI coding agents in software engineering is transforming how developers implement and maintain code. While these systems provide short-term productivity benefits, their long-term impact on software quality and technical debt remains unclear. Aim. We aim to investigate how agent-generated code affects software quality, focusing on issue localization accuracy, the introduction of technical debt, and its evolution over time. Method. We will conduct a large-scale mining software repositories study using the SQuaD dataset, employing a candidate set of 628k issue tickets. We will generate agent-based implementations for these issues, and compare them with the actual commits done by developers using static analysis metrics and tools. We will analyze differences at the commit level and across releases, and we will consider multiple LLM-based Agents selected through a systematic benchmarking strategy. Expected Results. We expect to provide empirical evidence on the trade-offs introduced by agent-based development, including differences in localization accuracy, variations in technical debt introduction, and potential divergence in long-term evolution. We expect the results to highlight variability across LLMs, to enrich our understanding of software evolution with Agents, and to inform more responsible adoption of Agents in software development.

cs.SE↗

arXiv2026

Practitioner Views on Mobile App Accessibility: Practices and Challenges

As mobile applications (apps) become ubiquitous in everyday life, it is crucial for developers to prioritize accessibility for users with diverse abilities. While previous research has identified widespread accessibility issues and raised awareness of developer challenges, there remains a lack of cross-platform, globally representative insights into how practitioners approach accessibility in practice. This paper presents findings from a mixed-methods survey of 110 mobile app developers across 43 countries, examining how platform ecosystems (iOS vs. Android) and developer experience shape accessibility practices. Results show that while developers recognize the importance of accessibility, they often rely on platform-specific guidelines and typically perform compliance testing late in the development process. Developers primarily implement text-focused features while struggling with API limitations and organizational constraints. Through systematic cross-platform comparison, we identify novel platform-specific barriers and demonstrate how accessibility practices differ across developer experience levels. Our findings offer new insights into the challenges of achieving accessibility in practice and provide actionable steps for various stakeholders to promote more consistent and inclusive app development.

cs.SE↗

arXiv2025

AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents

The emergence of foundation models (FMs) has enabled the development of highly capable and autonomous agents, unlocking new application opportunities across a wide range of domains. Evaluating the architecture of agents is particularly important as the architectural decisions significantly impact the quality attributes of agents given their unique characteristics, including compound architecture, autonomous and non-deterministic behaviour, and continuous evolution. However, these traditional methods fall short in addressing the evaluation needs of agent architecture due to the unique characteristics of these agents. Therefore, in this paper, we present AgentArcEval, a novel agent architecture evaluation method designed specially to address the complexities of FM-based agent architecture and its evaluation. Moreover, we present a catalogue of agent-specific general scenarios, which serves as a guide for generating concrete scenarios to design and evaluate the agent architecture. We demonstrate the usefulness of AgentArcEval and the catalogue through a case study on the architecture evaluation of a real-world tax copilot, named Luna.

cs.SE↗

arXiv2025

Oops!... I did it again. Analysing and Handling Conclusion (In-)Stability in Socio-Technical Software Engineering

Context: Mining software repositories is a popular means to gain insights into a software project's evolution, monitor project health, support decisions and derive best practices. Tools supporting the mining process are commonly applied by researchers and practitioners, but their limitations and agreement are often not well understood. Objective: This study investigates some threats to validity in complex tool pipelines for evolutionary socio-technical software analyses. We evaluate the tools' agreement in terms of data, study outcomes and conclusions for the same research questions to derive actionable advice for researchers and practitioners. Method: We conduct a lightweight literature review to select \emph{three} studies on collaboration and coordination, software maintenance and software quality from high-ranked venues, which we formally replicate with \emph{four} independent, systematically selected mining tools to quantitatively and qualitatively compare the extracted data, analysis results and conclusions. Results: We find and summarise numerous technical details in tool design and implementation which accumulate along the complex mining pipelines and can cause substantial differences in the extracted baseline data, its derivatives, subsequent results of statistical analyses and, under specific circumstances, conclusions. Conclusions: Users should evaluate mining tools and their limitations carefully to scope the validity of their conclusions, for instance with the checklist we provide. Researchers and tool authors can further reduce uncertainty through reproduction packages and comparative studies in other MSR fields following our methodology.

cs.SE↗

arXiv2025

Moderately Mighty: To What Extent Can Internal Software Metrics Predict App Popularity at Launch?

Predicting a mobile app's popularity before its first release can provide developers with a strategic advantage in a competitive marketplace, yet it remains a challenging problem. This study explores the extent to which internal software metrics, measurable from source code before deployment, can predict an app's popularity (i.e., ratings and downloads per year) at inception. For our analysis, we constructed a rigorously filtered dataset of 446 open-source Java-based Android apps that are available on both F-Droid and Google Play Store. Using app source code from F-Droid, we extracted a wide array of internal metrics, including system-, class-, and method-level code metrics, code smells, and app metadata. Popularity-related information, including reviews and download counts, was collected from the Play Store. We evaluate regression and classification models across three feature sets: a minimal Size-only baseline, a domain-informed Handpicked set, and a Voting set derived via feature selection algorithms. Our results show that, for both app ratings and number of downloads, regression models perform poorly due to skewed rating distributions and a highly scattered range of download counts in our dataset. However, when reframed as a binary classification (Popular vs. Unpopular), performance improves significantly-the best model, a Multilayer Perceptron, achieves an F1-score of 0.72. We conclude that, although internal code metrics alone are insufficient for accurately predicting an app's future popularity, they do exhibit meaningful correlations with it. Thus, our findings challenge prior studies that have entirely dismissed internal metrics as valid indicators of software quality. Instead, our results align with research suggesting that internal code metrics can be valuable when evaluated within the appropriate context-specifically, we found them useful for classification tasks.

cs.SE↗

arXiv2025

An LLM-assisted approach to designing software architectures using ADD

Designing effective software architectures is a complex, iterative process that traditionally relies on expert judgment. This paper proposes an approach for Large Language Model (LLM)-assisted software architecture design using the Attribute-Driven Design (ADD) method. By providing an LLM with an explicit description of ADD, an architect persona, and a structured iteration plan, our method guides the LLM to collaboratively produce architecture artifacts with a human architect. We validate the approach through case studies, comparing generated designs against proven solutions and evaluating them with professional architects. Results show that our LLM-assisted ADD process can generate architectures closely aligned with established solutions and partially satisfying architectural drivers, highlighting both the promise and current limitations of using LLMs in architecture design. Our findings emphasize the importance of human oversight and iterative refinement when leveraging LLMs in this domain.

cs.SE↗

arXiv2025

Does the Tool Matter? Exploring Some Causes of Threats to Validity in Mining Software Repositories

Software repositories are an essential source of information for software engineering research on topics such as project evolution and developer collaboration. Appropriate mining tools and analysis pipelines are therefore an indispensable precondition for many research activities. Ideally, valid results should not depend on technical details of data collection and processing. It is, however, widely acknowledged that mining pipelines are complex, with a multitude of implementation decisions made by tool authors based on their interests and assumptions. This raises the questions if (and to what extent) tools agree on their results and are interchangeable. In this study, we use two tools to extract and analyse ten large software projects, quantitatively and qualitatively comparing results and derived data to better understand this concern. We analyse discrepancies from a technical point of view, and adjust code and parametrisation to minimise replication differences. Our results indicate that despite similar trends, even simple metrics such as the numbers of commits and developers may differ by up to 500%. We find that such substantial differences are often caused by minor technical details. We show how tool-level and data post-processing changes can overcome these issues, but find they may require considerable efforts. We summarise identified causes in our lessons learned to help researchers and practitioners avoid common pitfalls, and reflect on implementation decisions and their influence in ensuring obtained data meets explicit and implicit expectations. Our findings lead us to hypothesise that similar uncertainties exist in other analysis tools, which may limit the validity of conclusions drawn in tool-centric research.

cs.SE↗

arXiv2025

Leveraging Sustainable Systematic Literature Reviews

Systematic Literature Reviews (SLRs) are a widely employed research method in software engineering. However, there are several problems with SLRs, including the enormous time and effort to conduct them and the lack of obvious impacts of SLR results on software engineering practices and industry projects. To address these problems, the concepts of \textit{sustainability} and \textit{sustainable SLR} have been proposed, aiming to raise awareness among researchers about the importance of dealing with SLR problems in a consistent way; however, practical and concrete actions are still lacking. This paper presents concrete directions towards sustainable SLRs. We first identified 18 ``green drivers'' (GD) that could directly impact SLR sustainability, and we distilled 25 sustainability indicators (SI) associated with the GD to assess SLRs regarding their sustainability. A preliminary evaluation was conducted on the ten top-cited SLRs in software engineering published over the last decade. From this analysis, we synthesized our insights into 12 leverage points for sustainability. Our results indicate that even in high-quality reviews, there are threats to sustainability, such as: flaws in the search process, lack of essential details in the documentation, weak collaboration with stakeholders, poor knowledge management, lack of use of supporting tools, and a dearth of practical insights for software engineering practitioners. The good news is that moving towards sustainable SLRs only requires some simple actions, which can pave the way for a profound change in the software engineering community's mindset about how to create and sustain SLRs.

cs.SE↗

arXiv2024

Detection of Technical Debt in Java Source Code

Technical debt (TD) describes the additional costs that emerge when developers have opted for a quick and easy solution to a problem, rather than a more effective and well-designed, but time-consuming approach. Self-Admitted Technical Debts (SATDs) are a specific type of technical debts that developers intentionally document and acknowledge, typically via textual comments. While these comments are a useful tool for identifying TD, most of the existing approaches focus on capturing tokens associated with various categories of TD, neglecting the rich information embedded within the source code. Recent research has focused on detecting SATDs by analyzing comments, and there has been little work dealing with TD contained in the source code. In this study, through the analysis of comments and their source code from 974 Java projects, we curated the first ever dataset of TD identified by code comments, coupled with its code. We found that including the classified code significantly improves the accuracy in predicting various types of technical debt. We believe that our dataset will catalyze future work in the domain, inspiring various research related to the recognition of technical debt; The proposed classifiers may serve as baselines for studies on the detection of TD.

cs.SE↗

arXiv2024

A Systematic Mapping Study on Architectural Approaches to Software Performance Analysis

Software architecture is the foundation of a system's ability to achieve various quality attributes, including software performance. However, there lacks comprehensive and in-depth understanding of why and how software architecture and performance analysis are integrated to guide related future research. To fill this gap, this paper presents a systematic mapping study of 109 papers that integrate software architecture and performance analysis. We focused on five research questions that provide guidance for researchers and practitioners to gain an in-depth understanding of this research area. These questions addressed: a systematic mapping of related studies based on the high-level research purposes and specific focuses (RQ1), the software development activities these studies intended to facilitate (RQ2), the typical study templates of different research purposes (RQ3), the available tools and instruments for automating the analysis (RQ4), and the evaluation methodology employed in the studies (RQ5). Through these research questions, we also identified critical research gaps and future directions, including: 1) the lack of available tools and benchmark datasets to support replication, cross-validation and comparison of studies; 2) the need for architecture and performance analysis techniques that handle the challenges in emerging software domains; 3) the lack of consideration of practical factors that impact the adoption of the architecture and performance analysis approaches; and finally 4) the need for the adoption of modern ML/AI techniques to efficiently integrate architecture and performance analysis.

cs.SE↗

arXiv2024

Mobile App Security Trends and Topics: An Examination of Questions From Stack Overflow

The widespread use of smartphones and tablets has made society heavily reliant on mobile applications (apps) for accessing various resources and services. These apps often handle sensitive personal, financial, and health data, making app security a critical concern for developers. While there is extensive research on software security topics like malware and vulnerabilities, less is known about the practical security challenges mobile app developers face and the guidance they seek. In this study, we mine Stack Overflow for questions on mobile app security, which we analyze using quantitative and qualitative techniques. The findings reveal that Stack Overflow is a major resource for developers seeking help with mobile app security, especially for Android apps, and identifies seven main categories of security questions: Secured Communications, Database, App Distribution Service, Encryption, Permissions, File-Specific, and General Security. Insights from this research can inform the development of tools, techniques, and resources by the research and vendor community to better support developers in securing their mobile apps.

cs.CR↗

arXiv2024

Exploring Accessibility Trends and Challenges in Mobile App Development: A Study of Stack Overflow Questions

The proliferation of mobile applications (apps) has made it crucial to ensure their accessibility for users with disabilities. However, there is a lack of research on the real-world challenges developers face in implementing mobile accessibility features. This study presents a large-scale empirical analysis of accessibility discussions on Stack Overflow to identify the trends and challenges Android and iOS developers face. We examine the growth patterns, characteristics, and common topics mobile developers discuss. Our results show several challenges, including integrating assistive technologies like screen readers, ensuring accessible UI design, supporting text-to-speech across languages, handling complex gestures, and conducting accessibility testing. We envision our findings driving improvements in developer practices, research directions, tool support, and educational resources.

cs.SE↗

arXiv2024

A Developer-Centric Study Exploring Mobile Application Security Practices and Challenges

Mobile applications (apps) have become an essential part of everyday life, offering convenient access to services such as banking, healthcare, and shopping. With these apps handling sensitive personal and financial data, ensuring their security is paramount. While previous research has explored mobile app developer practices, there is limited knowledge about the common practices and challenges that developers face in securing their apps. Our study addresses this need through a global survey of 137 experienced mobile app developers, providing a developer-centric view of mobile app security. Our findings show that developers place high importance on security, frequently implementing features such as authentication and secure storage. They face challenges with managing vulnerabilities, permissions, and privacy concerns, and often rely on resources like Stack Overflow for help. Many developers find that existing learning materials do not adequately prepare them to build secure apps and provide recommendations, such as following best practices and integrating security at the beginning of the development process. We envision our findings leading to improved security practices, better-designed tools and resources, and more effective training programs.

cs.CR↗

arXiv2024

Explaining the Contributing Factors for Vulnerability Detection in Machine Learning

There is an increasing trend to mine vulnerabilities from software repositories and use machine learning techniques to automatically detect software vulnerabilities. A fundamental but unresolved research question is: how do different factors in the mining and learning process impact the accuracy of identifying vulnerabilities in software projects of varying characteristics? Substantial research has been dedicated in this area, including source code static analysis, software repository mining, and NLP-based machine learning. However, practitioners lack experience regarding the key factors for building a baseline model of the state-of-the-art. In addition, there lacks of experience regarding the transferability of the vulnerability signatures from project to project. This study investigates how the combination of different vulnerability features and three representative machine learning models impact the accuracy of vulnerability detection in 17 real-world projects. We examine two types of vulnerability representations: 1) code features extracted through NLP with varying tokenization strategies and three different embedding techniques (bag-of-words, word2vec, and fastText) and 2) a set of eight architectural metrics that capture the abstract design of the software systems. The three machine learning algorithms include a random forest model, a support vector machines model, and a residual neural network model. The analysis shows a recommended baseline model with signatures extracted through bag-of-words embedding, combined with the random forest, consistently increases the detection accuracy by about 4% compared to other combinations in all 17 projects. Furthermore, we observe the limitation of transferring vulnerability signatures across domains based on our experiments.

cs.SE↗