arXiv subjects

Michael Unterkalmsteiner

Publications and source records attributed to Michael Unterkalmsteiner.

At least 19 recordsLinked to original sources

arXiv2026

Replications, Revisions, and Reanalyses: Managing Empirical Evidence in Software Engineering

One aspired outcome of empirical research on quantitative data is a variance theory, i.e., a quantification of the effect of an independent on a dependent variables. The validity of variance theories stems from the synthesis of multiple pieces of evidence, which increases its validity beyond the findings of a single study. However, research synthesis in SE is rare and if done mostly limited to purely narrative syntheses. At best, researchers perform meta-analyses to synthesize variance theories from several quantitative results. But even meta-analyses only produce reliable results when synthesizing exact replications yet fail to generalize from variations. We aim to extend the frontier of research synthesis beyond the state-of-the-art to systematically manage empirical evidence and its evolution. We apply method engineering to construct a framework for research synthesis from proven, individual method fragments. The framework allows researchers to put new evidence in a clear relation to an existing body of evidence and systematically expand knowledge about a studied phenomenon. We demonstrate the application of this framework to two fields of research by explicitly modeling the relationship between existing pieces of evidence. The framework puts three types of evolution of evidence into relation: (1) replications investigate the same hypothesis in a new context to improve external validity, (2) revisions challenge an existing hypothesis to improve internal validity, and (3) reanalyses replace analysis methods to improve conclusion validity. Through a systematic evolution of evidence and clear assessment criteria for each dimension of validity, the proposed framework can determine the frontier of a field of research. The framework provides a perspective to systematically evolve empirical evidence in SE, supporting more constructive and productive advances in our field.

cs.SE↗

arXiv2026

Empirical Evaluation of Taxonomic Trace Links: A Case Study

Context: Traceability is a key quality attribute of artifacts that are used in knowledge-intensive tasks and supports software engineers in producing higher-quality software. Despite its clear benefits, traceability is often neglected in practice due to challenges such as granularity of traces, lack of a common artifact structure, and unclear responsibility. The Taxonomic Trace Links (TTL) approach connects source and target artifacts through a domain-specific taxonomy, aiming to address these common traceability challenges. Objective: In this study, we empirically evaluate TTL in an industrial setting to identify its strengths and weaknesses for real-world adoption. Method: We conducted a mixed-methods study at Ericsson involving one of its software products. Quantitative and qualitative data were collected across two traceability use cases. We established trace links between 463 business use cases, 64 test cases, and 277 ISO-standard requirements. Additionally, we held three focus group sessions with practitioners. Results: We identified two practically relevant scenarios where traceability is required and evaluated TTL in each. Overall, practitioners found TTL to be a useful solution for one of the scenarios, while less useful for the other. However, developing a domain-specific taxonomy and managing heterogeneous artifact structures were noted as significant challenges. Moreover, the precision of the classifier that is used to create trace links needs to be improved to make the solution practical. Conclusion: TTL is a promising approach that can be adopted in practice and enables traceability use cases. However, TTL is not a replacement for traditional trace links, but rather complements them to enable more traceability use cases and encourage the early creation of trace links.

cs.SE↗

arXiv2026

Experience Report on the Adaptable Integration of Requirements Engineering Courses into Curricula for Professionals

There is a growing demand for software engineering education (SEE) for professionals because of the increasing demand, active evolution of the technological landscape, and changes in the skills required by the practice. Integrating requirements engineering (RE) courses into SEE curricula for professionals systematically and effectively is challenging. In particular, curricula for professionals have different demands, are more dynamic, and modular in nature. In this study, we report on our experience in the development of three SEE curricula for professionals and the integration of RE courses into such curricula. We suggest basic principles for such integration and describe the systematic approach focused on course content mapping that we have developed.

cs.SE↗

arXiv2025

Privacy by Design: Aligning GDPR and Software Engineering Specifications with a Requirements Engineering Approach

Context: Consistent requirements and system specifications are essential for the compliance of software systems towards the General Data Protection Regulation (GDPR). Both artefacts need to be grounded in the original text and conjointly assure the achievement of privacy by design (PbD). Objectives: There is little understanding of the perspectives of practitioners on specification objectives and goals to address PbD. Existing approaches do not account for the complex intersection between problem and solution space expressed in GDPR. In this study we explore the demand for conjoint requirements and system specification for PbD and suggest an approach to address this demand. Methods: We reviewed secondary and related primary studies and conducted interviews with practitioners to (1) investigate the state-of-practice and (2) understand the underlying specification objectives and goals (e.g., traceability). We developed and evaluated an approach for requirements and systems specification for PbD, and evaluated it against the specification objectives. Results: The relationship between problem and solution space, as expressed in GDPR, is instrumental in supporting PbD. We demonstrate how our approach, based on the modeling GDPR content with original legal concepts, contributes to specification objectives of capturing legal knowledge, supporting specification transparency, and traceability. Conclusion: GDPR demands need to be addressed throughout different levels of abstraction in the engineering lifecycle to achieve PbD. Legal knowledge specified in the GDPR text should be captured in specifications to address the demands of different stakeholders and ensure compliance. While our results confirm the suitability of our approach to address practical needs, we also revealed specific needs for the future effective operationalization of the approach.

cs.SE↗

arXiv2025

Quality attributes of test cases and test suites -- importance & challenges from practitioners' perspectives

Context: The quality of the test suites and the constituent test cases significantly impacts confidence in software testing. While research has identified several quality attributes of test cases and test suites, there is a need for a better understanding of their relative importance in practice. Objective: We investigate practitioners' perceptions regarding the relative importance of quality attributes of test cases and test suites and the challenges they face in ensuring the perceived important quality attributes. Method: We conducted an industrial survey using a questionnaire based on the quality attributes identified in an extensive literature review. We used a sampling strategy that leverages LinkedIn to draw a large and heterogeneous sample of professionals with experience in software testing. Results: We collected 354 responses from practitioners with a wide range of experience. We found that the majority of practitioners rated Fault Detection, Usability, Maintainability, Reliability, and Coverage to be the most important quality attributes. Resource Efficiency, Reusability, and Simplicity received the most divergent opinions, which, according to our analysis, depend on the software-testing contexts. We identified common challenges that apply to the important attributes, namely inadequate definition, lack of useful metrics, lack of an established review process, and lack of external support. Conclusion: The findings point out where practitioners actually need further support with respect to achieving high-quality test cases and test suites under different software testing contexts. The findings can serve as a guideline for academic researchers when looking for research directions on the topic. The findings can also be used to encourage companies to provide more support to practitioners to achieve high-quality test cases and test suites.

cs.SE↗

arXiv2025

A proposal and assessment of an improved heuristic for the Eager Test smell detection

Context: The evidence for the prevalence of test smells at the unit testing level has relied on the accuracy of detection tools, which have seen intense research in the last two decades. The Eager Test smell, one of the most prevalent, is often identified using simplified detection rules that practitioners find inadequate. Objective: We aim to improve the rules for detecting the Eager Test smell. Method: We reviewed the literature on test smells to analyze the definitions and detection rules of the Eager Test smell. We proposed a novel, unambiguous definition of the test smell and a heuristic to address the limitations of the existing rules. We evaluated our heuristic against existing detection rules by manually applying it to 300 unit test cases in Java. Results: Our review identified 56 relevant studies. We found that inadequate interpretations of original definitions of the Eager Test smell led to imprecise detection rules, resulting in a high level of disagreement in detection outcomes. Also, our heuristic detected patterns of eager and non-eager tests that existing rules missed. Conclusion: Our heuristic captures the essence of the Eager Test smell more precisely; hence, it may address practitioners' concerns regarding the adequacy of existing detection rules.

cs.SE↗

arXiv2025

Taxonomic Trace Links: Rethinking Traceability and its Benefits

Traceability greatly supports knowledge-intensive tasks, e.g., coverage check and impact analysis. Despite its clear benefits, the \emph{practical} implementation of traceability poses significant challenges, leading to a reduced focus on the creation and maintenance of trace links. We propose a new approach -- Taxonomic Trace Links (TTL) -- which rethinks traceability and its benefits. With TTL, trace links are created indirectly through a domain-specific taxonomy, a simplified version of a domain model. TTL has the potential to address key traceability challenges, such as the granularity of trace links, the lack of a common data structure among software development artifacts, and unclear responsibility for traceability. We explain how TTL addresses these challenges and perform an initial validation with practitioners. We identified six challenges associated with TTL implementation that need to be addressed. Finally, we propose a research roadmap to further develop and evaluate the technical solution of TTL. TTL appears to be particularly feasible in practice where a domain taxonomy is already established

cs.SE↗

arXiv2025

Challenges of Requirements Communication and Digital Assets Verification in Infrastructure Projects

Background: Poor communication of requirements between clients and suppliers contributes to project overruns,in both software and infrastructure projects. Existing literature offers limited insights into the communication challenges at this interface. Aim: Our research aim to explore the processes and associated challenges with requirements activities that include client-supplier interaction and communication. Method: we study requirements validation, communication, and digital asset verification processes through two case studies in the road and railway sectors, involving interviews with ten experts across three companies. Results: We identify 13 challenges, along with their causes and consequences, and suggest solution areas from existing literature. Conclusion: Interestingly, the challenges in infrastructure projects mirror those found in software engineering, highlighting a need for further research to validate potential solutions.

cs.SE↗

arXiv2025

Auxiliary Artifacts in Requirements Traceability: A Systematic Mapping Study

Background: Traceability between software artifacts enhances the value of the information those artifacts contain, but only when the links themselves are reliable. Link quality is known to depend on explicit factors such as the traced artifacts and the expertise of the practitioner who judges each connection. Other factors, however, remain largely unexplored. We contend that one of these factors is the set of auxiliary artifacts -- artifacts that are produced and/or used during the tracing process yet are neither the source nor target artifacts. Because such auxiliary artifacts can subtly steer how links are created and validated, they merit a literature survey to identify these artifacts and further investigate them. Objective: We identify and map auxiliary artifacts used in requirements tracing, which could be additional factors that affect the quality of the trace links. Method: We conducted a systematic mapping study on auxiliary artifacts in requirements traceability. Results: We found 110 studies in which auxiliary artifacts are used in requirements tracing, and identified 49 auxiliary artifacts, and 13 usage scenarios. Conclusion: This study provides a systematic mapping of auxiliary artifacts in requirement tracing, including their usage, origin, type and tool support. The use of auxiliary artifacts in requirements tracing seems to be the norm, thus, these artifacts should be studied in depth to identify how they effect the quality of traced links.

cs.SE↗

arXiv2025

Language Models to Support Multi-Label Classification of Industrial Data

Multi-label requirements classification is a challenging task, especially when dealing with numerous classes at varying levels of abstraction. The difficulties increases when a limited number of requirements is available to train a supervised classifier. Zero-shot learning (ZSL) does not require training data and can potentially address this problem. This paper investigates the performance of zero-shot classifiers (ZSCs) on a multi-label industrial dataset. We focuse on classifying requirements according to a taxonomy designed to support requirements tracing. We compare multiple variants of ZSCs using different embeddings, including 9 language models (LMs) with a reduced number of parameters (up to 3B), e.g., BERT, and 5 large LMs (LLMs) with a large number of parameters (up to 70B), e.g., Llama. Our ground truth includes 377 requirements and 1968 labels from 6 output spaces. For the evaluation, we adopt traditional metrics, i.e., precision, recall, F1, and $F_\beta$, as well as a novel label distance metric Dn. This aims to better capture the classification's hierarchical nature and provides a more nuanced evaluation of how far the results are from the ground truth. 1) The top-performing model on 5 out of 6 output spaces is T5-xl, with maximum $F_\beta$ = 0.78 and Dn = 0.04, while BERT base outperformed the other models in one case, with maximum $F_\beta$ = 0.83 and Dn = 0.04. 2) LMs with smaller parameter size produce the best classification results compared to LLMs. Thus, addressing the problem in practice is feasible as limited computing power is needed. 3) The model architecture (autoencoding, autoregression, and sentence-to-sentence) significantly affects the classifier's performance. We conclude that using ZSL for multi-label requirements classification offers promising results. We also present a novel metric that can be used to select the top-performing model for this problem

cs.SE↗

arXiv2024

Replications, Revisions, and Reanalyses: Managing Variance Theories in Software Engineering

Variance theories quantify the variance that one or more independent variables cause in a dependent variable. In software engineering (SE), variance theories are used to quantify -- among others -- the impact of tools, techniques, and other treatments on software development outcomes. To acquire variance theories, evidence from individual empirical studies needs to be synthesized to more generally valid conclusions. However, research synthesis in SE is mostly limited to meta-analysis, which requires homogeneity of the synthesized studies to infer generalizable variance. In this paper, we aim to extend the practice of research synthesis beyond meta-analysis. To this end, we derive a conceptual framework for the evolution of variance theories and demonstrate its use by applying it to an active research field in SE. The resulting framework allows researchers to put new evidence in a clear relation to an existing body of knowledge and systematically expand the scientific frontier of a studied phenomenon.

cs.SE↗

arXiv2024

Systematic Mapping Study on Requirements Engineering for Regulatory Compliance of Software Systems

Context: As the diversity and complexity of regulations affecting Software-Intensive Products and Services (SIPS) is increasing, software engineers need to address the growing regulatory scrutiny. As with any other non-negotiable requirements, SIPS compliance should be addressed early in SIPS engineering - i.e., during requirements engineering (RE). Objectives: In the conditions of the expanding regulatory landscape, existing research offers scattered insights into regulatory compliance of SIPS. This study addresses the pressing need for a structured overview of the state of the art in software RE and its contribution to regulatory compliance of SIPS. Method: We conducted a systematic mapping study to provide an overview of the current state of research regarding challenges, principles and practices for regulatory compliance of SIPS related to RE. We focused on the role of RE and its contribution to other SIPS lifecycle phases. We retrieved 6914 studies published from 2017 until 2023 from four academic databases, which we filtered down to 280 relevant primary studies. Results: We identified and categorized the RE-related challenges in regulatory compliance of SIPS and their potential connection to six types of principles and practices. We found that about 13.6% of the primary studies considered the involvement of both software engineers and legal experts. About 20.7% of primary studies considered RE in connection to other process areas. Most primary studies focused on a few popular regulation fields and application domains. Our results suggest that there can be differences in terms of challenges and involvement of stakeholders across different fields of regulation. Conclusion: Our findings highlight the need for an in-depth investigation of stakeholders' roles, relationships between process areas, and specific challenges for distinct regulatory fields to guide research and practice.

cs.SE↗

arXiv2024

Regulatory Requirements Engineering in Large Enterprises: An Interview Study on the European Accessibility Act

Context: Regulations, such as the European Accessibility Act (EAA), impact the engineering of software products and services. Managing that impact while providing meaningful inputs to development teams is one of the emerging requirements engineering (RE) challenges. Problem: Enterprises conduct Regulatory Impact Analysis (RIA) to consider the effects of regulations on software products offered and formulate requirements at an enterprise level. Despite its practical relevance, we are unaware of any studies on this large-scale regulatory RE process. Methodology: We conducted an exploratory interview study of RIA in three large enterprises. We focused on how they conduct RIA, emphasizing cross-functional interactions, and using the EAA as an example. Results: RIA, as a regulatory RE process, is conducted to address the needs of executive management and central functions. It involves coordination between different functions and levels of enterprise hierarchy. Enterprises use artifacts to support interpretation and communication of the results of RIA. Challenges to RIA are mainly related to the execution of such coordination and managing the knowledge involved. Conclusion: RIA in large enterprises demands close coordination of multiple stakeholders and roles. Applying interpretation and compliance artifacts is one approach to support such coordination. However, there are no established practices for creating and managing such artifacts.

cs.SE↗

arXiv2024

Multi-Label Requirements Classification with Large Taxonomies

Classification aids software development activities by organizing requirements in classes for easier access and retrieval. The majority of requirements classification research has, so far, focused on binary or multi-class classification. Multi-label classification with large taxonomies could aid requirements traceability but is prohibitively costly with supervised training. Hence, we investigate zero-short learning to evaluate the feasibility of multi-label requirements classification with large taxonomies. We associated, together with domain experts from the industry, 129 requirements with 769 labels from taxonomies ranging between 250 and 1183 classes. Then, we conducted a controlled experiment to study the impact of the type of classifier, the hierarchy, and the structural characteristics of taxonomies on the classification performance. The results show that: (1) The sentence-based classifier had a significantly higher recall compared to the word-based classifier; however, the precision and F1-score did not improve significantly. (2) The hierarchical classification strategy did not always improve the performance of requirements classification. (3) The total and leaf nodes of the taxonomies have a strong negative correlation with the recall of the hierarchical sentence-based classifier. We investigate the problem of multi-label requirements classification with large taxonomies, illustrate a systematic process to create a ground truth involving industry participants, and provide an analysis of different classification pipelines using zero-shot learning.

cs.SE↗

arXiv2024

Requirements Quality Research Artifacts: Recovery, Analysis, and Management Guideline

Requirements quality research, which is dedicated to assessing and improving the quality of requirements specifications, is dependent on research artifacts like data sets (containing information about quality defects) and implementations (automatically detecting and removing these defects). However, recent research exposed that the majority of these research artifacts have become unavailable or have never been disclosed, which inhibits progress in the research domain. In this work, we aim to improve the availability of research artifacts in requirements quality research. To this end, we (1) extend an artifact recovery initiative, (2) empirically evaluate the reasons for artifact unavailability using Bayesian data analysis, and (3) compile a concise guideline for open science artifact disclosure. Our results include 10 recovered data sets and 7 recovered implementations, empirical support for artifact availability improving over time and the positive effect of public hosting services, and a pragmatic artifact management guideline open for community comments. With this work, we hope to encourage and support adherence to open science principles and improve the availability of research artifacts for the requirements research quality community.

cs.SE↗

arXiv2024

Measuring the Fitness-for-Purpose of Requirements: An initial Model of Activities and Attributes

Requirements engineering aims to fulfill a purpose, i.e., inform subsequent software development activities about stakeholders' needs and constraints that must be met by the system under development. The quality of requirements artifacts and processes is determined by how fit for this purpose they are, i.e., how they impact activities affected by them. However, research on requirements quality lacks a comprehensive overview of these activities and how to measure them. In this paper, we specify the research endeavor addressing this gap and propose an initial model of requirements-affected activities and their attributes. We construct a model from three distinct data sources, including both literature and empirical data. The results yield an initial model containing 24 activities and 16 attributes quantifying these activities. Our long-term goal is to develop evidence-based decision support on how to optimize the fitness for purpose of the RE phase to best support the subsequent, affected software development process. We do so by measuring the effect that requirements artifacts and processes have on the attributes of these activities. With the contribution at hand, we invite the research community to critically discuss our research roadmap and support the further evolution of the model.

cs.SE↗

arXiv2024

Practices, Challenges, and Opportunities When Inferring Requirements From Regulations in the FinTech Sector - An Industrial Study

[Context and motivation]: Understanding and interpreting regulatory norms and inferring software requirements from them is a critical step towards regulatory compliance, a matter of significant importance in various industrial sectors. [Question/ problem]: However, interpreting regulations still largely depends on individual legal expertise and experience within the respective domain, with little to no systematic methodologies and supportive tools to guide this practice. In fact, research in this area is too often detached from practitioners' experiences, rendering the proposed solutions not transferable to industrial practice. As we argue, one reason is that we still lack a profound understanding of industry- and domain-specific practices and challenges. [Principal ideas/ results]: We aim to close this gap and provide such an investigation at the example of the banking and insurance domain. We conduct an industrial multi-case study as part of a long-term academia-industry collaboration with a medium-sized software development and renovation company. We explore contemporary industrial practices and challenges when inferring requirements from regulations to support more problem-driven research. Our study investigates the complexities of requirement engineering in regulatory contexts, pinpointing various issues and discussing them in detail. We highlight the gathered insights and the practical challenges encountered and suggest avenues for future research. [Contribution]: Our contribution is a comprehensive case study focused on the FinTech domain, offering a detailed understanding of the specific needs within this sector. We have identified key practices for managing regulatory requirements in software development, and have pinpointed several challenges. We conclude by offering a set of recommendations for future problem-driven research directions.

cs.SE↗

arXiv2024

On Developing an Artifact-based Approach to Regulatory Requirements Engineering

Context: Regulatory acts are a challenging source when eliciting, interpreting, and analyzing requirements. Requirements engineers often need to involve legal experts who, however, may often not be available. This raises the need for approaches to regulatory Requirements Engineering (RE) covering and integrating both legal and engineering perspectives. Problem: Regulatory RE approaches need to capture and reflect both the elementary concepts and relationships from a legal perspective and their seamless transition to concepts used to specify software requirements. No existing approach considers explicating and managing legal domain knowledge and engineering-legal coordination. Method: We conducted focus group sessions with legal researchers to identify the core challenges to establishing a regulatory RE approach. Based on our findings, we developed a candidate solution and conducted a first conceptual validation to assess its feasibility. Results: We introduce the first version of our Artifact Model for Regulatory Requirements Engineering (AM4RRE) and its conceptual foundation. It provides a blueprint for applying legal (modelling) concepts and well-established RE concepts. Our initial results suggest that artifact-centric RE can be applied to managing legal domain knowledge and engineering-legal coordination. Conclusions: The focus groups that served as a basis for building our model and the results from the expert validation both strengthen our confidence that we already provide a valuable basis for systematically integrating legal concepts into RE. This overcomes contemporary challenges to regulatory RE and serves as a basis for exposure to critical discussions in the community before continuing with the development of tool-supported extensions and large-scale empirical evaluations in practice.

cs.SE↗