arXiv subjects

Simon Diemert

Publications and source records attributed to Simon Diemert.

6 recordsLinked to original sources

arXiv2026

A Scalability Analysis of Quantitative Confidence Assessment Methods for Assurance Cases

This paper proposes a model to estimate the decision complexity and effort required to apply quantitative confidence assessment methods to assurance cases. The model considers both the worst and average case for these measures and characterizes how these quantities scale with argument size. Prior work has indicated that the additional effort required to apply these methods is a barrier to their adoption by assurance case practitioners. Researchers developing new methods, or improving existing methods, can use this model to estimate the effort required to apply their method. The proposed model is parameterized using data from published case studies and is applied to three existing quantitative confidence assessment methods: the Bayesian Belief Network method, the Dempster-Shafer Theory method, and the Certus method. The results show that, while Certus has the highest worst-case decision complexity, its average-case effort is lower than the BBN and DST methods.

cs.SE↗

arXiv2025

Certus: A domain specific language for confidence assessment in assurance cases

Assurance cases (ACs) are prepared to argue that a system has satisfied critical quality attributes. Many methods exist to assess confidence in ACs, including quantitative methods that represent confidence numerically. While quantitative methods are attractive in principle, existing methods suffer from issues related to interpretation, subjectivity, scalability, dialectic reasoning, and trustworthiness, which have limited their adoption. This paper introduces Certus, a domain specific language for quantitative confidence assessment. In Certus, users describe their confidence with fuzzy sets, which allow them to represent their judgment using vague, but linguistically meaningful terminology. Certus includes syntax to specify confidence propagation using expressions that can be easily inspected by users. To demonstrate the concept of the language, Certus is applied to a worked example from the automotive domain.

cs.SE↗

arXiv2024

How do practitioners gain confidence in assurance cases?

CONTEXT: Assurance Cases (ACs) are prepared to argue that the system's desired quality attributes (e.g., safety or security) are satisfied. While there is strong adoption of ACs, practitioners are often left asking an important question: are we confident that the claims made by the case are true? While many confidence assessment methods (CAMs) exist, little is known about the use of these methods in practice OBJECTIVE: Develop an understanding of the current state of practice for AC confidence assessment: what methods are used in practice and what barriers exist for their use? METHOD: Structured interviews were performed with practitioners with experience contributing to real-world ACs. Open-coding was performed on transcripts. A description of the current state of AC practice and future considerations for researchers was synthesized from the results. RESULTS: A total of n = 19 practitioners were interviewed. The most common CAMs were (peer-)review of ACs, dialectic reasoning ("defeaters"), and comparing against checklists. Participants preferred qualitative methods and expressed concerns about quantitative CAMs. Barriers to using CAMs included additional work, inadequate guidance, subjectivity and interpretation of results, and trustworthiness of methods. CONCLUSION: While many CAMs are described in the literature there is a gap between the proposed methods and needs of practitioners. Researchers working in this area should consider the need to: connect CAMs to established practices, use CAMs to communicate with interest holders, crystallize the details of CAM application, curate accessible guidance, and confirm that methods are trustworthy.

cs.SE↗

arXiv2023

Hazard Analysis for Self-Adaptive Systems Using System-Theoretic Process Analysis

Self-adaptive systems are able to change their behaviour at run-time in response to changes. Self-adaptation is an important strategy for managing uncertainty that is present during the design of modern systems, such as autonomous vehicles. However, assuring the safety of self-adaptive systems remains a challenge, particularly when the adaptations have an impact on safety-critical functions. The field of safety engineering has established practices for analyzing the safety of systems. System Theoretic Process and Analysis (STPA) is a hazard analysis method that is well-suited for self-adaptive systems. This paper describes a design-time extension of STPA for self-adaptive systems. Then, it derives a reference model and analysis obligations to support the STPA activities. The method is applied to three self-adaptive systems described in the literature. The results demonstrate that STPA, when used in the manner described, is an applicable hazard analysis method for safety-critical self-adaptive systems.

eess.SY↗

arXiv2023

Can Large Language Models assist in Hazard Analysis?

Large Language Models (LLMs), such as GPT-3, have demonstrated remarkable natural language processing and generation capabilities and have been applied to a variety tasks, such as source code generation. This paper explores the potential of integrating LLMs in the hazard analysis for safety-critical systems, a process which we refer to as co-hazard analysis (CoHA). In CoHA, a human analyst interacts with an LLM via a context-aware chat session and uses the responses to support elicitation of possible hazard causes. In this experiment, we explore CoHA with three increasingly complex versions of a simple system, using Open AI's ChatGPT service. The quality of ChatGPT's responses were systematically assessed to determine the feasibility of CoHA given the current state of LLM technology. The results suggest that LLMs may be useful for supporting human analysts performing hazard analysis.

cs.HC↗

arXiv2022

Safety-Critical Adaptation in Self-Adaptive Systems

Modern systems are designed to operate in increasingly variable and uncertain environments. Not only are these environments complex, in the sense that they contain a tremendous number of variables, but they also change over time. Systems must be able to adjust their behaviour at run-time to manage these uncertainties. These self-adaptive systems have been studied extensively. This paper proposes a definition of a safety-critical self-adaptive system and then describes a taxonomy for classifying adaptations into different types based on their impact on the system's safety and the system's safety case. The taxonomy expresses criteria for classification and then describes specific criteria that the safety case for a self-adaptive system must satisfy, depending on the type of adaptations performed. Each type in the taxonomy is illustrated using the example of a safety-critical self-adaptive water heating system.

cs.SE↗