arXiv subjects

Andrea Bondavalli

Publications and source records attributed to Andrea Bondavalli.

5 recordsLinked to original sources

arXiv2023

Ensembling Uncertainty Measures to Improve Safety of Black-Box Classifiers

Machine Learning (ML) algorithms that perform classification may predict the wrong class, experiencing misclassifications. It is well-known that misclassifications may have cascading effects on the encompassing system, possibly resulting in critical failures. This paper proposes SPROUT, a Safety wraPper thROugh ensembles of UncertainTy measures, which suspects misclassifications by computing uncertainty measures on the inputs and outputs of a black-box classifier. If a misclassification is detected, SPROUT blocks the propagation of the output of the classifier to the encompassing system. The resulting impact on safety is that SPROUT transforms erratic outputs (misclassifications) into data omission failures, which can be easily managed at the system level. SPROUT has a broad range of applications as it fits binary and multi-class classification, comprising image and tabular datasets. We experimentally show that SPROUT always identifies a huge fraction of the misclassifications of supervised classifiers, and it is able to detect all misclassifications in specific cases. SPROUT implementation contains pre-trained wrappers, it is publicly available and ready to be deployed with minimal effort.

cs.LG↗

arXiv2021

Stochastic Activity Networks Templates: Supporting Variability in Performability Models

Model-based evaluation is extensively used to estimate performance and reliability of dependable systems. Traditionally, those systems were small and self-contained, and the main challenge for model-based evaluation has been the efficiency of the solution process. Recently, the problem of specifying and maintaining complex models has increasingly gained attention, as modern systems are characterized by many components and complex interactions. Components share similarities, but also exhibit variations in their behavior due to different configurations or roles in the system. From the modeling perspective, variations lead to replicating and altering a small set of base models multiple times. Variability is taken into account only informally, by defining a sample model and explaining its possible variations. In this paper we address the problem of including variability in performability models, focusing on Stochastic Activity Networks (SANs). We introduce the formal definition of Stochastic Activity Networks Templates (SAN-T), a formalism based on SANs with the addition of variability aspects. Differently from other approaches, parameters can also affect the structure of the model, like the number of cases of activities. We apply the SAN-T formalism to the modeling of the backbone network of an environmental monitoring infrastructure. In particular, we show how existing SAN models from the literature can be generalized using the newly introduced formalism.

cs.LO↗

arXiv2020

Unsupervised Anomaly Detectors to Detect Intrusions in the Current Threat Landscape

Anomaly detection aims at identifying unexpected fluctuations in the expected behavior of a given system. It is acknowledged as a reliable answer to the identification of zero-day attacks to such extent, several ML algorithms that suit for binary classification have been proposed throughout years. However, the experimental comparison of a wide pool of unsupervised algorithms for anomaly-based intrusion detection against a comprehensive set of attacks datasets was not investigated yet. To fill such gap, we exercise seventeen unsupervised anomaly detection algorithms on eleven attack datasets. Results allow elaborating on a wide range of arguments, from the behavior of the individual algorithm to the suitability of the datasets to anomaly detection. We conclude that algorithms as Isolation Forests, One-Class Support Vector Machines and Self-Organizing Maps are more effective than their counterparts for intrusion detection, while clustering algorithms represent a good alternative due to their low computational complexity. Further, we detail how attacks with unstable, distributed or non-repeatable behavior as Fuzzing, Worms and Botnets are more difficult to detect. Ultimately, we digress on capabilities of algorithms in detecting anomalies generated by a wide pool of unknown attacks, showing that achieved metric scores do not vary with respect to identifying single attacks.

cs.LG↗

arXiv2020

Governance & Autonomy: Towards a Governance-based Analysis of Autonomy in Cyber-Physical Systems-of-Systems

One of the main challenges in integrating Cyber-Physical System-of-Systems (CPSoS) to function as a single unified system is the autonomy of its Cyber-Physical Systems (CPSs), which may lead to a lack of coordination among CPSs and results in various kinds of conflicts. We advocate that to efficiently integrate CPSs within the CPSoS, we may need to adjust the autonomy of some CPSs in a way that allows them to coordinate their activities to avoid any potential conflict among one another. To achieve that, we need to incorporate the notion of governance within the design of CPSoS, which defines rules that can be used for clearly specifying who and how can adjust the autonomy of a CPS. In this paper, we try to tackle this problem by proposing a new conceptual model that can be used for performing a governance-based analysis of autonomy for CPSs within CPSoS. We illustrate the utility of the model with an example from the automotive domain.

cs.SE↗

arXiv2012

Modeling the resilience of large and evolving systems

This paper summarizes the state of knowledge and ongoing research on methods and techniques for resilience evaluation, taking into account the resilience-scaling challenges and properties related to the ubiquitous computerized systems. We mainly focus on quantitative evaluation approaches and, in particular, on model-based evaluation techniques that are commonly used to evaluate and compare, from the dependability point of view, different architecture alternatives at the design stage. We outline some of the main modeling techniques aiming at mastering the largeness of analytical dependability models at the construction level. Actually, addressing the model largeness problem is important with respect to the investigation of the scalability of current techniques to meet the complexity challenges of ubiquitous systems. Finally we present two case studies in which some of the presented techniques are applied for modeling web services and General Packet Radio Service (GPRS) mobile telephone networks, as prominent examples of large and evolving systems.

cs.PF↗