arXiv subjects

Paolo Arcaini

Publications and source records attributed to Paolo Arcaini.

At least 37 records · Page 2Linked to original sources

arXiv2025

Search-based Generation of Waypoints for Triggering Self-Adaptations in Maritime Autonomous Vessels

Self-adaptation in maritime autonomous vessels (AVs) enables them to adapt their behaviors to address unexpected situations while maintaining dependability requirements. During the design of such AVs, it is crucial to understand and identify the settings that should trigger adaptations, enabling validation of their implementation. To this end, we focus on the navigation software of AVs, which must adapt their behavior during operation through adaptations. AVs often rely on predefined waypoints to guide them along designated routes, ensuring safe navigation. We propose a multiobjective search-based approach, called WPgen, to generate minor modifications to the predefined set of waypoints, keeping them as close as possible to the original waypoints, while causing the AV to navigate inappropriately when navigating with the generated waypoints. WPgen uses NSGA-II as the multi-objective search algorithm with three seeding strategies for its initial population, resulting in three variations of WPgen. We evaluated these variations on three AVs (one overwater tanker and two underwater). We compared the three variations of WPgen with Random Search as the baseline and with each other. Experimental results showed that the effectiveness of these variations varied depending on the AV. Based on the results, we present the research and practical implications of WPgen.

cs.SE↗

arXiv2025

Foundation Models for the Digital Twin Creation of Cyber-Physical Systems

Foundation models are trained on a large amount of data to learn generic patterns. Consequently, these models can be used and fine-tuned for various purposes. Naturally, studying such models' use in the context of digital twins for cyber-physical systems (CPSs) is a relevant area of investigation. To this end, we provide perspectives on various aspects within the context of developing digital twins for CPSs, where foundation models can be used to increase the efficiency of creating digital twins, improve the effectiveness of the capabilities they provide, and used as specialized fine-tuned foundation models acting as digital twins themselves. We also discuss challenges in using foundation models in a more generic context. We use the case of an autonomous driving system as a representative CPS to give examples. Finally, we provide discussions and open research directions that we believe are valuable for the digital twin community.

cs.SE↗

arXiv2025

MultiDrive: A Co-Simulation Framework Bridging 2D and 3D Driving Simulation for AV Software Validation

Scenario-based testing using simulations is a cornerstone of Autonomous Vehicles (AVs) software validation. So far, developers needed to choose between low-fidelity 2D simulators to explore the scenario space efficiently, and high-fidelity 3D simulators to study relevant scenarios in more detail, thus reducing testing costs while mitigating the sim-to-real gap. This paper presents a novel framework that leverages multi-agent co-simulation and procedural scenario generation to support scenario-based testing across low- and high-fidelity simulators for the development of motion planning algorithms. Our framework limits the effort required to transition scenarios between simulators and automates experiment execution, trajectory analysis, and visualization. Experiments with a reference motion planner show that our framework uncovers discrepancies between the planner's intended and actual behavior, thus exposing weaknesses in planning assumptions under more realistic conditions. Our framework is available at: https://github.com/TUM-AVS/MultiDrive

cs.RO↗

arXiv2025

Quantum Circuit Mutants: Empirical Analysis and Recommendations

As a new research area, quantum software testing lacks systematic testing benchmarks to assess testing techniques' effectiveness. Recently, some open-source benchmarks and mutation analysis tools have emerged. However, there is insufficient evidence on how various quantum circuit characteristics (e.g., circuit depth, number of quantum gates), algorithms (e.g., Quantum Approximate Optimization Algorithm), and mutation characteristics (e.g., mutation operators) affect the detection of mutants in quantum circuits. Studying such relations is important to systematically design faulty benchmarks with varied attributes (e.g., the difficulty in detecting a seeded fault) to facilitate assessing the cost-effectiveness of quantum software testing techniques efficiently. To this end, we present a large-scale empirical evaluation with more than 700K faulty benchmarks (quantum circuits) generated by mutating 382 real-world quantum circuits. Based on the results, we provide valuable insights for researchers to define systematic quantum mutation analysis techniques. We also provide a tool to recommend mutants to users based on chosen characteristics (e.g., a quantum algorithm type) and the required difficulty of detecting mutants. Finally, we also provide faulty benchmarks that can already be used to assess the cost-effectiveness of quantum software testing techniques.

cs.SE↗

arXiv2024

Quantum Software Engineering: Roadmap and Challenges Ahead

As quantum computers advance, the complexity of the software they can execute increases as well. To ensure this software is efficient, maintainable, reusable, and cost-effective -key qualities of any industry-grade software-mature software engineering practices must be applied throughout its design, development, and operation. However, the significant differences between classical and quantum software make it challenging to directly apply classical software engineering methods to quantum systems. This challenge has led to the emergence of Quantum Software Engineering as a distinct field within the broader software engineering landscape. In this work, a group of active researchers analyse in depth the current state of quantum software engineering research. From this analysis, the key areas of quantum software engineering are identified and explored in order to determine the most relevant open challenges that should be addressed in the next years. These challenges help identify necessary breakthroughs and future research directions for advancing Quantum Software Engineering.

cs.SE↗

arXiv2024

Quantum Neural Network Classifier for Cancer Registry System Testing: A Feasibility Study

The Cancer Registry of Norway (CRN) is a part of the Norwegian Institute of Public Health (NIPH) and is tasked with producing statistics on cancer among the Norwegian population. For this task, CRN develops, tests, and evolves a software system called Cancer Registration Support System (CaReSS). It is a complex socio-technical software system that interacts with many entities (e.g., hospitals, medical laboratories, and other patient registries) to achieve its task. For cost-effective testing of CaReSS, CRN has employed EvoMaster, an AI-based REST API testing tool combined with an integrated classical machine learning model. Within this context, we propose Qlinical to investigate the feasibility of using, inside EvoMaster, a Quantum Neural Network (QNN) classifier, i.e., a quantum machine learning model, instead of the existing classical machine learning model. Results indicate that Qlinical can achieve performance comparable to that of EvoClass. We further explore the effects of various QNN configurations on performance and offer recommendations for optimal QNN settings for future QNN developers.

cs.SE↗

arXiv2024

Quantum Program Testing Through Commuting Pauli Strings on IBM's Quantum Computers

The most promising applications of quantum computing are centered around solving search and optimization tasks, particularly in fields such as physics simulations, quantum chemistry, and finance. However, the current quantum software testing methods face practical limitations when applied in industrial contexts: (i) they do not apply to quantum programs most relevant to the industry, (ii) they require a full program specification, which is usually not available for these programs, and (iii) they are incompatible with error mitigation methods currently adopted by main industry actors like IBM. To address these challenges, we present QOPS, a novel quantum software testing approach. QOPS introduces a new definition of test cases based on Pauli strings to improve compatibility with different quantum programs. QOPS also introduces a new test oracle that can be directly integrated with industrial APIs such as IBM's Estimator API and can utilize error mitigation methods for testing on real noisy quantum computers. We also leverage the commuting property of Pauli strings to relax the requirement of having complete program specifications, making QOPS practical for testing complex quantum programs in industrial settings. We empirically evaluate QOPS on 194,982 real quantum programs, demonstrating effective performance in test assessment compared to the state-of-the-art with a perfect F1-score, precision, and recall. Furthermore, we validate the industrial applicability of QOPS by assessing its performance on IBM's three real quantum computers, incorporating both industrial and open-source error mitigation methods.

cs.SE↗

arXiv2024

A Machine Learning-Based Error Mitigation Approach For Reliable Software Development On IBM'S Quantum Computers

Quantum computers have the potential to outperform classical computers for some complex computational problems. However, current quantum computers (e.g., from IBM and Google) have inherent noise that results in errors in the outputs of quantum software executing on the quantum computers, affecting the reliability of quantum software development. The industry is increasingly interested in machine learning (ML)--based error mitigation techniques, given their scalability and practicality. However, existing ML-based techniques have limitations, such as only targeting specific noise types or specific quantum circuits. This paper proposes a practical ML-based approach, called Q-LEAR, with a novel feature set, to mitigate noise errors in quantum software outputs. We evaluated Q-LEAR on eight quantum computers and their corresponding noisy simulators, all from IBM, and compared Q-LEAR with a state-of-the-art ML-based approach taken as baseline. Results show that, compared to the baseline, Q-LEAR achieved a 25% average improvement in error mitigation on both real quantum computers and simulators. We also discuss the implications and practicality of Q-LEAR, which, we believe, is valuable for practitioners.

cs.SE↗

arXiv2024

Application of Quantum Extreme Learning Machines for QoS Prediction of Elevators' Software in an Industrial Context

Quantum Extreme Learning Machine (QELM) is an emerging technique that utilizes quantum dynamics and an easy-training strategy to solve problems such as classification and regression efficiently. Although QELM has many potential benefits, its real-world applications remain limited. To this end, we present QELM's industrial application in the context of elevators, by proposing an approach called QUELL. In QUELL, we use QELM for the waiting time prediction related to the scheduling software of elevators, with applications for software regression testing, elevator digital twins, and real-time performance prediction. The scheduling software has been implemented by our industrial partner Orona, a globally recognized leader in elevator technology. We demonstrate that QUELL can efficiently predict waiting times, with prediction quality significantly better than that of classical ML models employed in a state-of-the-practice approach. Moreover, we show that the prediction quality of QUELL does not degrade when using fewer features. Based on our industrial application, we further provide insights into using QELM in other applications in Orona, and discuss how QELM could be applied to other industrial applications.

cs.SE↗

arXiv2024

Mitigating Noise in Quantum Software Testing Using Machine Learning

Quantum Computing (QC) promises computational speedup over classic computing for solving complex problems. However, noise exists in current and near-term quantum computers. Quantum software testing (for gaining confidence in quantum software's correctness) is inevitably impacted by noise, to the extent that it is impossible to know if a test case failed due to noise or real faults. Existing testing techniques test quantum programs without considering noise, i.e., by executing tests on ideal quantum computer simulators. Consequently, they are not directly applicable to testing quantum software on real quantum computers or noisy simulators. To this end, we propose a noise-aware approach (named QOIN) to alleviate the noise effect on test results of quantum programs. QOIN employs machine learning techniques (e.g., transfer learning) to learn the noise effect of a quantum computer and filter it from a quantum program's outputs. Such filtered outputs are then used as the input to perform test case assessments (determining the passing or failing of a test case execution against a test oracle). We evaluated QOIN on IBM's 23 noise models, Google's two available noise models, and Rigetti's Quantum Virtual Machine (QVM), with nine real-world quantum programs and 1000 artificial quantum programs. Results show that QOIN can reduce the noise effect by more than $80\%$ on the majority of noise models. For quantum software testing, we used an existing test oracle and showed that QOIN attained scores of $99\%$, $75\%$, and $86\%$ for precision, recall, and F1-score, respectively, for the test oracle across six real-world programs. For artificial programs, QOIN achieved scores of $93\%$, $79\%$, and $86\%$ for precision, recall, and F1-score. This highlights QOIN's effectiveness in learning noise patterns for noise-aware quantum software testing.

cs.SE↗

arXiv2023

Safety Assessment of Vehicle Characteristics Variations in Autonomous Driving Systems

Autonomous driving systems (ADSs) must be sufficiently tested to ensure their safety. Though various ADS testing methods have shown promising results, they are limited to a fixed set of vehicle characteristics settings (VCSs). The impact of variations in vehicle characteristics (e.g., mass, tire friction) on the safety of ADSs has not been sufficiently and systematically studied.Such variations are often due to wear and tear, production errors, etc., which may lead to unexpected driving behaviours of ADSs. To this end, in this paper, we propose a method, named SAFEVAR, to systematically find minimum variations to the original vehicle characteristics setting, which affect the safety of the ADS deployed on the vehicle. To evaluate the effectiveness of SAFEVAR, we employed two ADSs and conducted experiments with two driving simulators. Results show that SAFEVAR, equipped with NSGA-II, generates more critical VCSs that put the vehicle into unsafe situations, as compared with two baseline algorithms: Random Search and a mutation-based fuzzer. We also identified critical vehicle characteristics and reported to which extent varying their settings put the ADS vehicles in unsafe situations.

cs.SE↗

arXiv2023

Towards Quantum Software Requirements Engineering

Quantum software engineering (QSE) is receiving increasing attention, as evidenced by increasing publications on topics, e.g., quantum software modeling, testing, and debugging. However, in the literature, quantum software requirements engineering (QSRE) is still a software engineering area that is relatively less investigated. To this end, in this paper, we provide an initial set of thoughts about how requirements engineering for quantum software might differ from that for classical software after making an effort to map classical requirements classifications (e.g., functional and extra-functional requirements) into the context of quantum software. Moreover, we provide discussions on various aspects of QSRE that deserve attention from the quantum software engineering community.

cs.SE↗

arXiv2023

QuCAT: A Combinatorial Testing Tool for Quantum Software

With the increased developments in quantum computing, the availability of systematic and automatic testing approaches for quantum programs is becoming increasingly essential. To this end, we present the quantum software testing tool QuCAT for combinatorial testing of quantum programs. QuCAT provides two functionalities of use. With the first functionality, the tool generates a test suite of a given strength (e.g., pair-wise). With the second functionality, it generates test suites with increasing strength until a failure is triggered or a maximum strength is reached. QuCAT uses two test oracles to check the correctness of test outputs. We assess the cost and effectiveness of QuCAT with 3 faulty versions of 5 quantum programs. Results show that combinatorial test suites with a low strength can find faults with limited cost, while a higher strength performs better to trigger some difficult faults with relatively higher cost. Repository: https://github.com/Simula-COMPLEX/qucat-tool Video: https://youtu.be/UsqgOudKLio

cs.SE↗

arXiv2023

Test Case Minimization with Quantum Annealers

Quantum annealers are specialized quantum computers for solving combinatorial optimization problems using special characteristics of quantum computing (QC), such as superposition, entanglement, and quantum tunneling. Theoretically, quantum annealers can outperform classical computers. However, the currently available quantum annealers are small-scale, i.e., they have limited quantum bits (qubits); hence, they currently cannot demonstrate the quantum advantage. Nonetheless, research is warranted to develop novel mechanisms to formulate combinatorial optimization problems for quantum annealing (QA). However, solving combinatorial problems with QA in software engineering remains unexplored. Toward this end, we propose BootQA, the very first effort at solving the test case minimization (TCM) problem with QA. In BootQA, we provide a novel formulation of TCM for QA, followed by devising a mechanism to incorporate bootstrap sampling to QA to optimize the use of qubits. We also implemented our TCM formulation in three other optimization processes: classical simulated annealing (SA), QA without problem decomposition, and QA with an existing D-Wave problem decomposition strategy, and conducted an empirical evaluation with three real-world TCM datasets. Results show that BootQA outperforms QA without problem decomposition and QA with the existing decomposition strategy in terms of effectiveness. Moreover, BootQA's effectiveness is similar to SA. Finally, BootQA has higher efficiency in terms of time when solving large TCM problems than the other three optimization processes.

cs.SE↗

arXiv2023

Online Causation Monitoring of Signal Temporal Logic

Online monitoring is an effective validation approach for hybrid systems, that, at runtime, checks whether the (partial) signals of a system satisfy a specification in, e.g., Signal Temporal Logic (STL). The classic STL monitoring is performed by computing a robustness interval that specifies, at each instant, how far the monitored signals are from violating and satisfying the specification. However, since a robustness interval monotonically shrinks during monitoring, classic online monitors may fail in reporting new violations or in precisely describing the system evolution at the current instant. In this paper, we tackle these issues by considering the causation of violation or satisfaction, instead of directly using the robustness. We first introduce a Boolean causation monitor that decides whether each instant is relevant to the violation or satisfaction of the specification. We then extend this monitor to a quantitative causation monitor that tells how far an instant is from being relevant to the violation or satisfaction. We further show that classic monitors can be derived from our proposed ones. Experimental results show that the two proposed monitors are able to provide more detailed information about system evolution, without requiring a significantly higher monitoring cost.

eess.SY↗

arXiv2023

Using a Variational Autoencoder to Learn Valid Search Spaces of Safely Monitored Autonomous Robots for Last-Mile Delivery

The use of autonomous robots for delivery of goods to customers is an exciting new way to provide a reliable and sustainable service. However, in the real world, autonomous robots still require human supervision for safety reasons. We tackle the realworld problem of optimizing autonomous robot timings to maximize deliveries, while ensuring that there are never too many robots running simultaneously so that they can be monitored safely. We assess the use of a recent hybrid machine-learningoptimization approach COIL (constrained optimization in learned latent space) and compare it with a baseline genetic algorithm for the purposes of exploring variations of this problem. We also investigate new methods for improving the speed and efficiency of COIL. We show that only COIL can find valid solutions where appropriate numbers of robots run simultaneously for all problem variations tested. We also show that when COIL has learned its latent representation, it can optimize 10% faster than the GA, making it a good choice for daily re-optimization of robots where delivery requests for each day are allocated to robots while maintaining safe numbers of robots running at once.

cs.NE↗

arXiv2022

Does Road Diversity Really Matter in Testing Automated Driving Systems? -- A Registered Report

Background/Context. The use of automated driving systems (ADSs) in the real world requires rigorous testing to ensure safety. To increase trust, ADSs should be tested on a large set of diverse road scenarios. Literature suggests that if a vehicle is driven along a set of geometrically diverse roads-measured using various diversity measures (DMs)-it will react in a wide range of behaviours, thereby increasing the chances of observing failures (if any), or strengthening the confidence in its safety, if no failures are observed. To the best of our knowledge, however, this assumption has never been tested before, nor have road DMs been assessed for their properties. Objective/Aim. Our goal is to perform an exploratory study on 47 currently used and new, potentially promising road DMs. Specifically, our research questions look into the road DMs themselves, to analyse their properties (e.g. monotonicity, computation efficiency), and to test correlation between DMs. Furthermore, we look at the use of road DMs to investigate whether the assumption that diverse test suites of roads expose diverse driving behaviour holds. Method. Our empirical analysis relies on a state-of-the-art, open-source ADSs testing infrastructure and uses a data set containing over 97,000 individual road geometries and matching simulation data that were collected using two driving agents. By sampling random test suites of various sizes and measuring their roads' geometric diversity, we study road DMs properties, the correlation between road DMs, and the correlation between road DMs and the observed behaviour.

cs.SE↗

arXiv2022

QuSBT: Search-Based Testing of Quantum Programs

Generating a test suite for a quantum program such that it has the maximum number of failing tests is an optimization problem. For such optimization, search-based testing has shown promising results in the context of classical programs. To this end, we present a test generation tool for quantum programs based on a genetic algorithm, called QuSBT (Search-based Testing of Quantum Programs). QuSBT automates the testing of quantum programs, with the aim of finding a test suite having the maximum number of failing test cases. QuSBT utilizes IBM's Qiskit as the simulation framework for quantum programs. We present the tool architecture in addition to the implemented methodology (i.e., the encoding of the search individual, the definition of the fitness function expressing the search problem, and the test assessment w.r.t. two types of failures). Finally, we report results of the experiments in which we tested a set of faulty quantum programs with QuSBT to assess its effectiveness. Repository (code and experimental results): https://github.com/Simula-COMPLEX/qusbt-tool Video: https://youtu.be/3apRCtluAn4

cs.SE↗