arXiv subjects

Toshiaki Aoki

Publications and source records attributed to Toshiaki Aoki.

10 recordsLinked to original sources

arXiv2026

Bit-Precise Conformance Testing of Simulink Model Checkers

MATLAB/Simulink provides a practical modeling language and a simulation engine for the development of cyber-physical systems. To ensure the quality of the developed models, there are formal verification tools available, such as Simulink Design Verifier (SLDV) and third-party SMT-based model checkers (SmtMC). However, due to the absence of a semantics of Simulink that covers every element of models and the details of its numerical behavior, the reliability of the model checkers themselves is often doubtful, potentially analyzing models differently from the simulator. This work aims to verify the quality of the Simulink model checkers by addressing the following items. 1) Formalization of the basic block types of Simulink. It involves defining block type feature sets and the bit-precise behavior of the blocks. 2) A method for testing bit-precise conformance relations among the tools for each block type. The pass rate of our test suite measures (i) conformance of model checking results with simulation results by Simulink and (ii) conformance between the results of SmtMC and SLDV. 3) Experiment to perform tests on 10 block types. We confirmed that SmtMC efficiently passed all test cases, while SLDV achieved pass rates of only 94-96% and 80-90% for conformance (i) and (ii), respectively. We analyzed the causes of failed tests, such as errors, corner cases, and timeouts.

cs.SE↗

arXiv2026

Quantifying Competitive Relationships Among Open-Source Software Projects

Throughout the history of software, evolution has occurred in cycles of rise and fall driven by competition, and open-source software (OSS) is no exception. This cycle is accelerating, particularly in rapidly evolving domains such as web development and deep learning. However, the impact of competitive relationships among OSS projects on their survival remains unclear, and there are risks of losing a competitive edge to rivals. To address this, this study proposes a new automated method called ``Mutual Impact Analysis of OSS (MIAO)'' to quantify these competitive relationships. The proposed method employs a structural vector autoregressive model and impulse response functions, normally used in macroeconomic analysis, to analyze the interactions among OSS projects. In an empirical analysis involving mining and analyzing 187 OSS project groups, MIAO identified projects that were forced to cease development owing to competitive influences with up to 78\% accuracy, and the resulting features supported predictive experiments that anticipate cessation one year ahead with up to 74\% accuracy. This suggests that MIAO could be a valuable tool for OSS project maintainers to understand the dynamics of OSS ecosystems and predict the rise and fall of OSS projects.

cs.SE↗

arXiv2025

A Case Study on Model Checking and Runtime Verification for Awkernel

In operating system development, concurrency poses significant challenges. It is difficult for humans to manually review concurrent behaviors or to write test cases covering all possible executions, often resulting in critical bugs. Preemption in schedulers serves as a typical example. This paper proposes a development method for concurrent software, such as schedulers. Our method incorporates model checking as an aid for tracing code, simplifying the analysis of concurrent behavior; we refer to this as model checking-assisted code review. While this approach aids in tracing behaviors, the accuracy of the results is limited because of the semantics gap between the modeling language and the programming language. Therefore, we also introduce runtime verification to address this limitation in model checking-assisted code review. We applied our approach to a real-world operating system, Awkernel, as a case study. This new operating system, currently under development for autonomous driving, is designed for preemptive task execution using asynchronous functions in Rust. After implementing our method, we identified several bugs that are difficult to detect through manual reviews or simple tests.

cs.SE↗

arXiv2025

Modeling Language for Scenario Development of Autonomous Driving Systems

Autonomous driving systems are typically verified based on scenarios. To represent the positions and movements of cars in these scenarios, diagrams that utilize icons are typically employed. However, the interpretation of such diagrams is typically ambiguous, which can lead to misunderstandings among users, making them unsuitable for the development of high-reliability systems. To address this issue, this study introduces a notation called the car position diagram (CPD). The CPD allows for the concise representation of numerous scenarios and is particularly suitable for scenario analysis and design. In addition, we propose a method for converting CPD-based models into propositional logic formulas and enumerating all scenarios using a SAT solver. A tool for scenario enumeration is implemented, and experiments are conducted on both typical car behaviors and international standards. The results demonstrate that the CPD enables the concise description of numerous scenarios, thereby confirming the effectiveness of our scenario analysis method.

cs.SE↗

arXiv2024

Generating Critical Scenarios for Testing Automated Driving Systems

Autonomous vehicles (AVs) have demonstrated significant potential in revolutionizing transportation, yet ensuring their safety and reliability remains a critical challenge, especially when exposed to dynamic and unpredictable environments. Real-world testing of an Autonomous Driving System (ADS) is both expensive and risky, making simulation-based testing a preferred approach. In this paper, we propose AVASTRA, a Reinforcement Learning (RL)-based approach to generate realistic critical scenarios for testing ADSs in simulation environments. To capture the complexity of driving scenarios, AVASTRA comprehensively represents the environment by both the internal states of an ADS under-test (e.g., the status of the ADS's core components, speed, or acceleration) and the external states of the surrounding factors in the simulation environment (e.g., weather, traffic flow, or road condition). AVASTRA trains the RL agent to effectively configure the simulation environment that places the AV in dangerous situations and potentially leads it to collisions. We introduce a diverse set of actions that allows the RL agent to systematically configure both environmental conditions and traffic participants. Additionally, based on established safety requirements, we enforce heuristic constraints to ensure the realism and relevance of the generated test scenarios. AVASTRA is evaluated on two popular simulation maps with four different road configurations. Our results show AVASTRA's ability to outperform the state-of-the-art approach by generating 30% to 115% more collision scenarios. Compared to the baseline based on Random Search, AVASTRA achieves up to 275% better performance. These results highlight the effectiveness of AVASTRA in enhancing the safety testing of AVs through realistic comprehensive critical scenario generation.

cs.RO↗

arXiv2023

Attack Tree Analysis for Adversarial Evasion Attacks

Recently, the evolution of deep learning has promoted the application of machine learning (ML) to various systems. However, there are ML systems, such as autonomous vehicles, that cause critical damage when they misclassify. Conversely, there are ML-specific attacks called adversarial attacks based on the characteristics of ML systems. For example, one type of adversarial attack is an evasion attack, which uses minute perturbations called "adversarial examples" to intentionally misclassify classifiers. Therefore, it is necessary to analyze the risk of ML-specific attacks in introducing ML base systems. In this study, we propose a quantitative evaluation method for analyzing the risk of evasion attacks using attack trees. The proposed method consists of the extension of the conventional attack tree to analyze evasion attacks and the systematic construction method of the extension. In the extension of the conventional attack tree, we introduce ML and conventional attack nodes to represent various characteristics of evasion attacks. In the systematic construction process, we propose a procedure to construct the attack tree. The procedure consists of three steps: (1) organizing information about attack methods in the literature to a matrix, (2) identifying evasion attack scenarios from methods in the matrix, and (3) constructing the attack tree from the identified scenarios using a pattern. Finally, we conducted experiments on three ML image recognition systems to demonstrate the versatility and effectiveness of our proposed method.

cs.CR↗

arXiv2023

Model-Checking in the Loop Model-Based Testing for Automotive Operating Systems

While vehicles have primarily been controlled through mechanical means in years past, an increasing number of embedded control systems are being installed and used, keeping pace with advances in electronic control technology and performance. Automotive systems consist of multiple components developed by a range of vendors. To accelerate developments in embedded control systems, industrial standards such as AUTOSAR are being defined for automotive systems, including the design of operating system and middleware technologies. Crucial to ensuring the safety of automotive systems, the operating system is foundational software on which many automotive applications are executed. In this paper, we propose an integrated model-based method for verifying automotive operating systems; our method is called Model-Checking in the Loop Model-Based Testing (MCIL-MBT). In MCIL-MBT, we create a model that formalizes specifications of automotive operating systems and verifies the specifications via model-checking. Next, we conduct model-based testing with the verified model to ensure that a specific operating system implementation conforms to the model. These verification and testing stages are iterated over until no flaws are detected. Our method has already been introduced to an automotive system supplier and an operating system vendor. Through our approach, we successfully identified flaws that were not detected by conventional review and testing methods.

cs.SE↗

arXiv2022

SMT-Based Model Checking of Industrial Simulink Models

The development of embedded systems requires formal analysis of models such as those described with MATLAB/Simulink. However, the increasing complexity of industrial models makes analysis difficult. This paper proposes a model checking method for Simulink models using SMT solvers. The proposed method aims at (1) automated, efficient and comprehensible verification of complex models, (2) numerically accurate analysis of models, and (3) demonstrating the analysis of Simulink models using an SMT solver (we use Z3). It first encodes a target model into a predicate logic formula in the domain of mathematical arithmetic and bit vectors. We explore how to encode various Simulink blocks exactly. Then, the method verifies a given invariance property using the k-induction-based algorithm that extracts a subsystem involving the target block and unrolls the execution paths incrementally. In the experiment, we applied the proposed method and other tools to a set of models and properties. Our method successfully verified most of the properties including those unverified with other tools.

cs.LO↗

arXiv2021

Compositional Test Generation of Industrial Synchronous Systems

Synchronous systems provide a basic model of embedded systems and industrial systems are modeled as Simulink diagrams and/or Lustre programs. Although the test generation problem is critical in the development of safe systems, it often fails because of the spatial and temporal complexity of the system descriptions. This paper presents a compositional test generation method to address the complexity issue. We regard a test case as a counterexample in safety verification, and represent a test generation process as a deductive proof tree built with dedicated inference rules; we conduct both spatial- and temporal-compositional reasoning along with a modular system structure. A proof tree is generated using our semi-automated scheme involving manual effort on contract generation and automatic processes for counterexample search with SMT solvers. As case studies, the proposed method is applied to four industrial examples involving such features as enabled/triggered subsystems, multiple execution rates, filter components, and nested counters. In the experiments, we successfully generated test cases for target systems that were difficult to deal with using the existing tools.

cs.SE↗

arXiv2021

Approximate Translation from Floating-Point to Real-Interval Arithmetic

Floating-point arithmetic (FPA) is a mechanical representation of real arithmetic (RA), where each operation is replaced with a rounded counterpart. Various numerical properties can be verified by using SMT solvers that support the logic of FPA. However, the scalability of the solving process remains limited when compared to RA. In this paper, we present a decision procedure for FPA that takes advantage of the efficiency of RA solving. The proposed method abstracts FP numbers as rational intervals and FPA expressions as interval arithmetic (IA) expressions; then, we solve IA formulas to check the satisfiability of an FPA formula using an off-the-shelf RA solver (we use CVC4 and Z3). In exchange for the efficiency gained by abstraction, the solving process becomes quasi-complete; we allow to output unknown when the satisfiability is affected by possible numerical errors. Furthermore, our IA is meticulously formalized to handle the special value NaN. We implemented the proposed method and compared it to four existing SMT solvers in the experiments. As a result, we confirmed that our solver was efficient for instances where rounding modes were parameterized.

cs.LO↗