Random Testing of Model Checkers for Timed Automata with Automated Oracle Generation

Abstract

A key challenge in formal verification, particularly in Model Checking, is ensuring the correctness of the verification tools. Erroneous results on complex models can be difficult to detect, yet a high level of confidence in the outcome is expected. Indeed, these tools are frequently novel and may not have been thoroughly tested. When standard benchmarks may be insufficient or unavailable, random test case generation offers a promising approach. To scale up, random testing requires comparing actual versus expected results, i.e., solving the oracle problem. To address this challenge, this work introduces a novel theoretical framework based on a modular variant of Timed Automata (TA), called Tiled Timed Automata (TTA), for testing model checkers operating with variations of TA, by building oracles based on Weighted Automata. The framework is initially applied to verify model checkers solving the emptiness problem for Parametric TA and it is validated, in this specific scenario, by our tool, TABEC, which randomly generates tests predicting their expected outcome through automated oracle generation. Furthermore, the general nature of TTA facilitates the framework adaptation to model checkers solving other decidable problems on TA, as detailed for the minimum-cost reachability problem of Priced TA.