Searcharxiv⌕ Search

arXiv subjects

Robert Modderman

Publications and source records attributed to Robert Modderman.

3 recordsLinked to original sources

Path Abstraction for Markov Reward Models: An Isabelle Formalisation in a Free Monoid View

The classical definition of discrete-time Markov chains maps pairs of states to transition probabilities. An alternative, proposed in the context of path abstraction in probabilistic counterexample refinement, is to view them as a transition kernel on the free monoid on the state space, mapping paths directly to path probabilities. In this paper, we first extend the free monoid view and path abstraction algorithm to cumulative rewards on Markov reward models. We then use Isabelle/HOL to formalise Markov chains and reward models in this view, exploiting the discrete finite-prefix semantics to work with Isabelle's standard HOL-Analysis library. We mechanise the path abstraction algorithm and its key correctness properties of well-definedness and monotonic absorption, establishing the preservation of reachability probabilities and expected rewards. Proving the convergence of these quantities on relevant path sets is aided by our use of the lightweight free monoid semantics in place of the standard measure-theoretic one used in established Isabelle formalisations of Markov models. We obtain a verified abstract implementation of path abstraction in Isabelle/HOL, providing a foundation for refinement to efficient executable code and integration with probabilistic model checkers.

cs.LO↗

Path Abstraction for Markov Reward Models

Path abstraction originated as a technique for counterexample refinement in probabilistic model checking. Given a discrete-time Markov chain, it summarises the probabilities passing through a subset of the states onto new transitions of a smaller chain. In earlier work, we proved its correctness and that it is monotonically absorbing. In this paper, we extend path abstraction from reachability probabilities on discrete-time Markov chains to expected rewards on Markov reward models. Working in a novel free monoid view of Markov chains throughout, we prove that path abstraction preserves the Markov reward model structure when abstracting over arbitrary sets of states, and that it remains monotonically absorbing. Finally, we give a numerical recipe, accompanied by a reference implementation in PARI/GP, that computes path abstraction by solving linear equation systems. Its correctness rests on the relationship between expected rewards and expected visiting times of transitions.

cs.FL↗

DTMC Model Checking by Path Abstraction Revisited (extended version)

Computing the probability of reaching a set of goal states G in a discrete-time Markov chain (DTMC) is a core task of probabilistic model checking. We can do so by directly computing the probability mass of the set of all finite paths from the initial state to G; however, when refining counterexamples, it is also interesting to compute the probability mass of subsets of paths. This can be achieved by splitting the computation into path abstractions that calculate "local" reachability probabilities as shown by Ábrahám et al. in 2010. In this paper, we complete and extend their work: We prove that splitting the computation into path abstractions indeed yields the same result as the direct approach, and that the splitting does not need to follow the SCC structure. In particular, we prove that path abstraction can be performed along any finite sequence of sets of non-goal states. Our proofs proceed in a novel way by interpreting the DTMC as a structure on the free monoid on its state space, which makes them clean and concise. Additionally, we provide a compact reference implementation of path abstraction in PARI/GP.

cs.FL↗