Finding the Smallest Possible Exact Aggregation of a Markov Chain

Abstract

Markov chains are an important tool for modelling and evaluating systems in computer science, economics, biology and numerous other fields. Thus, approximating Markov chains is a useful tool for decreasing the computational effort needed for analysis by reducing the size of the state space. So far, most approximative algorithms focused on finding approximations, which are, again, Markov chains. We remove this restriction and present an algorithm using the Arnoldi iteration. Further, we show that if these approximations are without error, they are of minimal size. Lastly, we implement the algorithm for Markov chains via the Arnoldi iteration. If another type of aggregation, so-called Exlump aggregations, can find an exact aggregation for a Markov chain, they usually yield similar or smaller errors at much lower runtime. On the contrary, approximations computed via the Arnoldi iteration can find usable aggregations where Exlump yields none. Still, runtime is a primary concern, as our criterion for determining the approximate state space size is exceptionally costly. As such, further work is needed to decrease the runtime of the Arnoldi approach.