arXiv subjects

Thomas Krak

Publications and source records attributed to Thomas Krak.

12 recordsLinked to original sources

arXiv2026

Lower and Upper Expected Hitting Times for Weighted Imprecise Markov Chains

In this paper, we extend hitting times for imprecise Markov chains to the framework of weighted imprecise Markov chains (WIMCs), in which each transition is associated with a strictly positive weight encoded by a matrix $W$. Given a convex set $\mathcal{T}$ of admissible transition matrices, we define lower and upper expected hitting times for WIMCs as the infimum and supremum of the (weighted) expected hitting times over $\mathcal{T}$, and we characterise these quantities as the unique solutions of nonlinear fixed-point equations. We show that any weighted hitting time problem can be transformed into an unweighted hitting time problem on an augmented state space, enabling the reuse of existing IMC theory and algorithms. In particular, we are able to adapt known iterative methods for the numerical computation of expected hitting times for WIMCs.

math.PR↗

arXiv2026

Relation between Hitting Times and Probabilities for Imprecise Markov Chains

In the present paper, we investigate the relationship between hitting times and hitting probabilities in discrete-time imprecise Markov chains (IMCs). We define lower and upper hitting times and probabilities for IMCs whose set of transition matrices $\T$ is compact, convex, and has separately specified rows. Building on reachability-based partitions of the state space, we prove two key implications: (i) finiteness of the upper expected hitting time entails the lower hitting probability equals one, and (ii) finiteness of the lower expected hitting time entails the upper hitting probability equals one. We further show an equivalence: the upper expected hitting time is finite if and only if the lower hitting probability is one. Finally, by presenting a counterexample, we show that the converse of the second implication can fail.

math.PR↗

arXiv2025

Computing Lower and Upper Hitting Probabilities for Imprecise Markov Chains

We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting probabilities are the minimal nonnegative solution of a linear system and admit a closed-form expression. We investigate the notion of reachability in the imprecise setting. The literature review highlights several different definitions of lower reachability; thus, we explore the relations among them and present examples to clarify their logical implications. Using this revised definition of reachability for imprecise Markov chains, we partition the state space into classes of states whose hitting probabilities are trivially zero or one, and those which require further computation. For these nontrivial states, we show that the lower hitting probability is the unique solution of a nonlinear fixed-point equation, while the same does not hold for upper hitting probabilities. For the practical computation of lower and upper hitting probabilities, we propose iterative algorithms that alternate between solving a linear system and choosing an extreme point from the set of transition matrices. Numerical experiments demonstrate that, in practice, these algorithms converge in substantially fewer iterations than the theoretically established worst-case bound.

math.PR↗

arXiv2025

Towards conservative inference in credal networks using belief functions: the case of credal chains

This paper explores belief inference in credal networks using Dempster-Shafer theory. By building on previous work, we propose a novel framework for propagating uncertainty through a subclass of credal networks, namely chains. The proposed approach efficiently yields conservative intervals through belief and plausibility functions, combining computational speed with robust uncertainty representation. Key contributions include formalizing belief-based inference methods and comparing belief-based inference against classical sensitivity analysis. Numerical results highlight the advantages and limitations of applying belief inference within this framework, providing insights into its practical utility for chains and for credal networks in general.

cs.AI↗

arXiv2025

Upper Expected Meeting Times for Interdependent Stochastic Agents

We analyse the problem of meeting times for interdependent stochastic agents: random walkers whose behaviour is stochastic but controlled by their selections from some set of allowed actions, and the inference problem of when these agents will be in the same state for the first time. We consider the case where we are epistemically uncertain about the selected actions of these agents, and show how their behaviour can be modelled using imprecise Markov chains. This allows us to use results and algorithms from the literature, to exactly compute bounds on their meeting time, which are tight with respect to our epistemic uncertainty models. We focus on the two-agent case, but discuss how it can be naturally extended to an arbitrary number of agents, and how the corresponding combinatorial explosion can be partly mitigated by exploiting symmetries inherent in the problem.

math.PR↗

arXiv2022

Hitting Times for Continuous-Time Imprecise-Markov Chains

We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work essentially constitute sets of stochastic processes. Their inferences are tight lower- and upper bounds with respect to variation within these sets. We consider three distinct types of these models, corresponding to different levels of generality and structural independence assumptions on the constituent processes. Our main results are twofold; first, we demonstrate that the hitting times for all three types are equivalent. Moreover, we show that these inferences are described by a straightforward generalization of a well-known linear system of equations that characterizes expected hitting times for traditional time-homogeneous continuous-time Markov chains.

math.PR↗

arXiv2021

Computing Expected Hitting Times for Imprecise Markov Chains

We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially specified. We also briefly sketch how this method can be modified to solve a conjugate system of equations that gives rise to the corresponding upper bound. We prove the correctness of our method, and show that it converges to the correct solution in a finite number of steps under mild conditions on the system. We compare the runtime complexity of our method to a previously published method from the literature, and identify conditions under which our novel method is more efficient.

math.PR↗

arXiv2020

Hitting Times and Probabilities for Imprecise Markov Chains

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains.

math.PR↗

arXiv2019

A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains

We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other types of model uncertainty. The class of inferences that we consider contains, as special cases, tight lower and upper bounds on expected hitting times, on hitting probabilities and on expectations of functions that are a sum or product of simpler ones. Our algorithm exploits the specific structure that is inherent in all these inferences: they admit a general recursive decomposition. This allows us to achieve a computational complexity that scales linearly in the number of time points on which the inference depends, instead of the exponential scaling that is typical for a naive approach.

math.PR↗

arXiv2018

An Imprecise Probabilistic Estimator for the Transition Rate Matrix of a Continuous-Time Markov Chain

We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior distributions on the unknown transition rate matrix. The resulting estimator is a set of transition rate matrices that, for reasons of conjugacy, is easy to find. To determine the hyperparameters for our set of priors, we reconsider the problem in discrete time, where we can use the well-known Imprecise Dirichlet Model. In particular, we show how the limit of the resulting discrete-time estimators is a continuous-time estimator. It corresponds to a specific choice of hyperparameters and has an exceptionally simple closed-form expression.

stat.ML↗

arXiv2017

Imprecise Continuous-Time Markov Chains

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models computationally tractable, they rely on a number of assumptions that may not be realistic for the domain of application; in particular, the ability to provide exact numerical parameter assessments, and the applicability of time-homogeneity and the eponymous Markov property. In this work, we extend these models to imprecise continuous-time Markov chains (ICTMC's), which are a robust generalisation that relaxes these assumptions while remaining computationally tractable. More technically, an ICTMC is a set of "precise" continuous-time finite-state stochastic processes, and rather than computing expected values of functions, we seek to compute lower expectations, which are tight lower bounds on the expectations that correspond to such a set of "precise" models. Note that, in contrast to e.g. Bayesian methods, all the elements of such a set are treated on equal grounds; we do not consider a distribution over this set. The first part of this paper develops a formalism for describing continuous-time finite-state stochastic processes that does not require the aforementioned simplifying assumptions. Next, this formalism is used to characterise ICTMC's and to investigate their properties. The concept of lower expectation is then given an alternative operator-theoretic characterisation, by means of a lower transition operator, and the properties of this operator are investigated as well. Finally, we use this lower transition operator to derive tractable algorithms (with polynomial runtime complexity w.r.t. the maximum numerical error) for computing the lower expectation of functions that depend on the state at any finite number of time points.

math.PR↗

arXiv2017

Efficient Computation of Updated Lower Expectations for Imprecise Continuous-Time Hidden Markov Chains

We consider the problem of performing inference with imprecise continuous-time hidden Markov chains, that is, imprecise continuous-time Markov chains that are augmented with random output variables whose distribution depends on the hidden state of the chain. The prefix `imprecise' refers to the fact that we do not consider a classical continuous-time Markov chain, but replace it with a robust extension that allows us to represent various types of model uncertainty, using the theory of imprecise probabilities. The inference problem amounts to computing lower expectations of functions on the state-space of the chain, given observations of the output variables. We develop and investigate this problem with very few assumptions on the output variables; in particular, they can be chosen to be either discrete or continuous random variables. Our main result is a polynomial runtime algorithm to compute the lower expectation of functions on the state-space at any given time-point, given a collection of observations of the output variables.

math.PR↗