Yaroslav Yeleyko | Searcharxiv

arXiv2013

Killed Markov Decision Processes on Finite Time Interval for Countable Models

We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is reduced to a similar problem for the derived model. We receive an optimality equation and a method for the construction of simple optimal policies. The sufficiency of simple policies for countable models is proven. We show the correctness of the Markovian property. Additionally, a dynamic programming principle is considered.

math.OC↗

arXiv2011

Asymptotic behaviour of the S-stopped branching processes with countable state space

he starting process with countable number of types μ(t) generates a stopped branching process ξ(t). The starting process stops, by falling into the nonempty set S. It is assumed, that the starting process is subcritical, indecomposable and noncyclic. It is proved, that the extinction probability converges to the cyclic function with period 1.

math.ST↗