Onésimo Hernández-Lerma

arXiv2026

The maximum principle for discrete-time control systems and applications to dynamic games

We study deterministic nonstationary discrete-time optimal control problems in both finite and infinite horizon. With the aid of Gateaux differentials, we prove a discrete-time maximum principle in analogy with the well-known continuous-time maximum principle. We show that this maximum principle, together with a transversality condition, is a necessary condition for optimality; we also show that it is sufficient under additional hypotheses. We use Gateaux differentials as a natural setting to derive first-order conditions. Additionally, we use the discrete-time maximum principle to derive the discrete-time Euler equation and to characterize Nash equilibria for discrete-time dynamic games.

math.OC↗

arXiv2008

Existence and regularity of a nonhomogeneous transition matrix under measurability conditions

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t) and conservative matrix Q(t)=[q_{ij}(t)] of nonhomogeneous transition rates q_{ij}(t), and use it to construct the transition probability matrix. Here we obtain the same result except that the q_{ij}(t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, which motivated this work in the first place.

math.PR↗