arXiv subjects

Tomás Prieto-Rumeau

Publications and source records attributed to Tomás Prieto-Rumeau.

4 recordsLinked to original sources

arXiv2024

A note on weak compactness of occupation measures for an absorbing Markov decision process

We consider an absorbing Markov decision process with Borel state and action spaces. We study conditions under which the MDP is uniformly absorbing and the set of occupation measures of the MDP is compact in the usual weak topology. These include suitable continuity requirements on the transition kernel and conditions on the dynamics of the system at the boundary of the absorbing set. We generalize previously known results and give an answer to some conjectures that have been mentioned in the related literature.

math.PR↗

arXiv2023

Absorbing Markov Decision Processes

In this paper, we study discrete-time absorbing Markov Decision Processes (MDP) with measurable state space and Borel action space with a given initial distribution. For such models, solutions to the characteristic equation that are not occupation measures may exist. Several necessary and sufficient conditions are provided to guarantee that any solution to the characteristic equation is an occupation measure. Under the so-called continuity-compactness conditions, it is shown that the set of occupation measures is compact in the weak-strong topology if and only if the model is uniformly absorbing. Finally, it is shown that the occupation measures are characterized by the characteristic equation and an additional condition. Several examples are provided to illustrate our results.

math.OC↗

arXiv2023

Nash equilibria for total expected reward absorbing Markov games: the constrained and unconstrained cases

We consider a nonzero-sum N-player Markov game on an abstract measurable state space with compact metric action spaces. The payoff functions are bounded Carathéodory functions and the transitions of the system are assumed to have a density function satisfying some continuity conditions. The optimality criterion of the players is given by a total expected payoff on an infinite discrete-time horizon. Under the condition that the game model is absorbing, we establish the existence of Markov strategies that are a noncooperative equilibrium in the family of all history-dependent strategies of the players for both the constrained and the unconstrained problems, We obtain, as a particular case of results, the existence of Nash equilibria for discounted constrained and unconstrained game models.

math.OC↗

arXiv2021

Stationary Markov Nash equilibria for nonzero-sum constrained ARAT Markov games

We consider a nonzero-sum Markov game on an abstract measurable state space with compact metric action spaces. The goal of each player is to maximize his respective discounted payoff function under the condition that some constraints on a discounted payoff are satisfied. We are interested in the existence of a Nash or noncooperative equilibrium. Under suitable conditions, which include absolute continuity of the transitions with respect to some reference probability measure, additivity of the payoffs and the transition probabilities (ARAT condition), and continuity in action of the payoff functions and the density function of the transitions of the system, we establish the existence of a constrained stationary Markov Nash equilibrium, that is, the existence of stationary Markov strategies for each of the players yielding an optimal profile within the class of all history-dependent profiles.

math.OC↗