arXiv subjects

Antoine Hochart

Publications and source records attributed to Antoine Hochart.

10 recordsLinked to original sources

arXiv2020

Unique ergodicity of deterministic zero-sum differential games

We study the ergodicity of deterministic two-person zero-sum differential games. This property is defined by the uniform convergence to a constant of either the infinite-horizon discounted value as the discount factor tends to zero, or equivalently, the averaged finite-horizon value as the time goes to infinity. We provide necessary and sufficient conditions for the unique ergodicity of a game. This notion extends the classical one for dynamical systems, namely when ergodicity holds with any (suitable) perturbation of the running payoff function. Our main condition is symmetric between the two players and involve dominions, i.e., subsets of states that one player can make approximately invariant.

math.OC↗

arXiv2018

A game theory approach to the existence and uniqueness of nonlinear Perron-Frobenius eigenvectors

We establish a generalized Perron-Frobenius theorem, based on a combinatorial criterion which entails the existence of an eigenvector for any nonlinear order-preserving and positively homogeneous map $f$ acting on the open orthant $\mathbb{R}_{\scriptscriptstyle >0}^n$. This criterion involves dominions, i.e., sets of states that can be made invariant by one player in a two-person game that only depends on the behavior of $f$ "at infinity". In this way, we characterize the situation in which for all $α, β> 0$, the "slice space" $\mathcal{S}_α^β:= \{ x \in \mathbb{R}_{\scriptscriptstyle >0}^n \mid αx \leq f(x) \leq βx \}$ is bounded in Hilbert's projective metric, or, equivalently, for all uniform perturbations $g$ of $f$, all the orbits of $g$ are bounded in Hilbert's projective metric. This solves a problem raised by Gaubert and Gunawardena (Trans. AMS, 2004). We also show that the uniqueness of an eigenvector is characterized by a dominion condition, involving a different game depending now on the local behavior of $f$ near an eigenvector. We show that the dominion conditions can be verified by directed hypergraph methods. We finally illustrate these results by considering specific classes of nonlinear maps, including Shapley operators, generalized means and nonnegative tensors.

math.OC↗

arXiv2018

An accretive operator approach to ergodic zero-sum stochastic games

We study some ergodicity property of zero-sum stochastic games with a finite state space and possibly unbounded payoffs. We formulate this property in operator-theoretical terms, involving the solvability of an optimality equation for the Shapley operators (i.e., the dynamic programming operators) of a family of perturbed games. The solvability of this equation entails the existence of the uniform value, and its solutions yield uniform optimal stationary strategies. We first provide an analytical characterization of this ergodicity property, and address the generic uniqueness, up to an additive constant, of the solutions of the optimality equation. Our analysis relies on the theory of accretive mappings, which we apply to maps of the form $Id - T$ where $T$ is nonexpansive. Then, we use the results of a companion work to characterize the ergodicity of stochastic games by a geometrical condition imposed on the transition probabilities. This condition generalizes classical notion of ergodicity for finite Markov chains and Markov decision processes.

math.OC↗

arXiv2018

Qualification Conditions in Semi-algebraic Programming

For an arbitrary finite family of semi-algebraic/definable functions, we consider the corresponding inequality constraint set and we study qualification conditions for perturbations of this set. In particular we prove that all positive diagonal perturbations, save perhaps a finite number of them, ensure that any point within the feasible set satisfies Mangasarian-Fromovitz constraint qualification. Using the Milnor-Thom theorem, we provide a bound for the number of singular perturbations when the constraints are polynomial functions. Examples show that the order of magnitude of our exponential bound is relevant. Our perturbation approach provides a simple protocol to build sequences of "regular" problems approximating an arbitrary semi-algebraic/definable problem. Applications to sequential quadratic programming methods and sum of squares relaxation are provided.

math.OC↗

arXiv2017

Minimax representation of nonexpansive functions and application to zero-sum recursive games

We show that a real-valued function on a topological vector space is positively homogeneous of degree one and nonexpansive with respect to a weak Minkowski norm if and only if it can be written as a minimax of linear forms that are nonexpansive with respect to the same norm. We derive a representation of monotone, additively and positively homogeneous functions on $L^\infty$ spaces and on $\mathbb{R}^n$, which extend results of Kolokoltsov, Rubinov, Singer, and others. We apply this representation to nonconvex risk measures and to zero-sum games. We derive in particular results of representation and polyhedral approximation for the class of Shapley operators arising from games without instantaneous payments (Everett's recursive games).

math.OC↗

arXiv2017

Generic uniqueness of the bias vector of finite stochastic games with perfect information

Mean-payoff zero-sum stochastic games can be studied by means of a nonlinear spectral problem. When the state space is finite, the latter consists in finding an eigenpair $(u,λ)$ solution of $T(u)=λe + u$, where $T:\mathbb{R}^n \to \mathbb{R}^n$ is the Shapley (or dynamic programming) operator, $λ$ is a scalar, $e$ is the unit vector, and $u \in \mathbb{R}^n$. The scalar $λ$ yields the mean payoff per time unit, and the vector $u$, called the bias, allows one to determine optimal stationary strategies. The existence of the eigenpair $(u,λ)$ is generally related to ergodicity conditions. A basic issue is to understand for which classes of games the bias vector is unique (up to an additive constant). In this paper, we consider perfect-information zero-sum stochastic games with finite state and action spaces, thinking of the transition payments as variable parameters, transition probabilities being fixed. We show that the bias vector, thought of as a function of the transition payments, is generically unique (up to an additive constant). The proof uses techniques of max-plus (or tropical) algebra and nonlinear Perron-Frobenius theory. As an application of our results, we obtain a perturbation scheme allowing one to solve degenerate instances of stochastic games by policy iteration.

math.OC↗

arXiv2016

An Accretive Operator Approach to Ergodic Problems for Zero-Sum Games

Mean payoff stochastic games can be studied by means of a nonlinear spectral problem involving the Shapley operator: the ergodic equation. A solution consists in a scalar, called the ergodic constant, and a vector, called bias. The existence of such a pair entails that the mean payoff per time unit is equal to the ergodic constant for any initial state, and the bias gives stationary strategies. By exploiting two fundamental properties of Shapley operators, monotonicity and additive homogeneity, we give a necessary and sufficient condition for the solvability of the ergodic equation for all the Shapley operators obtained by perturbation of the transition payments of a given stochastic game with finite state space. If the latter condition is satisfied, we establish that the bias is unique (up to an additive constant) for a generic perturbation of the transition payments. To show these results, we use the theory of accretive operators, and prove in particular some surjectivity condition.

math.OC↗

arXiv2015

Hypergraph conditions for the solvability of the ergodic equation for zero-sum games

The ergodic equation is a basic tool in the study of mean-payoff stochastic games. Its solvability entails that the mean payoff is independent of the initial state. Moreover, optimal stationary strategies are readily obtained from its solution. In this paper, we give a general sufficient condition for the solvability of the ergodic equation, for a game with finite state space but arbitrary action spaces. This condition involves a pair of directed hypergraphs depending only on the ``growth at infinity'' of the Shapley operator of the game. This refines a recent result of the authors which only applied to games with bounded payments, as well as earlier nonlinear fixed point results for order preserving maps, involving graph conditions.

math.OC↗

arXiv2015

Ergodicity conditions for zero-sum games

A basic question for zero-sum repeated games consists in determining whether the mean payoff per time unit is independent of the initial state. In the special case of "zero-player" games, i.e., of Markov chains equipped with additive functionals, the answer is provided by the mean ergodic theorem. We generalize this result to repeated games. We show that the mean payoff is independent of the initial state for all state-dependent perturbations of the rewards if and only if an ergodicity condition is verified. The latter is characterized by the uniqueness modulo constants of nonlinear harmonic functions (fixed points of the recession function associated to the Shapley operator), or, in the special case of stochastic games with finite action spaces and perfect information, by a reachability condition involving conjugate subsets of states in directed hypergraphs. We show that the ergodicity condition for games only depends on the support of the transition probability, and that it can be checked in polynomial time when the number of states is fixed.

math.OC↗

arXiv2014

Generic uniqueness of the bias vector of mean payoff zero-sum games

Zero-sum mean payoff games can be studied by means of a nonlinear spectral problem. When the state space is finite, the latter consists in finding an eigenpair $(u,λ)$ solution of $T(u)=λ\mathbf{1} + u$ where $T:\mathbb{R}^n \to \mathbb{R}^n$ is the Shapley (dynamic programming) operator, $λ$ is a scalar, $\mathbf{1}$ is the unit vector, and $u \in \mathbb{R}^n$. The scalar $λ$ yields the mean payoff per time unit, and the vector $u$, called the bias, allows one to determine optimal stationary strategies. The existence of the eigenpair $(u,λ)$ is generally related to ergodicity conditions. A basic issue is to understand for which classes of games the bias vector is unique (up to an additive constant). In this paper, we consider perfect information zero-sum stochastic games with finite state and action spaces, thinking of the transition payments as variable parameters, transition probabilities being fixed. We identify structural conditions on the support of the transition probabilities which guarantee that the spectral problem is solvable for all values of the transition payments. Then, we show that the bias vector, thought of as a function of the transition payments, is generically unique (up to an additive constant). The proof uses techniques of max-plus (tropical) algebra and nonlinear Perron-Frobenius theory.

math.OC↗