Searcharxiv⌕ Search

arXiv subjects

Jérôme Renault

Publications and source records attributed to Jérôme Renault.

At least 19 recordsLinked to original sources
arXiv2025

Strategic Experimentation with Private Payoffs

We study a strategic experimentation game with exponential bandits, in which experiment outcomes are private. The equilibrium amount of experimentation is always higher than in the benchmark case where experiment outcomes are publicly observed. In addition, for pure equilibria, the equilibrium amount of experimentation is at least socially optimal, and possibly higher. We provide a tight bound on the degree of over-experimentation. The analysis rests on a new form of encouragement effect, according to which a player may hide the absence of a success to encourage future experimentation by the other player, which incentivizes current experimentation.

cs.GT↗
arXiv2024

Games played by Exponential Weights Algorithms

This paper studies the last-iterate convergence properties of the exponential weights algorithm with constant learning rates. We consider a repeated interaction in discrete time, where each player uses an exponential weights algorithm characterized by an initial mixed action and a fixed learning rate, so that the mixed action profile $p^t$ played at stage $t$ follows an homogeneous Markov chain. At first, we show that whenever a strict Nash equilibrium exists, the probability to play a strict Nash equilibrium at the next stage converges almost surely to 0 or 1. Secondly, we show that the limit of $p^t$, whenever it exists, belongs to the set of ``Nash Equilibria with Equalizing Payoffs''. Thirdly, we show that in strong coordination games, where the payoff of a player is positive on the diagonal and 0 elsewhere, $p^t$ converges almost surely to one of the strict Nash equilibria. We conclude with open questions.

cs.AI↗
arXiv2022

Optimistic Gradient Descent Ascent in Zero-Sum and General-Sum Bilinear Games

We study the convergence of Optimistic Gradient Descent Ascent in unconstrained bilinear games. In a first part, we consider the zero-sum case and extend previous results by Daskalakis et al. in 2018, Liang and Stokes in 2019, and others: we prove, for any payoff matrix, the exponential convergence of OGDA to a saddle point and also provide a new, optimal, geometric ratio for the convergence. We also characterize the step sizes inducing convergence, and are able to deduce the optimal step size for the speed of convergence. In a second part, we introduce OGDA for general-sum bilinear games: we show that in an interesting class of games, either OGDA converges exponentially fast to a Nash equilibrium, or the payoffs for both players converge exponentially fast to $+\infty$ (which might be interpreted as endogenous emergence of coordination, or cooperation, among players). We also give sufficient conditions for convergence of OGDA to a Nash equilibrium. These conditions are used to increase the speed of convergence of a min-max problem involving a matrix $A$, by introducing a general-sum game using the Moore-Penrose inverse matrix of $A$. This shows for the first time, at our knowledge, that general-sum games can be used to optimally improve algorithms designed for min-max problems. We finally illustrate our results on simple examples of Generative Adversarial Networks.

math.OC↗
arXiv2022

Competition and Recall in Selection Problems

We consider the problem in which n items arrive to a market sequentially over time, where two agents compete to choose the best possible item. When an agent selects an item, he leaves the market and obtains a payoff given by the value of the item, which is represented by a random variable following a known distribution with support contained in [0, 1]. We consider two different settings for this problem. In the first one, namely competitive selection problem with no recall, agents observe the value of each item upon its arrival and decide whether to accept or reject it, in which case they will not select it in future. In the second setting, called competitive selection problem with recall, agents are allowed to select any of the available items arrived so far. For each of these problems, we describe the game induced by the selection problem as a sequential game with imperfect information and study the set of subgame-perfect Nash equilibrium payoffs. We also study the efficiency of the game equilibria. More specifically, we address the question of how much better is to have the power of getting any available item against the take-it-or-leave-it fashion. To this end, we define and study the price of anarchy and price of stability of a game instance as the ratio between the maximal sum of payoffs obtained by players under any feasible strategy and the sum of payoffs for the worst and best subgame-perfect Nash equilibrium, respectively. For the no recall case, we prove that if there are two agents and two items arriving sequentially over time, both the price of anarchy and price of stability are upper bounded by the constant 4/3 for any value distribution. Even more, we show that this bound is tight.

cs.GT↗
arXiv2021

Value-Based Distance Between Information Structures

We define the distance between two information structures as the largest possible difference in value across all zero-sum games. We provide a tractable characterization of distance and use it to discuss the relation between the value of information in games versus single-agent problems, the value of additional information, informational substitutes, complements, or joint information. The convergence to a countable information structure under value-based distance is equivalent to the weak convergence of belief hierarchies, implying, among other things, that for zero-sum games, approximate knowledge is equivalent to common knowledge. At the same time, the space of information structures under the value-based distance is large: there exists a sequence of information structures where players acquire increasingly more information, and $ε$ > 0 such that any two elements of the sequence have distance of at least $ε$. This result answers by the negative the second (and last unsolved) of the three problems posed by J.F. Mertens in his paper Repeated Games , ICM 1986.

math.OC↗
arXiv2020

Strategic information transmission with sender's approval

We consider a sender-receiver game with an outside option for the sender. After the cheap talk phase, the receiver makes a proposal to the sender, which the latter can reject. We study situations in which the sender's approval is crucial to the receiver.We show that a partitional, (perfect Bayesian Nash) equilibrium exists if the sender has only two types or if the receiver's preferences over decisions do not depend on the type of the sender as long as the latter participates. The result does not extend: we construct a counter-example (with three types for the sender and type-dependent affine utility functions) in which there is no mixed equilibrium. In the three type case, we provide a full characterization of (possibly mediated) equilibria.

math.OC↗
arXiv2019

Value-based distance between the information structures

We dene the distance between two information structures as the largest possible dierence in the value across all zero-sum games. We provide a tractable characterization of the distance, as the minimal distance between 2 polytopes. We use it to show various results about the relation between games and single-agent problems, the value of additional information, informational substitutes, complements, etc. We show that approximate knowledge is similar to approximate common knowledge with respect to the value-based distance. Nevertheless, contrary to the weak topology, the value-based distance does not have a compact completion: there exists a sequence of information structures, where players acquire more and more information, and $ε$ > 0 such that any two elements of the sequence have distance at least $ε$. This result answers by the negative the second (and last unsolved) of the three problems posed by J.F. Mertens in his paper Repeated Games", ICM 1986.

math.OC↗
arXiv2019

A tutorial on Zero-sum Stochastic Games

Zero-sum stochastic games generalize the notion of Markov Decision Processes (i.e. controlled Markov chains, or stochastic dynamic programming) to the 2-player competitive case : two players jointly control the evolution of a state variable, and have opposite interests. These notes constitute a short mathematical introduction to the theory of such games. Section 1 presents the basic model with finitely many states and actions. We give proofs of the standard results concerning : the existence and formulas for the values of the n-stage games, of the $λ$-discounted games, the convergence of these values when $λ$ goes to 0 (algebraic approach) and when n goes to +$\infty$, an important example called 'The Big Match' and the existence of the uniform value. Section 2 presents a short and subjective selection of related and more recent results : 1-player games (MDP) and the compact non expansive case, a simple compact continuous stochastic game with no asymptotic value, and the general equivalence between the uniform convergence of (v n) n and (v $λ$) $λ$. More references on the topic can be found for instance in the books by Mertens-Sorin

math.OC↗
arXiv2019

The Large Space of Information Structures

We revisit the question of modeling incomplete information among 2 Bayesian players, following an ex-ante approach based on values of zero-sum games. $K$ being the finite set of possible parameters, an information structure is defined as a probability distribution $u$ with finite support over $K\times \mathbb{N} \times \mathbb{N}$ with the interpretation that: $u$ is publicly known by the players, $(k,c,d)$ is selected according to $u$, then $c$ (resp. $d$) is announced to player 1 (resp. player 2). Given a payoff structure $g$, composed of matrix games indexed by the state, the value of the incomplete information game defined by $u$ and $g$ is denoted $\mathrm{val}(u,g)$. We evaluate the pseudo-distance $d(u,v)$ between 2 information structures $u$ and $v$ by the supremum of $|\mathrm{val}(u,g)-\mathrm{val}(v,g)|$ for all $g$ with payoffs in $[-1,1]$, and study the metric space $Z^*$ of equivalent information structures. We first provide a tractable characterization of $d(u,v)$, as the minimal distance between 2 polytopes, and recover the characterization of Peski (2008) for $u \succeq v$, generalizing to 2 players Blackwell's comparison of experiments via garblings. We then show that $Z^*$, endowed with a weak distance $d_W$, is homeomorphic to the set of consistent probabilities with finite support over the universal belief space of Mertens and Zamir. Finally we show the existence of a sequence of information structures, where players acquire more and more information, and of $\varepsilon>0$ such that any two elements of the sequence have distance at least $\varepsilon$: having more and more information may lead now here. As a consequence, the completion of $(Z^*,d)$ is not compact, hence not homeomorphic to the set of consistent probabilities over the states of the world {\it à la} Mertens and Zamir. This example answers by the negative the second (and last unsolved) of the three problems posed by J.F. Mertens in his paper ``Repeated Games", ICM 1986.

math.OC↗
arXiv2017

Acyclic Gambling Games

We consider 2-player zero-sum stochastic games where each player controls his own state variable living in a compact metric space. The terminology comes from gambling problems where the state of a player represents its wealth in a casino. Under natural assumptions (such as continuous running payoff and non expansive transitions), we consider for each discount factor the value v $λ$ of the $λ$-discounted stochastic game and investigate its limit when $λ$ goes to 0. We show that under a strong acyclicity condition, the limit exists and is characterized as the unique solution of a system of functional equations: the limit is the unique continuous excessive and depressive function such that each player, if his opponent does not move, can reach the zone when the current payoff is at least as good than the limit value, without degrading the limit value. The approach generalizes and provides a new viewpoint on the Mertens-Zamir system coming from the study of zero-sum repeated games with lack of information on both sides. A counterexample shows that under a slightly weaker notion of acyclicity, convergence of (v $λ$) may fail.

math.OC↗
arXiv2016

On values of repeated games with signals

We study the existence of different notions of value in two-person zero-sum repeated games where the state evolves and players receive signals. We provide some examples showing that the limsup value (and the uniform value) may not exist in general. Then we show the existence of the value for any Borel payoff function if the players observe a public signal including the actions played. We also prove two other positive results without assumptions on the signaling structure: the existence of the $\sup$ value in any game and the existence of the uniform value in recursive games with nonnegative payoffs.

math.OC↗
arXiv2015

Limit value for optimal control with general means

We consider optimal control problem with an integral cost which is a mean of a given function. As a particular case, the cost concerned is the Cesàro average. The limit of the value with Cesàro mean when the horizon tends to infinity is widely studied in the literature. We address the more general question of the existence of a limit when the averaging parameter converges, for values defined with means of general types. We consider a given function and a family of costs defined as the mean of the function with respect to a family of probability measures -- the evaluations -- on R_+. We provide conditions on the evaluations in order to obtain the uniform convergence of the associated value function (when the parameter of the family converges). Our main result gives a necessary and sufficient condition in term of the total variation of the family of probability measures on R_+. As a byproduct, we obtain the existence of a limit value (for general means) for control systems having a compact invariant set and satisfying suitable nonexpansive property.

math.OC↗
arXiv2014

Hidden Stochastic Games and Limit Equilibrium Payoffs

We consider 2-player stochastic games with perfectly observed actions, and study the limit, as the discount factor goes to one, of the equilibrium payoffs set. In the usual setup where current states are observed by the players, we show that the set of stationary equilibrium payoffs always converges, and provide a simple example where the set of equilibrium payoffs has no limit. We then introduce the more general model of hidden stochastic game, where the players publicly receive imperfect signals over current states. In this setup we present an example where not only the limit set of equilibrium payoffs does not exist, but there is no converging selection of equilibrium payoffs. This second example is robust in many aspects, in particular to perturbations of the payoffs and to the introduction of correlation or communication devices.

math.OC↗
arXiv2014

The value of Markov Chain Games with incomplete information on both sides

We consider zero-sum repeated games with incomplete information on both sides, where the states privately observed by each player follow independent Markov chains. It generalizes the model, introduced by Aumann and Maschler in the sixties and solved by Mertens and Zamir in the seventies, where the private states of the players were fixed. It also includes the model introduced in Renault \cite{R2006}, of Markov chain repeated games with lack of information on one side, where only one player privately observes the sequence of states. We prove here that the limit value exists, and we obtain a characterization via the Mertens-Zamir system, where the "non revealing value function" plugged in the system is now defined as the limit value of an auxiliary "non revealing" dynamic game. This non revealing game is defined by restricting the players not to reveal any information on the {\it limit behavior} of their own Markov chain, as in Renault 2006. There are two key technical difficulties in the proof: 1) proving regularity, in the sense of equicontinuity, of the $T$-stage non revealing value functions, and 2) constructing strategies by blocks in order to link the values of the non revealing games with the original values.

math.OC↗
arXiv2013

Ramsey-type results on singletons, co-singletons and monotone sequences in large collections of sets

We say that a 0-1 matrix $N$ of size $a\times b$ can be found in a collection of sets $\mathcal{H}$ if we can find sets $H_{1}, H_{2}, \dots, H_{a}$ in $\mathcal{H}$ and elements $e_1, e_2, \dots, e_b$ in $\cup_{H \in \mathcal{H}} H$ such that $N$ is the incidence matrix of the sets $H_{1}, H_{2}, \dots, H_{a}$ over the elements $e_1, e_2, \dots, e_b$. We prove the following Ramsey-type result: for every $n\in \N$, there exists a number S(n) such that in any collection of at least S(n) sets, one can find either the incidence matrix of a collection of $n$ singletons, or its complementary matrix, or the incidence matrix of a collection of $n$ sets completely ordered by inclusion. We give several results of the same extremal set theoretical flavour. For some of these, we give the exact value of the number of sets required.

math.CO↗
arXiv2013

General limit value in Dynamic Programming

We consider a dynamic programming problem with arbitrary state space and bounded rewards. Is it possible to define in an unique way a limit value for the problem, where the "patience" of the decision-maker tends to infinity ? We consider, for each evaluation $θ$ (a probability distribution over positive integers) the value function $v_θ$ of the problem where the weight of any stage $t$ is given by $θ_t$, and we investigate the uniform convergence of a sequence $(v_{θ^k})_k$ when the "impatience" of the evaluations vanishes, in the sense that $\sum_{t} |θ^k_{t}-θ^k_{t+1}| \rightarrow_{k \to \infty} 0$. We prove that this uniform convergence happens if and only if the metric space ${v_{θ^k}, k\geq 1}$ is totally bounded. Moreover there exists a particular function $v^*$, independent of the particular chosen sequence $({θ^k})_k$, such that any limit point of such sequence of value functions is precisely $v^*$. Consequently, while speaking of uniform convergence of the value functions, $v^*$ may be considered as the unique possible limit when the patience of the decision-maker tends to infinity. The result applies in particular to discounted payoffs when the discount factor vanishes, as well as to average payoffs where the number of stages goes to infinity, and also to models with stochastic transitions. We present tractable corollaries, and we discuss counterexamples and a conjecture.

math.OC↗
arXiv2012

A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games

Given a finite set $K$, we denote by $X=Δ(K)$ the set of probabilities on $K$ and by $Z=Δ_f(X)$ the set of Borel probabilities on $X$ with finite support. Studying a Markov Decision Process with partial information on $K$ naturally leads to a Markov Decision Process with full information on $X$. We introduce a new metric $d_*$ on $Z$ such that the transitions become 1-Lipschitz from $(X, \|.\|_1)$ to $(Z,d_*)$. In the first part of the article, we define and prove several properties of the metric $d_*$. Especially, $d_*$ satisfies a Kantorovich-Rubinstein type duality formula and can be characterized by using disintegrations. In the second part, we characterize the limit values in several classes of "compact non expansive" Markov Decision Processes. In particular we use the metric $d_*$ to characterize the limit value in Partial Observation MDP with finitely many states and in Repeated Games with an informed controller with finite sets of states and actions. Moreover in each case we can prove the existence of a generalized notion of uniform value where we consider not only the Cesàro mean when the number of stages is large enough but any evaluation function $θ\in Δ(\N^*)$ when the impatience $I(θ)=\sum_{t\geq 1} |θ_{t+1}-θ_t|$ is small enough.

math.OC↗