Thresholds for sensitive optimality and Blackwell optimality in stochastic games

Abstract

We investigate refinements of the mean-payoff criterion in two-player zero-sum perfect-information stochastic games. A strategy is Blackwell optimal if it is optimal in the discounted game for all discount factors sufficiently close to $1$. The notion of $d$-sensitive optimality interpolates between mean-payoff optimality (corresponding to the case $d=-1$) and Blackwell optimality ($d=+\infty$). The Blackwell threshold $\alpha_{\sf Bw} \in [0,1[$ is the discount factor above which all optimal strategies in the discounted game are guaranteed to be Blackwell optimal. The $d$-sensitive threshold $\alpha_{\sf d} \in [0,1[$ is defined analogously. Bounding $\alpha_{\sf Bw}$ and $\alpha_{\sf d}$ are fundamental problems in algorithmic game theory, since these thresholds control the complexity for computing Blackwell and $d$-sensitive optimal strategies, by reduction to discounted games which can be solved in $O\left((1-\alpha)^{-1}\right)$ iterations. We provide the first bounds on the $d$-sensitive threshold $\alpha_{\sf d}$ beyond the case $d=-1$, and we establish improved bounds for the Blackwell threshold $\alpha_{\sf Bw}$. This is achieved by leveraging separation bounds on algebraic numbers, relying on Lagrange bounds and more advanced techniques based on Mahler measures and multiplicity theorems.