On the Power of Deception in Repeated Games

Abstract

In repeated games, opponents often predict what we'll do next by looking at what we have done so far. This allows us to deceive them: we can deliberately behave one way for a period of time to shape their expectations, then switch strategies to profit from the induced response. We study deception in repeated two-player normal-form games against count-based learners, whose behavior depends only on how often we have played each action in the past. We formalize deceptive and non-deceptive play, and introduce the notion of a deception bonus, the payoff gain of the best deceptive strategy over the best fixed mixed strategy. We establish structural results on deception in general-sum games. We design exact dynamic programs for optimizing against any count-based learner when the action space or opponent's memory is small, and develop approximation algorithms for settings where the opponent's memory or the time horizon is large. We also provide an approximation algorithm for learning to deceive an opponent whose count-based learning rule is unknown. To complement our algorithmic results, we show that approximating the optimal deceptive payoff against the classic Empirical Risk Minimization (ERM) learning rule is NP-hard, including obtaining any constant-factor approximation or even a $T^\alpha$-additive approximation for any $0 < \alpha < 1$. Finally, we empirically measure the deception bonus in random games with i.i.d. payoffs.