arXiv subjects

Ondřej Kubíček

Publications and source records attributed to Ondřej Kubíček.

4 recordsLinked to original sources

arXiv2026

Regularized policy gradient with learned mixtures of Gaussians for games with continuous actions

Most successes of superhuman game-playing algorithms are in games with discrete actions, yet in auctions, robotics, sports, or trading, actions are nearly continuous. Prior techniques either rely on expert-designed discretizations or are sample inefficient. We present a scalable policy-gradient algorithm for large sequential games with continuous or mixed discrete and continuous actions. It combines magnetic mirror descent with a mixture of Gaussians reparametrization, trained via self-play. We show that it approximates equilibrium in games where gradient descent fails. In sequential games, it outperforms neural fictitious self-play and matches or outperforms the final strategies of policy space response oracles with 3.5--5.5$\times$ fewer samples. In heads-up no-limit Texas hold'em, it performs on par with Slumbot.

cs.MA↗

arXiv2025

Understanding Optimal Portfolios of Strategies for Solving Two-player Zero-sum Games

In large-scale games, approximating the opponent's strategy space with a small portfolio of representative strategies is a common and powerful technique. However, the construction of these portfolios often relies on domain-specific knowledge or heuristics with no theoretical guarantees. This paper establishes a formal foundation for portfolio-based strategy approximation. We define the problem of finding an optimal portfolio in two-player zero-sum games and prove that this optimization problem is NP-hard. We demonstrate that several intuitive heuristics-such as using the support of a Nash Equilibrium or building portfolios incrementally - can lead to highly suboptimal solutions. These negative results underscore the problem's difficulty and motivate the need for robust, empirically-validated heuristics. To this end, we introduce an analytical framework to bound portfolio quality and propose a methodology for evaluating heuristic approaches. Our evaluation of several heuristics shows that their success heavily depends on the specific game being solved. Our code is publicly available.

cs.GT↗

arXiv2025

Look-ahead Reasoning with a Learned Model in Imperfect Information Games

Test-time reasoning significantly enhances pre-trained AI agents' performance. However, it requires an explicit environment model, often unavailable or overly complex in real-world scenarios. While MuZero enables effective model learning for search in perfect information games, extending this paradigm to imperfect information games presents substantial challenges due to more nuanced look-ahead reasoning techniques and large number of states relevant for individual decisions. This paper introduces an algorithm LAMIR that learns an abstracted model of an imperfect information game directly from the agent-environment interaction. During test time, this trained model is used to perform look-ahead reasoning. The learned abstraction limits the size of each subgame to a manageable size, making theoretically principled look-ahead reasoning tractable even in games where previous methods could not scale. We empirically demonstrate that with sufficient capacity, LAMIR learns the exact underlying game structure, and with limited capacity, it still learns a valuable abstraction, which improves game playing performance of the pre-trained agents even in large games.

cs.AI↗

arXiv2024

Continual Depth-limited Responses for Computing Counter-strategies in Sequential Games

In zero-sum games, the optimal strategy is well-defined by the Nash equilibrium. However, it is overly conservative when playing against suboptimal opponents and it can not exploit their weaknesses. Limited look-ahead game solving in imperfect-information games allows defeating human experts in massive real-world games such as Poker, Liar's Dice, and Scotland Yard. However, since they approximate Nash equilibrium, they tend to only win slightly against weak opponents. We propose methods combining limited look-ahead solving with an opponent model in order to 1) approximate a best response in large games or 2) compute a robust response with control over the robustness of the response. Both methods can compute the response in real time to previously unseen strategies. We present theoretical guarantees of our methods. We show that existing robust response methods do not work combined with limited look-ahead solving of the shelf, and we propose a novel solution for the issue. Our algorithm performs significantly better than multiple baselines in smaller games and outperforms state-of-the-art methods against SlumBot.

cs.GT↗