flâneur

AlphaExploitem: Going Beyond the Nash Equilibrium in Poker by Learning to Exploit Suboptimal Play

arxiv.org · 8,819 words · saved by 1 readers

Poker is an imperfect information game that has served as a long-standing benchmark for decision-making under uncertainty. To maximize utility beyond the Nash equilibrium, an agent can deviate from Nash-equilibrium policies to exploit suboptimal play. We introduce AlphaExploitem, which extends the competitive RL poker agent AlphaHoldem by using a hierarchical transformer encoder that enables reasoning over previously played hands and modifying the training procedure with the inclusion of a diverse pool of exploitable opponents to facilitate learning to exploit. We train and evaluate AlphaExploitem on two standard benchmarks for imperfect-information games. Empirically, AlphaExploitem successfully exploits weak play by both in- and out-of-distribution opponents, without losing performance against NE opponents. Poker is a family of card games whose incomplete information setting, stochastic elements, and deep strategic complexity have made it a long-standing challenge for artificial inte

Vlad Murgoci Affiliation: Delft University of Technology Affiliation: 2628 CD Delft, The Netherlands Email: v.murgoci@tudelft.nl Matthijs Spaan Affiliation: Department of Intelligent Systems Affiliation: Delft University of Technology Affiliation: 2628 CD Delft, The Netherlands Email: M.T.J.Spaan@tudelft.nl Yaniv Oren Affiliation: Department of Intelligent Systems Affiliation: Delft University of Technology Affiliation: 2628 CD Delft, The Netherlands Email: y.oren@tudelft.nl Abstract Poker is an imperfect information game that has served as a long-standing benchmark for…

saved by

related reading