Super-Human AI for Strategic Reasoning: Beating Top Pros in Heads-Up No-Limit Texas Hold'em

Abstract

Poker has been a challenge problem in AI and game theory for decades. As a game of imperfect information it involves obstacles not present in games like chess and Go, and requires totally different techniques. No program had been able to beat top players in large poker games. Until now! In January 2017, our AI, Libratus, beat a team of four top specialist professionals in heads-up no-limit Texas hold'em, which has 10^161 decision points. This game is the main benchmark challenge for imperfect-information game solving. Libratus is the only AI that has beat top humans at this game. Libratus is powered by new algorithms in each of its three main modules: 1. computing blueprint (approximate Nash equilibrium) strategies before the event, 2. novel nested endgame solving during play, and 3. fixing its own strategy to play even closer to equilibrium based on what holes the opponents have been able to identify and exploit. These domain-independent algorithms have potential applicability to a variety of real-world imperfect-information games such as negotiation, business strategy, cybersecurity, physical security, military applications, strategic pricing, product portfolio planning, certain areas of finance, auctions, political campaigns, and steering biological adaptation and evolution, for example, for medical treatment planning.