Limit Texas Hold Research Articles

Research of artificial intelligence in imperfect information card games

Artificial intelligence (AI) in games has advanced significantly, notably in perfect information games such as Go and Chess. Imperfect information games, in which participants do not have complete information about the game state, create more difficulties. They incorporate both public and private observations, where strategies must be improved to achieve a Nash equilibrium. This study investigates artificial intelligence and reinforcement learning approaches, in which agents learn to maximize future rewards through interactions with their surroundings. The paper then focuses on card game research platforms such as RLCard and OpenAI Gym. It gives a comprehensive summary of research in No Limit Texas Hold'em, a difficult two-player poker game with a large decision space. DeepStack and Libratus are successful systems that have attained expert-level and superhuman play, respectively. Pluribus, a superhuman artificial intelligence for six-player poker, and DouZero, a pure reinforcement learning technique for the multiplayer card game, DouDiZhu, are both investigated. Overall, this paper provides background information on reinforcement learning and imperfect information games, analyzes commonly used research platforms, evaluates the effectiveness of AI algorithms in various card games, and offers future research areas and directions.

Finding Optimal Abstract Strategies in Extensive-Form Games

Extensive-form games are a powerful model for representing interactions between agents. Nash equilibrium strategies are a common solution concept for extensive-form games and, in two-player zero-sum games, there are efficient algorithms for calculating such strategies. In large games, this computation may require too much memory and time to be tractable. A standard approach in such cases is to apply a lossy state-space abstraction technique to produce a smaller abstract game that can be tractably solved, while hoping that the resulting abstract game equilibrium is close to an equilibrium strategy in the unabstracted game. Recent work has shown that this assumption is unreliable, and an arbitrary Nash equilibrium in the abstract game is unlikely to be even near the least suboptimal strategy that can be represented in that space. In this work, we present for the first time an algorithm which efficiently finds optimal abstract strategies --- strategies with minimal exploitability in the unabstracted game. We use this technique to find the least exploitable strategy ever reported for two-player limit Texas hold'em.

A View on Deep Reinforcement Learning in Imperfect Information Games

Many real-world applications can be described as large-scale games of imperfect information. This kind of games is particularly harder than the deterministic one as the search space is even more sizeable. In this paper, I want to explore the power of reinforcement learning in such an environment; that is why I take a look at one of the most popular game of such type, no limit Texas Hold’em Poker, yet unsolved, developing multiple agents with different learning paradigms and techniques and then comparing their respective performances. When applied to no-limit Hold’em Poker, deep reinforcement learning agents clearly outperform agents with a more traditional approach. Moreover, if these last agents rival a human beginner level of play, the ones based on reinforcement learning compare to an amateur human player. The main algorithm uses Fictitious Play in combination with ANNs and some handcrafted metrics. We also applied the main algorithm to another game of imperfect information, less complex than Poker, in order to show the scalability of this solution and the increase in performance when put neck in neck with established classical approaches from the reinforcement learning literature.

Heart-rate sonification biofeedback for poker

Abstract In this paper we show that heart-rate based audio/sonification biofeedback (HRSB) can be used to facilitate emotion regulation during poker play. We report on a laboratory experiment (N= 29) where participants play No Limit Texas Hold’em poker while hearing heartbeats synchronized with their actual heart-rate (biofeedback condition) or at steady pace (control condition). The synchronized heart-rate biofeedback decreased emotional reactivity in terms of arousal (as measured by skin conductance) and valence (as measured by facial electromyography). We also observed individual differences between participants in the effectiveness of the HRSB. The participants were profiled using the behavioural inhibition/activation system (BIS/BAS) questionnaires, and there was a significant correlation between the effectiveness of the biofeedback method and BIS/BAS scores; with biofeedback being effective primarily for participants with high BIS/BAS scores.

Heads-up limit hold'em poker is solved

Poker is a family of games that exhibit imperfect information, where players do not have full knowledge of past events. While many perfect information games have been solved (e.g., Connect-Four and checkers), no nontrivial imperfect information game played competitively by humans has previously been solved. In this paper, we announce that the smallest variant of poker in-play, heads-up limit Texas hold'em, is now essentially weakly solved. Furthermore, this computation formally proves the common wisdom that the dealer in the game holds a significant advantage. This result was enabled by a new algorithm, CFR + , which is capable of solving extensive-form games three orders of magnitude larger than previously possible. This paper is an extended version of the original 2015 Science article, with additional results showing Cepheus' in-game performance against computer and human opponents.

Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks

Poker is a family of card games that includes many varia- tions. We hypothesize that most poker games can be solved as a pattern matching problem, and propose creating a strong poker playing system based on a unified poker representa- tion. Our poker player learns through iterative self-play, and improves its understanding of the game by training on the results of its previous actions without sophisticated domain knowledge. We evaluate our system on three poker games: single player video poker, two-player Limit Texas Hold’em, and finally two-player 2-7 triple draw poker. We show that our model can quickly learn patterns in these very different poker games while it improves from zero knowledge to a competi- tive player against human experts. The contributions of this paper include: (1) a novel represen- tation for poker games, extendable to different poker vari- ations, (2) a Convolutional Neural Network (CNN) based learning model that can effectively learn the patterns in three different games, and (3) a self-trained system that signif- icantly beats the heuristic-based program on which it is trained, and our system is competitive against human expert players.

Computer science. Heads-up limit hold'em poker is solved.

Poker is a family of games that exhibit imperfect information, where players do not have full knowledge of past events. Whereas many perfect-information games have been solved (e.g., Connect Four and checkers), no nontrivial imperfect-information game played competitively by humans has previously been solved. Here, we announce that heads-up limit Texas hold'em is now essentially weakly solved. Furthermore, this computation formally proves the common wisdom that the dealer in the game holds a substantial advantage. This result was enabled by a new algorithm, CFR(+), which is capable of solving extensive-form games orders of magnitude larger than previously possible.

“You Fold Like a Little Girl:” (Hetero)Gender Framing and Competitive Strategies of Men and Women in No Limit Texas Hold Em Poker Games

The formal rules, structure and practice of most sports in contemporary society prohibit men and women from competing on a “level playing field” and diminish women’s ability to launch a legitimate challenge to the masculine superiority embedded in sports competition. This study examines a relatively unique case—No Limit Texas Hold Em poker games in which men and women compete directly against one another under the same rules—to explore how the conditions under which men and women compete enable or impede the development of more gender egalitarian interactions and ideological frameworks. Drawing on ethnographic data, this examination reveals that, even in a more gender-neutral context, men and women learn to use heterogender frames to conceptualize poker. In doing so, they develop competitive strategies and interactions that predominately fit into, rather than subvert, gender hierarchy.

Automated Action Abstraction of Imperfect Information Extensive-Form Games

Multi-agent decision problems can often be formulated as extensive-form games. We focus on imperfect information extensive-form games in which one or more actions at many decision points have an associated continuous or many-valued parameter. A stock trading agent, in addition to deciding whether to buy or not, must decide how much to buy. In no-limit poker, in addition to selecting a probability for each action, the agent must decide how much to bet for each betting action. Selecting values for these parameters makes these games extremely large. Two-player no-limit Texas Hold'em poker with stacks of 500 big blinds has approximately 1071 states, which is more than 1050 times more states than two-player limit Texas Hold'em. The main contribution of this paper is a technique that abstracts a game's action space by selecting one, or a small number, of the many values for each parameter. We show that strategies computed using this new algorithm for no-limit Leduc poker exhibit significant utility gains over epsilon-Nash equilibrium strategies computed with standard, hand-crafted parameter value abstractions.