Computer Poker Research Articles

World-class interpretable poker

We address the problem of interpretability in iterative game solving for imperfect-information games such as poker. This lack of interpretability has two main sources: first, the use of an uninterpretable feature representation, and second, the use of black box methods such as neural networks, for the fitting procedure. In this paper, we present advances on both fronts. Namely, first we propose a novel, compact, and easy-to-understand game-state feature representation for Heads-up No-limit (HUNL) Poker. Second, we make use of globally optimal decision trees, paired with a counterfactual regret minimization (CFR) self-play algorithm, to train our poker bot which produces an entirely interpretable agent. Through experiments against Slumbot, the winner of the most recent Annual Computer Poker Competition, we demonstrate that our approach yields a HUNL Poker agent that is capable of beating the Slumbot. Most exciting of all, the resulting poker bot is highly interpretable, allowing humans to learn from the novel strategies it discovers.

Survey on intelligent game of computer poker

Survey on intelligent game of computer poker

Auto-encoder neural network based prediction of Texas poker opponent's behavior

Imperfect information game is the difficulty in the game field. In the study of imperfect information games, an accurate and effective opponent modeling method is essential. Texas Hold 'em is a typical game with imperfect information. The main form of opponent modeling is to predict the opponent's behavior. This article takes Texas Hold 'em as the specific research object and proposes an opponent's behavior prediction method based on an auto-encoding neural network. We use the International Computer Poker Tournament historical data as experimental data to model Texas Hold 'em poker players' behavior and train an auto-encoding neural network model that can be applied to different stages of the game. In the article, the auto-encoding network model's structure and parameters are experimentally studied, and the quantized conjugate gradient convergence algorithm (SCG) and softmax classifier are used. Experimental test results show that this method can accurately predict the opponent's behavior information in a short time, which will provide precious help for the decision-making of the Texas Hold 'em agent.

Abstraction Using Analysis of Subgames

Normal form games are one of the most familiar representations for modeling interactions among multiple agent. However, modeling many realistic interactions between agents results in games that are extremely large. In these cases computing standard solutions like Nash equilibrium may be intractable. To overcome this issue the idea of abstraction has been investigated, most prominently in research on computer Poker. Solving a game using abstraction requires using some method to simplify the game before it is analyzed. We study a new variation for solving normal form games using abstraction that is based on finding and solving suitable sub games. We compare this method with several variations of a common type of abstraction based on clustering similar strategies.

Refining Subgames in Large Imperfect Information Games

The leading approach to solving large imperfect information games is to pre-calculate an approximate solution using a simplified abstraction of the full game; that solution is then used to play the original, full-scale game. The abstraction step is necessitated by the size of the game tree. However, as the original game progresses, the remaining portion of the tree (the subgame) becomes smaller. An appealing idea is to use the simplified abstraction to play the early parts of the game and then, once the subgame becomes tractable, to calculate a solution using a finer-grained abstraction in real time, creating a combined final strategy. While this approach is straightforward for perfect information games, it is a much more complex problem for imperfect information games. If the subgame is solved locally, the opponent can alter his play in prior to this subgame to exploit our combined strategy. To prevent this, we introduce the notion of subgame margin, a simple value with appealing properties. If any best response reaches the subgame, the improvement of exploitability of the combined strategy is (at least) proportional to the subgame margin. This motivates subgame refinements resulting in large positive margins. Unfortunately, current techniques either neglect subgame margin (potentially leading to a large negative subgame margin and drastically more exploitable strategies), or guarantee only non-negative subgame margin (possibly producing the original, unrefined strategy, even if much stronger strategies are possible). Our technique remedies this problem by maximizing the subgame margin and is guaranteed to find the optimal solution. We evaluate our technique using one of the top participants of the AAAI-14 Computer Poker Competition, the leading playground for agents in imperfect information setting

Reports on the 2015 AAAI Workshop Series

AAAI's 2015 Workshop Program was held Sunday and Monday, January 25–26, 2015 at the Hyatt Regency Austin Hotel in Austion, Texas, USA. The AAAI-15 workshop program included 15 workshops covering a wide range of topics in artificial intelligence. Most workshops were held on a single day. The titles of the workshops included AI and Ethics, AI for Cities, AI for Transportation: Advice, Interactivity and Actor Modeling, Algorithm Configuration, Artificial Intelligence Applied to Assistive Technologies and Smart Environments, Beyond the Turing Test, Computational Sustainability, Computer Poker and Imperfect Information, Incentive and Trust in E-Communities, Multiagent Interaction without Prior Coordination, Planning, Search, and Optimization, Scholarly Big Data: AI Perspectives, Challenges, and Ideas, Trajectory-Based Behaviour Analytics, World Wide Web and Public Health Intelligence, Knowledge, Skill, and Behavior Transfer in Autonomous Robots, and Learning for General Competency in Video Games.

Tartanian7: A Champion Two-Player No-Limit Texas Hold'em Poker-Playing Program

The leading approach for solving large imperfect-information games is automated abstraction followed by running an equilibrium-finding algorithm. We introduce a distributed version of the most commonly used equilibrium-finding algorithm, counterfactual regret minimization (CFR), which enables CFR to scale to dramatically larger abstractions and numbers of cores. The new algorithm begets constraints on the abstraction so as to make the pieces running on different computers disjoint. We introduce an algorithm for generating such abstractions while capitalizing on state-of-the-art abstraction ideas such as imperfect recall and the earth-mover's-distance similarity metric. Our techniques enabled an equilibrium computation of unprecedented size on a supercomputer with a high inter-blade memory latency. Prior approaches run slowly on this architecture. Our approach also leads to a significant improvement over using the prior best approach on a large shared-memory server with low memory latency. Finally, we introduce a family of post-processing techniques that outperform prior ones. We applied these techniques to generate an agent for two-player no-limit Texas Hold'em. It won the 2014 Annual Computer Poker Competition, beating each opponent with statistical significance.

Reports of the AAAI 2014 Conference Workshops

The AAAI-14 Workshop program was held Sunday and Monday, July 27–28, 2012, at the Québec City Convention Centre in Québec, Canada. Canada. The AAAI-14 workshop program included fifteen workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Robotics; Artificial Intelligence Applied to Assistive Technologies and Smart Environments; Cognitive Computing for Augmented Human Intelligence; Computer Poker and Imperfect Information; Discovery Informatics; Incentives and Trust in Electronic Communities; Intelligent Cinematography and Editing; Machine Learning for Interactive Systems: Bridging the Gap between Perception, Action and Communication; Modern Artificial Intelligence for Health Analytics; Multiagent Interaction without Prior Coordination; Multidisciplinary Workshop on Advances in Preference Handling; Semantic Cities — Beyond Open Data to Models, Standards and Reasoning; Sequential Decision Making with Big Data; Statistical Relational AI; and The World Wide Web and Public Health Intelligence. This article presents short summaries of those events.

Modified UCT Algorithm with Risk Dominance Methods in Imperfect Information Game

UCT (Upper confidential bounds on Trees) has been applied quite well as a selection approach in MCTS(Monte Carlo Tree Search) in imperfect information games like poker. By using risk dominance as complementary part of decision method besides payoff dominance, opponent strategies is better characterized as their risk factors, like bluff actions in Texas Hold’em Poker . In this paper, estimation method about the influence of risk factors on computing game equilibrium is provided. A novel algorithm, UCT-risk is proposed as modification about UCT algorithm basing on risk estimation methods. To verify the performance of new algorithm, Texas Hold’em, a popular test-bed for AI research is chosen as the experiment platform. The Agent adopted UCT-risk algorithm performs as well or better as the best previous approaches in experiments. And also it is applied in a poker agent named HITSZ_CS_13 in the 2013 AAAI Computer Poker Competition, which confirms the effectiveness of the UCT-risk provided in this paper.

The Annual Computer Poker Competition

Now entering its eighth year, the Annual Computer Poker Competition (ACPC) is the premier event within the field of computer poker. With both academic and nonacademic competitors from around the world, the competition provides an open and international venue for benchmarking computer poker agents. We describe the competition's origins and evolution, current events, and winning techniques

Case-based strategies in computer poker

The state-of-the-art within Artificial Intelligence has directly benefited from research conducted within the computer poker domain. One such success has been the advancement of bottom up equilibrium finding algorithms via computational game theory. On the other hand, alternative top down approaches, that attempt to generalise decisions observed within a collection of data, have not received as much attention. In this work we employ a top down approach in order to construct case-based strategies within three computer poker domains. Our analysis begins within the simplest variation of Texas Hold'em poker, i.e. two-player, limit Hold'em. We trace the evolution of our case-based architecture and evaluate the effect that modifications have on strategy performance. The end result of our experimentation is a coherent framework for producing strong case-based strategies based on the observation and generalisation of expert decisions. The lessons learned within this domain offer valuable insights, that we use to apply the framework to the more complicated domains of two-player, no-limit Hold'em and multi-player, limit Hold'em. For each domain we present results obtained from the Annual Computer Poker Competition, where the best poker agents in the world are challenged against each other. We also present results against human opposition.

Decision-making and thought processes among poker players

This study was aimed at delineating decision-making and thought processing among poker players who vary in skill-level. Forty-five participants, 15 in each group, comprised expert, intermediate, and novice poker players. They completed the Computer Poker Simulation Task (CPST) comprised of 60 hands of No-Limit Texas Hold ‘Em. During the CPST, they were asked to ‘think out loud’ throughout the hand. The 60 hands were broken down into two time conditions (e.g., 15 seconds and no time restriction). Findings indicated that expert and intermediate players outperformed novice players in decision-making (DM) performance. This difference was largest at later stages of the hand. Expert players reported processing more thoughts than intermediate players and novice players. Additionally, experts and intermediates reported attending to situational-relevant cues while novices focused on basic poker considerations and irrelevant cues. Contrary to research in other areas, the novice players displayed the greatest increase in DM performance during the limited time condition. The uniqueness and constraints of poker are used to explain and elaborate on the study’s findings.

The State of Solving Large Incomplete‐Information Games, and Application to Poker

Game‐theoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incomplete‐information games. They encompass many practical problems such as auctions, negotiations, and security applications. I will discuss them in the context of how they have transformed computer poker. In short, game‐theoretic reasoning now scales to many large problems, outperforms the alternatives on those problems, and in some games beats the best humans.

Algorithms and Assessment in Computer Poker

Algorithms and Assessment in Computer Poker