Poker Game Research Articles

Skewness preferences: Evidence from online poker

We test for skewness preferences in a large set of observational panel data on online poker games (n=4,450,585). Each observation refers to a choice between a safe option and a binary risk of winning or losing the game. Our setting offers a real-world choice situation with substantial incentives where probability distributions are simple, transparent, and known to the decision-makers. Individuals reveal a strong and robust preference for skewness, which is inconsistent with expected utility theory. The effect of skewness is most pronounced among experienced and unsuccessful players but remains significant in all subsamples that we investigate, in contrast to the effect of variance.

Research on the Ways to Win in a Simple Poker Game

Research on the Ways to Win in a Simple Poker Game

Poker in light of the Gambling Act and European Union regulations

The authors of the paper undertook an analysis of the regulations on the organization of a poker game under the Gambling Act. The main subject of the analysis revolves around the question to what extent state authorities should monitor and restrict the activity of poker players. The scope of the statutory restrictions on arranging and participating in poker games depends on the culture of perception of gambling in a given country. Assigning poker to the appropriate regulatory category, either as a gambling activity carrying the highest social risks in terms of public health or safety, or as a game primarily for entertainment, is by no means obvious and depends on determining such factors as the purpose of organizing the poker game and the manner in which it is played, i.e. whether players are playing against a casino or against each other. The regulatory differentiation depending on the variety of poker games is the result of years of efforts by national legislators and the jurisprudence to define and categorize the game correctly. The article cites the jurisprudence of the Constitutional Court justifying the formal and material prerequisites of the existing restrictions on the gambling services market. The authors also relied on the extensive body of case law of the Court of Justice of the European Union explicating the permissibility of restrictions on freedom of establishment and freedom to provide services in the sphere of gambling. The conclusions drawn from these reflections confirm that the current provisions of the Polish Gambling Act are part of a consistent and systematic state policy aimed at ensuring the highest possible level of consumer protection, and the introduced restrictions are proportionate and necessary.

Modeling Rationality: Toward Better Performance Against Unknown Agents in Sequential Games.

Opponent modeling is necessary for autonomous agents to capture the intents of others during strategic interactions. Most previous works assume that they can access enough interaction history to build the model. However, it may not be realistic. To solve this problem, we present a novel rationality-consistent opponent modeling (ROM) method for games with imperfect information. In our approach, a game-theoretical concept of consistence about rationality is proposed to take advantage of the characteristic of imperfect information sequential games that rational behavior at disjoint information sets is correlated through anticipated opponent's behavior. With the correlation between different information sets, agents could infer the opponents' strategies at information sets correlated to observed behavior. To exploit the correlation, ROM attempts to conduct reasoning from the opponent's perspective and rationalize its past behavior. In this way, ROM acquires the ability to better adapt to different opponents and achieves a more accurate opponent model with insufficient observation history, which is verified by experiments in different settings. A heuristic adaptation approach is also applied in ROM, which updates the opponent model in an online manner and significantly reduces the computation cost. We evaluate ROM in both a grid world game and a poker game. Compared with other opponent modeling methods, ROM shows better performance and has more accurate predictions in both games against different types of opponents with limited action interactions. Experimental results also show that ROM's time cost is significantly reduced through heuristic adaptation.

Delegation Poker Game in Nursing.

Delegation Poker Game in Nursing.

Learning Not to Regret

The literature on game-theoretic equilibrium finding predominantly focuses on single games or their repeated play. Nevertheless, numerous real-world scenarios feature playing a game sampled from a distribution of similar, but not identical games, such as playing poker with different public cards or trading correlated assets on the stock market. As these similar games feature similar equilibra, we investigate a way to accelerate equilibrium finding on such a distribution. We present a novel ``learning not to regret'' framework, enabling us to meta-learn a regret minimizer tailored to a specific distribution. Our key contribution, Neural Predictive Regret Matching, is uniquely meta-learned to converge rapidly for the chosen distribution of games, while having regret minimization guarantees on any game. We validated our algorithms' faster convergence on a distribution of river poker games. Our experiments show that the meta-learned algorithms outpace their non-meta-learned counterparts, achieving more than tenfold improvements.

Research of artificial intelligence in imperfect information card games

Artificial intelligence (AI) in games has advanced significantly, notably in perfect information games such as Go and Chess. Imperfect information games, in which participants do not have complete information about the game state, create more difficulties. They incorporate both public and private observations, where strategies must be improved to achieve a Nash equilibrium. This study investigates artificial intelligence and reinforcement learning approaches, in which agents learn to maximize future rewards through interactions with their surroundings. The paper then focuses on card game research platforms such as RLCard and OpenAI Gym. It gives a comprehensive summary of research in No Limit Texas Hold'em, a difficult two-player poker game with a large decision space. DeepStack and Libratus are successful systems that have attained expert-level and superhuman play, respectively. Pluribus, a superhuman artificial intelligence for six-player poker, and DouZero, a pure reinforcement learning technique for the multiplayer card game, DouDiZhu, are both investigated. Overall, this paper provides background information on reinforcement learning and imperfect information games, analyzes commonly used research platforms, evaluates the effectiveness of AI algorithms in various card games, and offers future research areas and directions.

The State-Of-Art Applications of Conventional Probability Issues

Contemporarily, the applications of conventional statistics to state-of-art problems become increasingly ingenious and complicated. Specifically, poker games, secretary puzzles, and the newsvendor theory will be fully discussed and elaborated in this paper. First, we discuss how to select the initial hand poker cards with the same card power research. The experiment proves that the method of equal card power generation is effective and feasible. In addition, we discuss the secretary puzzle, based on the optimal stopping theory, with the basic condition discussed by using different methods, probability, and calculus. Finally, we elaborate on the application of the newsvendor theory to the hotel overbooking problem. The theory is suitable for helping make decisions in manufacturing and service industries. When resolving the hotel overbooking problem, we successfully decide the number of rooms that the hotel reservations need to overbook in order to maximize the profit through the newsvendor model and the probabilistic approach, whose results turn out to be the same. Overall, these results shed light on guiding further exploration of statistics and probability.

Increased volatility in video poker results in more winning players but shorter winning streaks - Evidence from simulations.

Electronic gambling machines are a prominent cause of significant gambling harms globally. We use simulations of a simplified video poker game to show how changes in game volatility, defined primarily by the size of the main prize, affect patterns of wins and losses as well as winning streaks. We found that in low- and medium volatility games the proportion of winning players quickly drops to zero after about 30 h of play, while in the high volatility game 5% of players are still winning after playing for 100 h. However, the proportion of winning streaks was significantly higher in the low- and medium volatility games compared with high volatility: the simulated players were on a winning streak about 26.3, 25.6 and 18% of the time in the low-, medium- and high volatility games, respectively. Fast-paced video poker with varying volatility levels but identical return-to-player rates and win frequencies can yield highly different result patterns across individuals. These patterns may be counter-intuitive for players and difficult to realize without simulations and visualizations. We argue that the findings have relevance for responsible gambling communication and for building a better understanding of how cognitive biases influence gambling behaviour.

Game theory--optimum strategy for drawing cards in 21 points

Blackjack is a popular casino poker game in which players make drawing decisions based on rules other than the size of their points. We found that despite the randomness of the card draw, blackjack is actually very mathematical and logical. By calculating probabilities and using game theory, we can allow players to make relatively "rational" decisions that maximize their expectations of winning, rather than relying solely on guesswork and vague feelings. Therefore, in this article, we built several models to simulate a blackjack game with different numbers of people. For players in each model, we give them a code that outputs the optimal decision, which helps them figure out the winning percentage (more specifically, the expectation, since once stakes are involved in multiplayer games, the winning percentage doesn't necessarily represent the expectation of winning money) for each decision, thus helping the player make a better decision.

Spatial attention to social information in poker: A neuropsychological study using the Posner cueing paradigm.

This research aimed to characterize social information processing abilities in a population of regular nondisordered poker players compared to controls. Participants completed the Posner cueing paradigm task including social cues (faces) to assess attention allocation towards social stimuli, including the effect of the presentation time (subliminal vs supraliminal) and of the emotion displayed. The study included two groups of participants: 30 regular nondisordered poker players (those who played at least three times a week in Texas Hold'em poker games for at least three months) and 30 control participants (those who did not gamble or gambled less than once a month, whatever the game). The group of regular nondisordered poker players displayed anenhancement of the inhibition of return during the Posner cueing task. This means that in valid trials, they took longer to respond to the already processed localization in supraliminal conditions compared to controls. However, our results did not evidence any particular engagement or disengagement attention abilities toward specific types of emotion. These results suggest that regular nondisordered poker players displayed social information processing abilities, which may be due to the importance to efficiently process social information that can serve as tells in live poker. The observed enhancement of the inhibition of return may permit poker players to not process a localization that has already processed to save attentional resources. Further research regarding the establishment of the IOR in other forms of gambling and with non-social cues needs to be performed.

Extensive game decision based on the PPO-CFR algorithm under incomplete information

Human-computer gaming under incomplete information is usually described by a two-player zero-sum game model. Counterfactual regret minimization (CFR) is a popular algorithm for two-player zero-sum games with incomplete information. However, the existing CFR and its variant algorithms use fixed regret calculation and strategy update type in the iteration process, which have their advantages and disadvantages in the incomplete information extensive game, and their generalization performance is weak. To solve this problem, this paper combines the proximal policy optimization (PPO) algorithm in reinforcement learning with the CFR algorithm to train rational agents to adaptively select appropriate regret calculation and strategy update types in the CFR iteration process to improve the generalization performance of the current CFR algorithms and realize the policy optimization of the incomplete information extensive game. In this paper, general poker game experiments are used to verify the proposed algorithm, and a stepwise reward function is formulated to train the action policy of the agent. Experimental results show that compared with existing state-of-the-art methods, the PPO-CFR algorithm has better generalization performance and lower exploitability, and the iteration policy is closer to the Nash equilibrium policy.

Pandemic Revealed Unique Aspects of Sexual Addictions

Pandemic Revealed Unique Aspects of Sexual Addictions

Needle in a Haystack: Spotting and recognising micro-expressions “in the wild”

Computational research on facial micro-expressions has long focused on videos captured under constrained laboratory conditions due to the challenging elicitation process and limited samples that are publicly available. Moreover, processing micro-expressions is extremely challenging under unconstrained scenarios. This paper introduces, for the first time, a completely automatic micro-expression “spot-and-recognize” framework that is performed on in-the-wild videos, such as in poker games and political interviews. The proposed method first spots the apex frame from a video by handling head movements and unconscious actions which are typically larger in motion intensity, with alignment employed to enforce a canonical face pose. Optical flow guided features play a central role in our method: they can robustly identify the location of the apex frame, and are used to learn a shallow neural network model for emotion classification. Experimental results demonstrate the feasibility of the proposed methodology, establishing good baselines for both spotting and recognition tasks – ASR of 0.33 and F1-score of 0.6758 respectively on the MEVIEW micro-expression database. In addition, we present comprehensive qualitative and quantitative analyses to further show the effectiveness of the proposed framework, with new suggestion for an appropriate evaluation protocol. In a nutshell, this paper provides a new benchmark for apex spotting and emotion recognition in an in-the-wild setting.

Fast Payoff Matrix Sparsification Techniques for Structured Extensive-Form Games

The practical scalability of many optimization algorithms for large extensive-form games is often limited by the games' huge payoff matrices. To ameliorate the issue, Zhang and Sandholm recently proposed a sparsification technique that factorizes the payoff matrix A into a sparser object A = Â + UVᵀ, where the total combined number of nonzeros of Â, U, and V, is significantly smaller. Such a factorization can be used in place of the original payoff matrix in many optimization algorithm, such as interior-point and second-order methods, thus increasing the size of games that can be handled. Their technique significantly sparsifies poker (end)games, standard benchmarks used in computational game theory, AI, and more broadly. We show that the existence of extremely sparse factorizations in poker games can be tied to their particular Kronecker-product structure. We clarify how such structure arises and introduce the connection between that structure and sparsification. By leveraging such structure, we give two ways of computing strong sparsifications of poker games (as well as any other game with a similar structure) that are i) orders of magnitude faster to compute, ii) more numerically stable, and iii) produce a dramatically smaller number of nonzeros than the prior technique. Our techniques enable—for the first time—effective computation of high-precision Nash equilibria and strategies subject to constraints on the amount of allowed randomization. Furthermore, they significantly speed up parallel first-order game-solving algorithms; we show state-of-the-art speed on a GPU.

Optimal Policy of Multiplayer Poker via Actor-Critic Reinforcement Learning

Poker has been considered a challenging problem in both artificial intelligence and game theory because poker is characterized by imperfect information and uncertainty, which are similar to many realistic problems like auctioning, pricing, cyber security, and operations. However, it is not clear that playing an equilibrium policy in multi-player games would be wise so far, and it is infeasible to theoretically validate whether a policy is optimal. Therefore, designing an effective optimal policy learning method has more realistic significance. This paper proposes an optimal policy learning method for multi-player poker games based on Actor-Critic reinforcement learning. Firstly, this paper builds the Actor network to make decisions with imperfect information and the Critic network to evaluate policies with perfect information. Secondly, this paper proposes a novel multi-player poker policy update method: asynchronous policy update algorithm (APU) and dual-network asynchronous policy update algorithm (Dual-APU) for multi-player multi-policy scenarios and multi-player sharing-policy scenarios, respectively. Finally, this paper takes the most popular six-player Texas hold ’em poker to validate the performance of the proposed optimal policy learning method. The experiments demonstrate the policies learned by the proposed methods perform well and gain steadily compared with the existing approaches. In sum, the policy learning methods of imperfect information games based on Actor-Critic reinforcement learning perform well on poker and can be transferred to other imperfect information games. Such training with perfect information and testing with imperfect information models show an effective and explainable approach to learning an approximately optimal policy.

Efficient Opponent Exploitation in No-Limit Texas Hold’em Poker: A Neuroevolutionary Method Combined with Reinforcement Learning

In the development of artificial intelligence (AI), games have often served as benchmarks to promote remarkable breakthroughs in models and algorithms. No-limit Texas Hold’em (NLTH) is one of the most popular and challenging poker games. Despite numerous studies having been conducted on this subject, there are still some important problems that remain to be solved, such as opponent exploitation, which means to adaptively and effectively exploit specific opponent strategies; this is acknowledged as a vital issue especially in NLTH and many real-world scenarios. Previous researchers tried to use an off-policy reinforcement learning (RL) method to train agents that directly learn from historical strategy interactions but suffered from challenges of sparse rewards. Other researchers instead adopted neuroevolutionary (NE) method to replace RL for policy parameter updates but suffered from high sample complexity due to the large-scale problem of NLTH. In this work, we propose NE_RL, a novel method combing NE with RL for opponent exploitation in NLTH. Our method contains a hybrid framework that uses NE’s advantage of evolutionary computation with a long-term fitness metric to address the sparse rewards feedback in NLTH and retains RL’s gradient-based method for higher learning efficiency. Experimental results against multiple baseline opponents have proved the feasibility of our method with significant improvement compared to previous methods. We hope this paper provides an effective new approach for opponent exploitation in NLTH and other large-scale imperfect information games.

Winning Rate Prediction Model Based on Monte Carlo Tree Search for Computer Dou Dizhu

Poker is the typical game of incomplete information, and remains a longstanding challenge problem in artificial intelligence (AI). The poker game of Dou Dizhu has been viewed as a thorny topic in AI because of its own characteristics. This article introduces a developed Monte Carlo tree search (MCTS) method for Dou Dizhu to solve the decision making effectively. We built the winning rate prediction model (WRPM) to predict the winning rate of moves as the initial situation estimation and improve the model to be more applicable to different player roles. Then, the WRPM is embedded as the core algorithm into MCTS for extension and simulation and named it WRPM-MCTS. In addition, we also train a card distribution prediction model to predict the holding cards of opponents for further improving the performance of WRPM-MCTS on the agent of Dou Dizhu. Experiments show that the WRPM-MCTS has a statistically significant performance better than the pure MCTS and the pure WRPM. In the game with human players from an online game platform, the WRPM-MCTS-based agent had the winning rate of 52.86% in 4 000 000 games and ranked in top 1.22% among 500 000 human players, indicating that this agent had reached the expert level of humans.

Faster Game Solving via Predictive Blackwell Approachability: Connecting Regret Matching and Mirror Descent

Blackwell approachability is a framework for reasoning about repeated games with vector-valued payoffs. We introduce predictive Blackwell approachability, where an estimate of the next payoff vector is given, and the decision maker tries to achieve better performance based on the accuracy of that estimator. In order to derive algorithms that achieve predictive Blackwell approachability, we start by showing a powerful connection between four well-known algorithms. Follow-the-regularized-leader (FTRL) and online mirror descent (OMD) are the most prevalent regret minimizers in online convex optimization. In spite of this prevalence, the regret matching (RM) and regret matching+ (RM+) algorithms have been preferred in the practice of solving large-scale games (as the local regret minimizers within the counterfactual regret minimization framework). We show that RM and RM+ are the algorithms that result from running FTRL and OMD, respectively, to select the halfspace to force at all times in the underlying Blackwell approachability game. By applying the predictive variants of FTRL or OMD to this connection, we obtain predictive Blackwell approachability algorithms, as well as predictive variants of RM and RM+. In experiments across 18 common zero-sum extensive-form benchmark games, we show that predictive RM+ coupled with counterfactual regret minimization converges vastly faster than the fastest prior algorithms (CFR+, DCFR, LCFR) across all games but two of the poker games, sometimes by two or more orders of magnitude.

"Time to Take a Break"

Although often ignored, adult gamers, as with children and adolescents, can suffer from problematic gaming. This study explores the use of a built-in gaming gradual intervention system (G-GIS) designed to help adult gamers achieve their desired gaming habits in an autonomous and acceptable manner. In this study, we interviewed 26 heavy adult gamers (i.e., adult gamers who played frequently and for relatively long periods of time) of an online poker game using an in-built G-GIS. Then, we triangulated the interviewees' results with their real-world behavioral data collected over eight months to explore how demographics, attitudes, and contextual factors influenced their use of the G-GIS. The results indicate that family and occupation demographics play key roles in determining adult gamers' gaming habits and their self-control under G-GIS intervention. We also revealed that adult gamers' attitudes and contextual factors can facilitate or hinder the effectiveness of the G-GIS. The findings of this study extend our understanding of heavy adult gamers and reveal how a G-GIS influences adult gamers at the individual level, which can be applied in the design of future game intervention systems, particularly for adult gamers.