Countering Evolutionary Forgetting in No-Limit Texas Hold’em Poker Agents

Abstract

No-Limit Texas Hold’em Poker is a stochastic game of imperfect information. Each player receives cards dealt randomly and does not know which cards his opponents have been dealt. These simple features result in No-Limit Texas Hold’em Poker having a large decision space in comparison to other classic games such as Backgammon and Chess. Evolutionary algorithms and neural networks have been shown to find solutions in large and non-linear decision spaces and have proven to aid decision making in No-Limit Texas Hold’em Poker. In this paper, a hybrid method known as evolving neural networks is used by No-Limit Texas Hold’em Poker playing agents to make betting decisions. When selecting a new generation of agents, evolutionary forgetting can result in selecting an agent with betting behaviour that has previously been shown to be inferior. To prevent this from occurring, we utilize two heuristics: halls of fame and co-evolution. In addition, we evaluate agent fitness using three fitness functions based upon, respectively, the length of time an agent survives in a tournament, the number of hands won in a tournament, and the average amount of money won across all hands in a tournament. Results show that the length of time an agent survives is indeed an appropriate measure of fitness. Results also show that utilizing halls of fame and co-evolution serve to further improve the fitness of agents. Finally, through monitoring the evolutionary progress of agents, we find that the skill level of agents improves when using our evolutionary heuristics.