Minimax Search Research Articles

A program for Progressive chess

In Progressive chess, rather than just making one move per turn, players play progressively longer series of moves. Combinatorial complexity generated by many sequential moves represents a difficult challenge for classic search algorithms. In this article, we present the design of a state-of-the-art program for Progressive chess. The program follows the generally recommended strategy for this game, which consists of three phases: looking for possibilities to checkmate the opponent, playing sequences of generally good moves when checkmate is not available, and preventing checkmates from the opponent. For efficient and effective checkmate search we considered two versions of the A* algorithm, and developed five different heuristics for guiding the search. For finding promising sequences of moves we developed another set of heuristics, and combined the A* algorithm with minimax search, in order to fight the combinatorial complexity. We constructed an opening book, and designed specialized heuristics for playing Progressive chess endgames. An application with a graphical user interface was implemented in order to enable human players to play Progressive chess against the computer, and to use the computer to analyze their games. The program performed excellently in experiments with checkmate search, and won both mini-matches against a human chess master. We also present the findings of self-play experiments between different versions of the program.

A new expected-improvement algorithm for continuous minimax optimization

Worst-case design is important whenever robustness to adverse environmental conditions must be ensured regardless of their probability. It leads to minimax optimization, which is most often treated assuming that prior knowledge makes the worst environmental conditions obvious, or that a closed-form expression for the performance index is available. This paper considers the important situation where none of these assumptions is true and where the performance index must be evaluated via costly numerical simulations. Strategies to limit the number of these evaluations are then of paramount importance. One such strategy is proposed here, which further improves the performance of an algorithm recently presented that combines a relaxation procedure for minimax search with the well-known Kriging-based EGO algorithm. Expected Improvement is computed in the minimax optimization context, which allows to further reduce the number of costly evaluations of the performance index. The interest of the approach is demonstrated on test cases and a simple engineering problem from the literature, which facilitates comparison with alternative approaches.

MCTS-Minimax Hybrids

Monte Carlo tree search (MCTS) is a sampling-based search algorithm that is state of the art in a variety of games. In many domains, its Monte Carlo rollouts of entire games give it a strategic advantage over traditional depth-limited minimax search with αβ pruning. These rollouts can often detect long-term consequences of moves, freeing the programmer from having to capture these consequences in a heuristic evaluation function. But due to its highly selective tree, MCTS runs a higher risk than full-width minimax search of missing individual moves and falling into traps in tactical situations. This paper proposes MCTS-minimax hybrids that integrate shallow minimax searches into the MCTS framework. Three approaches are outlined, using minimax in the selection/expansion phase, the rollout phase, and the backpropagation phase of MCTS. Without assuming domain knowledge in the form of evaluation functions, these hybrid algorithms are a first step towards combining the strategic strength of MCTS and the tactical strength of minimax. We investigate their effectiveness in the test domains of Connect-4, Breakthrough, Othello, and Catch the Lion, and relate this performance to the tacticality of the domains.

Pruning Game Tree by Rollouts

In this paper we show that the alpha-beta algorithm and its successor MT-SSS*, as two classic minimax search algorithms, can be implemented as rollout algorithms, a generic algorithmic paradigm widely used in many domains. Specifically, we define a family of rollout algorithms, in which the rollout policy is restricted to select successor nodes only from a certain subset of the children list. We show that any rollout policy in this family (either deterministic or randomized) is guaranteed to evaluate the game tree correctly with a finite number of rollouts. Moreover, we identify simple rollout policies in this family that ``implement'' alpha-beta and MT-SSS*. Specifically, given any game tree, the rollout algorithms with these particular policies always visit the same set of leaf nodes in the same order with alpha-beta and MT-SSS*, respectively. Our results suggest that traditional pruning techniques and the recent Monte Carlo Tree Search algorithms, as two competing approaches for game tree evaluation, may be unified under the rollout paradigm.

LS-VisionDraughts: improving the performance of an agent for checkers by integrating computational intelligence, reinforcement learning and a powerful search method

This paper presents LS-VisionDraughts: an efficient unsupervised evolutionary learning system for Checkers whose contribution is to automate the process of selecting an appropriate representation for the board states --- by means of Evolutionary Computation --- keeping a deep look-ahead (search depth) at the moment of choosing an adequate move. It corresponds to a player Multi Layer Perceptron Neural Network whose weights are updated through an evaluation function that is automatically adjusted by means of the Temporal Difference methods. A Genetic Algorithm automatically chooses a concise and efficient set of functions, which describe various scenarios associated with Checkers --- called features --- to represent the board states in the input layer of the Neural Network. It means that each individual of the Genetic Algorithm is a candidate set of features that is associated to a distinct Multi Layer Perceptron Neural Network. The output layer of the Neural Network is a real number (prediction) that indicates to which extent the input state is favorable to provide a better agent performance. In LS-VisionDraughts, a particular version of the search algorithm Alpha-Beta, called fail-soft Alpha-Beta, combined with Table Transposition, Iterative Deepening and ordered tree, uses this prediction value to choose the best move corresponding to the current board state. The best individual is chosen by means of numerous tournaments involving these selfsame Neural Networks. The architecture of LS-VisionDraught is inspired on the agent NeuroDraughts. However, the former system enhances the performance of the latter by automating the selection of the features through Evolutionary Computation and by replacing its Minimax search algorithm with the improved search strategy resumed above. This procedure allows for a 95 % reduction in the search runtime. Further, it remarkably increases the search tree depth. The results obtained from evaluative tournaments confirm the advances of LS-VisionDraughts compared to its opponents. It is however important to point out that LS-VisionDraughts learns practically without human supervision, contrary to the current automatic world champion Chinook, which has been built in a strongly supervised manner.

Large-Scale Optimization for Evaluation Functions with Minimax Search

This paper presents a new method, Minimax Tree Optimization (MMTO), to learn a heuristic evaluation function of a practical alpha-beta search program. The evaluation function may be a linear or non-linear combination of weighted features, and the weights are the parameters to be optimized. To control the search results so that the move decisions agree with the game records of human experts, a well-modeled objective function to be minimized is designed. Moreover, a numerical iterative method is used to nd local minima of the objective function, and more than forty million parameters are adjusted by using a small number of hyper parameters. This method was applied to shogi, a major variant of chess in which the evaluation function must handle a larger state space than in chess. Experimental results show that the large-scale optimization of the evaluation function improves the playing strength of shogi programs, and the new method performs signicantly better than other methods. Implementation of the new method in our shogi program Bonanza made substantial contributions to the program's rst-place nish in the 2013 World Computer Shogi Championship. Additionally, we present preliminary evidence of broader applicability of our method to other two-player games such as chess.

Game playing.

Game playing has been a core domain of artificial intelligence research since the beginnings of the field. Game playing provides clearly defined arenas within which computational approaches can be readily compared to human expertise through head-to-head competition and other benchmarks. Game playing research has identified several simple core algorithms that provide successful foundations, with development focused on the challenges of defeating human experts in specific games. Key developments include minimax search in chess, machine learning from self-play in backgammon, and Monte Carlo tree search in Go. These approaches have generalized successfully to additional games. While computers have surpassed human expertise in a wide variety of games, open challenges remain and research focuses on identifying and developing new successful algorithmic foundations. WIREs Cogn Sci 2014, 5:193-205. doi: 10.1002/wcs.1278 CONFLICT OF INTEREST: The author has declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website.

On Constructing Static Evaluation Function using Temporal Difference Learning

Programming computers to play board games against human players has long been used as a measure for the development of artificial intelligence. The standard approach for computer game playing is to search for the best move from a given game state by using minimax search with static evaluation function. The static evaluation function is critical to the game playing performance but its design often relies on human expert players. This paper discusses how temporal differences (TD) learning can be used to construct a static evaluation function through self-playing and evaluates the effects for various parameter settings. The game of Kalah, a non-chance game of moderate complexity, is chosen as a testbed. The empirical result shows that TD learning is particularly promising for constructing a good evaluation function for the end games and can substantially improve the overall game playing performance in learning the entire game.DOI:Â 10.18495/comengapp.21.175184

Investigating the Performance of Minimax Search and Aggregate Mahalanobis Distance Function in Evolving an Ayo/Awale Player

Investigating the Performance of Minimax Search and Aggregate Mahalanobis Distance Function in Evolving an Ayo/Awale Player

Simulating Multiple Character Interactions with Collaborative and Adversarial Goals

This paper proposes a new methodology for synthesizing animations of multiple characters, allowing them to intelligently compete with one another in dense environments, while still satisfying requirements set by an animator. To achieve these two conflicting objectives simultaneously, our method separately evaluates the competition and collaboration of the interactions, integrating the scores to select an action that maximizes both criteria. We extend the idea of min-max search, normally used for strategic games such as chess. Using our method, animators can efficiently produce scenes of dense character interactions such as those in collective sports or martial arts. The method is especially effective for producing animations along story lines, where the characters must follow multiple objectives, while still accommodating geometric and kinematic constraints from the environment.

Independent-valued minimax: Pathological or beneficial?

Minimax search, which is used by most game-playing programs, is considered pathological when deeper searches produce worse evaluations than shallower ones. This phenomenon was first observed in theoretical analyses under seemingly reasonable conditions. It was most commonly explained by the lack of dependence between nearby positions in the analyses: if nearby positions have similar values, as is typically the case in real games, the pathology no longer occurs. In this paper, we show that the pathology can be eliminated even without position-value dependence, by assigning enough different values to the positions and modeling the heuristic error as normally distributed noise that is independent of the depth in the game tree. This leads to the conclusion that minimax is less prone to the pathology than was previously thought and indicates the importance of the number of different position values.

A New Approach to Gal’s Theory of Search Games on Weakly Eulerian Networks

A network is called weakly Eulerian if it consists of a finite number of disjoint Eulerian networks which are connected in a tree-like fashion. S. Gal and others developed a theory of (zero-sum) hide-and-seek games on such networks. The minimax search time for a network is called its search value. A network is called simply searchable if its search value is half the minimum time to tour it. A celebrated result of Gal is that a network is simply searchable if and only if it is weakly Eulerian. This expository article presents a new approach to the Gal theory, based on ideas borrowed from the author's recent extension of Gal's theory to networks which can be searched at speeds depending on the location and direction in the network. Most of the proofs are new.

Evaluation of Game Tree Search Methods by Game Records

This paper presents a method of evaluating game tree search methods including standard min-max search with heuristic evaluation functions and Monte Carlo tree search, which recently achieved drastic improvements in the strength of Computer Go programs. The basic idea of this paper is to use an averaged win probability of positions having similar evaluation values. Accuracy measures of evaluation values with respect to win probabilities can be used to assess the performance of game tree search methods. A plot of win probabilities against evaluation values should have consistency and monotonicity if the evaluation values are produced by a good game tree search method. By inspecting whether the plot has the properties for some subset of positions, we can detect specific deficiencies in the game tree search method. We applied our method to Go, Shogi, and Chess, and by comparing the results with empirical understanding of the performance of various game tree search methods and with the results of self-plays, we show that our method is efficient and effective.

Searching symmetric networks with Utilitarian‐Postman paths

AbstractWe introduce the notion of a Utilitarian Postman (UP) path on a network Q as one which minimizes the expected time required to find a random (uniformly distributed) point, and show that UP paths must be used in a minimax search of a symmetric network. For any network Q, one may consider the zero‐sum search game Γ(Q) in which the (minimizing) Searcher picks a unit speed path S(t) in Q, the Hider picks a point H in Q, and the payoff is the meeting time T = min{t : S(t) = H}. We show first that if Q is symmetric (edge and vertex transitive), then it is optimal for the Hider to pick H uniformly in Q, so that the Searcher must follow a UP path. We then show that if Q is symmetric of odd degree, with n vertices and m unit length edges, the value V of Γ(Q) satisfies $ V \geq {m \over 2} + {n^{2}-2n \over 8m} $, with equality if and only if (*): Q has a path P = v1, v2,…,vn−1 of distinct vertices, such that the edge set Q′ = Q−∪(v2i,v2i+1) is connected. In this case, there is a UP path for Q consisting of P followed by an Eulerian path E of Q′. The condition (*) is satisfied by many symmetric graphs, including all complete graphs, complete bipartite graphs, hypercube graphs, high valency graphs, and the Petersen graph. We know of no odd degree symmetric graph not satisfying (*). © 2009 Wiley Periodicals, Inc. NETWORKS, 2009

Bots Get Smart

The main challenge in making video games is to make computer-generated characters-dubbed bots-act realistically. They must, of course, look good and move naturally. But, ideally, they should also be able to engage in believable conversations, plan their actions, find their way around virtual worlds, and learn from their mistakes. That is, they need to be smart. Today many video games create only an illusion of intelligence, using a few programming tricks. But in the not-so -distant future, game bots will routinely use sophisticated AI techniques to shape their behavior. We and our colleagues in the University of Alberta GAMES (game-playing, analytical methods, minimax search and empirical studies) research group, in Edmonton, Canada, have been working to help bring about such a revolution.

Applying Genetic Algorithm and Self-Learning on Computer Othello

Abstract Artificial intelligence algorithms have been applied on computer board games since 1953. Among all computer board games, because of its low branching property, Othello can easily defeat humans by designing with min-max search and alpha-beta pruning. Nowadays, the goal of compute Othello is no longer to challenge people but to compete against other computer programs. This paper improves the computer Othello’s opening strategy, mid-game strategy, and end-game strategy. The opening strategy is enhanced by improving self-learning efficiency using pattern recognition. The evaluation function is designed by combining the Line-pattern evaluation with other evaluation functions and by using an improved genetic algorithm to optimize the parameters. Then implement dynamic programming in min-max search and alpha-beta pruning to make the searching engine more efficient and to improve the depth of perfect-searching. Keywords : Genetic Algorithm, Self-Learning, Computer Othello 1. Introduction

Pathology in heuristic search

Practice shows that game-playing programs using minimax search perform better when searching deeper. Mathematical analyses, however, showed the opposite. This paradox was termed minimax pathology. Our real-valued minimax model demonstrates that appropriate modeling of the heuristic error is enough to eliminate the pathology. We examine the conditions under which the pathology appears and explain the mechanism that makes minimax otherwise beneficial. The reasons for the pathology in single-agent search are also addressed.

Artificial Intelligence on Computer Based Chess Game: An Implementation of Alpha-Beta-Cutoff Search Method

AbstractA chess program usually consists of three main parts, that is, a move generator to generate all legal moves, an evaluation function to evaluate each move, and a search function to select the best move. The search function is the core of thinking process. The goal of this research is to implement the alpha beta cutoff as a search method. This method is derived from minimax search method and is more optimal than the minimax search method.In minimax, all nodes is searched and compared one by one to get the best value. On the other hand, the alpha beta cut of methd only searches nodes which make contribution to the previous value and cuts off nodes which are not useful. It means that the alpha beta method will not search and compare all nodes. The new node will be better than the previous one and replace the old value with the new one. This will make the alpha beta method requires smaller search time.The proposed method is tested by doing a series of matches between humans and a computer. The results show that the computer has ability to think well and performs a good artcial intelligence though it is very open to be modified and more optimized.Keywords: move generator function, evaluation function, search function, minimax, alpha beta cutoff

AN EFFICIENT HEURISTIC FOR EVOLVING AN AGENT IN THE STRATEGY GAME OF AYO

The game of Ayo is a member of the Mancala family. The problem of developing heuristics for playing Mancala games by a computer agent is an open issue. This note presents a heuristic that uses a hybrid combination of minimax search and case-based rea

A Study of Artificial Neural Network Architectures for Othello Evaluation Functions

In this study, we use temporal difference learning (TDL) to investigate the ability of 20 different artificial neural network (ANN) architectures to learn othello game board evaluation functions. The ANN evaluation functions are applied to create a strong othello player using only 1-ply search. In addition to comparing many of the ANN architectures seen in the literature, we introduce several new architectures that consider the game board symmetry. Both embedding the game board symmetry into the network architecture through weight sharing and the outright removal of symmetry through symmetry removal are explored. Experiments varying the number of inputs per game board square from one to three, the number of hidden nodes, and number of hidden layers are also performed. We found it advantageous to consider game board symmetry in the form of symmetry by weight sharing; and that an input encoding of three inputs per square outperformed the one input per square encoding that is commonly seen in the literature. Furthermore, architectures with only one hidden layer were strongly outperformed by architectures with multiple hidden layers. A standard weighted-square board heuristic evaluation function from the literature was used to evaluate the quality of the trained ANN othello players. One of the ANN architectures introduced in this study, an ANN implementing weight sharing and consisting of three hidden layers, using only a 1-ply search, outperformed a weighted-square test heuristic player using a 6-ply minimax search.