General Game Playing Research Articles

Fast and Knowledge-Free Deep Learning for General Game Playing (Student Abstract)

We develop a method of adapting the AlphaZero model to General Game Playing (GGP) that focuses on faster model generation and requires less knowledge to be extracted from the game rules. The dataset generation uses MCTS playing instead of self-play; only the value network is used, and attention layers replace the convolutional ones. This allows us to abandon any assumptions about the action space and board topology. We implement the method within the Regular Boardgames GGP system and show that we can build models outperforming the UCT baseline for most games efficiently.

Modern General Game Playing with Reinforcement Learning

In computational science and robotics, game playing and artificial intelligence (AI) are research areas that have long been studied. RL systems represent a major step towards autonomous systems that comprehend the visual world at an incredibly deep level and are poised to revolutionize the field of artificial intelligence (AI). Game Play is the testbench where environmental variables can be evaluated to generate adaptive, intelligent, or responsive behavior or simulate real-world scenarios. In this survey, we will explore general gameplay (GGP) with reinforcement learning and its various applications. This survey will cover central algorithms in deep RL, including the deep Qnetwork (DQN) and general mathematical and pragmatic approaches taken in the field. The objective of the current review is to examine the history, associations, and recent advances in the field of general game playing with reinforcement learning and its subfields. We conclude by describing several current areas of research within this field.

The 2022 Ludii AI competition

The Ludii AI Competition involves general game playing events focused on developing agents that can play a wide variety of board games. In the 2022 edition, three competition tracks were proposed: Kilothon, General Game Playing, and Learning. All tracks used the Ludii general game system to provide the necessary games and API. This paper reports the motivation, context, and results of the 2022 Ludii AI Competition.

Algorithm structure for real-time general game playing agents

General Game Playing is an examination of AI agents that are designed without any explicit knowledge of a particular game. Instead, they are required to play a variety of games that they have never seen before by merely reading the description of the rules at runtime, without any intervention from a human being. Previous successful agents have concentrated their efforts on turn-based games, employing either generic heuristics or the Monte Carlo/UCT simulation technique. Real-time games have dominated the market in recent years, and as a result, agents that have more reflexibility have emerged as an important area of research. In this paper, the author evaluateswhether or not it is possible to implement the already-existing algorithms into a real-time playing GGP agent, and we present a potential algorithm structure that, on the basis of this evaluation, could be able to solve the challenge.

Automated video game parameter tuning with XVGDL+

Usually, human participation is required in order to provide feedback during the game tuning or balancing process. Moreover, this is commonly an iterative process in which play-testing is required as well as human interaction for gathering all important information to improve and tune the game components’ specification. In this paper, a mechanism is proposed to accelerate this process and reduce significantly the costs of it, contributing with a solution to perform the game parameter tuning and game balancing using search algorithms and artificial intelligence (AI) techniques. The process is executed in a fully automated way, and just requires a game specification written in a particular video game description language. Automated play-testing, and game’s feedback information analysis, are related to perform game parameters’ tuning and balancing, leading to offer a solution for the problem of optimizing a video game specification. Recently, XVGDL, a new language for specifying video games which is based on the eXtensible Markup Language (XML), has been presented. This paper uses XVGDL+, an extension of this lan- guage that incorporates new components to specify, within the video game specification, desirable goals or requirements to be evaluated after each game execution. A prototypical implementation of a Game Engine (termed XGE+) was also presented. This game engine not only enables the execution of an XVGDL+ game specification but also provides feedback information once the game has finished. The paper demonstrates that the combination of XVGDL+ with XGE+ offers a powerful mechanism for helping solving game AI research problems, in this case, the game tuning of video game parameters, with respect to initial optimization goals. These goals, as one of the particularities of the proposal presented here, are included within the game specification, minimizing the input of the process. As a practical proof of it, two experiments have been conducted to optimize a game specification written in XVGDL via a hill climbing local search method, in a fully automated way.

Split Moves for Monte-Carlo Tree Search

In many games, moves consist of several decisions made by the player. These decisions can be viewed as separate moves, which is already a common practice in multi-action games for efficiency reasons. Such division of a player move into a sequence of simpler / lower level moves is called splitting. So far, split moves have been applied only in forementioned straightforward cases, and furthermore, there was almost no study revealing its impact on agents' playing strength. Taking the knowledge-free perspective, we aim to answer how to effectively use split moves within Monte-Carlo Tree Search (MCTS) and what is the practical impact of split design on agents' strength. This paper proposes a generalization of MCTS that works with arbitrarily split moves. We design several variations of the algorithm and try to measure the impact of split moves separately on efficiency, quality of MCTS, simulations, and action-based heuristics. The tests are carried out on a set of board games and performed using the Regular Boardgames General Game Playing formalism, where split strategies of different granularity can be automatically derived based on an abstract description of the game. The results give an overview of the behavior of agents using split design in different ways. We conclude that split design can be greatly beneficial for single- as well as multi-action games.

Analysis of the Impact of Randomization of Search-Control Parameters in Monte-Carlo Tree Search

Monte-Carlo Tree Search (MCTS) has been applied successfully in many domains, including games. However, its performance is not uniform on all domains, and it also depends on how parameters that control the search are set. Parameter values that are optimal for a task might be sub-optimal for another. In a domain that tackles many games with different characteristics, like general game playing (GGP), selecting appropriate parameter settings is not a trivial task. Games are unknown to the player, thus, finding optimal parameters for a given game in advance is not feasible. Previous work has looked into tuning parameter values online, while the game is being played, showing some promising results. This tuning approach looks for optimal parameter values, balancing exploitation of values that performed well so far in the search and exploration of less sampled values. Continuously changing parameter values while performing the search, combined also with exploration of multiple values, introduces some randomization in the process. In addition, previous research indicates that adding randomization to certain components of MCTS might increase the diversification of the search and improve the performance. Therefore, this article investigates the effect of randomly selecting values for MCTS search-control parameters online among predefined sets of reasonable values. For the GGP domain, this article evaluates four different online parameter randomization strategies by comparing them with other methods to set parameter values: online parameter tuning, offline parameter tuning and sub-optimal parameter choices. Results on a set of 14 heterogeneous abstract games show that randomizing parameter values before each simulation has a positive effect on the search in some of the tested games, with respect to using fixed offline-tuned parameters. Moreover, results show a clear distinction between games for which online parameter tuning works best and games for which online randomization works best. In addition, the overall performance of online parameter randomization is closer to the one of online parameter turning than the one of sub-optimal parameter values, showing that online randomization is a reasonable parameter selection strategy. When analyzing the structure of the search trees generated by agents that use the different parameters selection strategies, it is clear that randomization causes MCTS to become more explorative, which is helpful for alignment games that present many winning paths in their trees. Online parameter tuning, instead, seems more suitable for games that present narrow winning paths and many losing paths.

Automatic Design of Balanced Board Games

AI techniques are already widely used in game software to provide computer-controlled opponents for human players. However, game design is a more-challenging problem than game play. Designers typically expend great effort to ensure that their games are balanced and challenging. Dynamic game-balancing techniques have been developed to modify a game-engine’s parameters in response to user play. In this paper we describe a first attempt at using AI techniques to design balanced board games like checkers and Go by modifying the rules of the game, not just the rule parameters. Our approach involves the use of a commercial general game-playing (GGP) engine that plays according to rules that are specified in a general game-definition language. We use a genetic algorithm (GA) to search the space of game rules, looking for turn-based board games that are well balanced, i.e., those that the GGP engine in self-play finds equally hard to win from either side and rarely draws. The GA finds better games than a random-search strategy that uses equivalent computational effort.

HyperPlay: A Solution to General Game Playing with Imperfect Information

General Game Playing is the design of AI systems able to understand the rules of new games and to use such descriptions to play those games effectively. Games with imperfectinformation have recently been added as a new challenge forexisting general game-playing systems. The HyperPlay technique presents a solution to this challenge by maintaining a collection of models of the true game as a foundation for reasoning, and move selection. The technique provides existing game players with a bolt-on solution to convert from perfect-information games to imperfect-information games. In this paper we describe the HyperPlay technique, show how it was adapted for use with a Monte Carlo decision making process and give experimental results for its performance.

Generalized Monte-Carlo Tree Search Extensions for General Game Playing

General Game Playing (GGP) agents must be capable of playing a wide variety of games skillfully. Monte-Carlo Tree Search (MCTS) has proven an effective reasoning mechanism for this challenge, as is reflected by its popularity among designers of GGP agents. Providing GGP agents with the knowledge relevant to the game at hand in real time is, however, a challenging task. In this paper we propose two enhancements for MCTS in the context of GGP, aimed at improving the effectiveness of the simulations in real time based on in-game statistical feedback. The first extension allows early termination of lengthy and uninformative simulations while the second improves the action-selection strategy when both explored and unexplored actions are available. The methods are empirically evaluated in a state-of-the-art GGP agent and shown to yield an overall significant improvement in playing strength.

Research Summary

Monte-Carlo Tree Search (MCTS) is an online planning algorithm that combines the ideas of best-first tree search and Monte-Carlo evaluation. Since MCTS is based on sampling, it does not require a transition function in explicit form, but only a generative model of the domain. Because it grows a highly selective search tree guided by its samples, it can handle huge search spaces with large branching factors. By using Monte-Carlo playouts, MCTS can take long-term rewards into account even with distant horizons. Combined with multi-armed bandit algorithms to trade off exploration and exploitation, MCTS has been shown to guarantee asymptotic convergence to the optimal policy, while providing approximations when stopped at any time. The relatively new MCTS approach has started a revolution in computer Go. Furthermore, it has achieved considerable success in domains as diverse as the games of Hex, Amazons, LOA, and Ms. Pacman; in General Game Playing, planning, and optimization. Whereas the focus of previous MCTS research has been on the practical application, current research begins to address the problem of understanding the nature, the underlying principles, of MCTS. A careful understanding of MCTS will lead to more effective search algorithms. Hence, my two interrelated research questions are: How can we formulate models that increase our understanding of how MCTS works? and How can we use the developed understanding to create effective search algorithms? This research summary describes the first steps I undertook in these directions, as well as my plans for future work.

Towards Generating Novel Games Using Conceptual Blending

We sketch the process of creating a novel video game by blending two video games specified in the Video Game Description Language (VGDL), following the COINVENT computational model of conceptual blending. We highlight the choices that need to be made in this process, and discuss the prospects for a computational game designer based on blending.

Exception-Tolerant Hierarchical Knowledge Bases for Forward Model Learning

This article provides an overview of the recently proposed forward model approximation framework for learning games of the general video game artificial intelligence (GVGAI) framework. In contrast to other general game-playing algorithms, the proposed agent model does not need a full description of the game but can learn the game's rules by observing game state transitions. Based on hierarchical knowledge bases, the forward model can be learned and revised during game-play, improving the accuracy of the agent's state predictions over time. This allows the application of simulation-based search algorithms and belief revision techniques to previously unknown settings. We show that the proposed framework is able to quickly learn a model for dynamic environments in the context of the GVGAI framework.

Self-Adaptive Monte Carlo Tree Search in General Game Playing

Many enhancements for Monte Carlo tree search (MCTS) have been applied successfully in general game playing (GGP). MCTS and its enhancements are controlled by multiple parameters that require extensive and time-consuming offline optimization. Moreover, as the played games are unknown in advance, offline optimization cannot tune parameters specifically for single games. This paper proposes a self-adaptive MCTS strategy (SA-MCTS) that integrates within the search a method to automatically tune search-control parameters online per game. It presents five different allocation strategies that decide how to allocate available samples to evaluate parameter values. Experiments with $\boldsymbol {1}$ s play-clock on multiplayer games show that for all the allocation strategies the performance of SA-MCTS that tunes two parameters is at least equal to or better than the performance of MCTS tuned offline and not optimized per-game. The allocation strategy that performs the best is N-Tuple Bandit Evolutionary Algorithm (NTBEA). This strategy also achieves a good performance when tuning four parameters. SA-MCTS can be considered as a successful strategy for domains that require parameter tuning for every single problem, and it is also a valid alternative for domains where offline parameter tuning is costly or infeasible.

Classification of Video Game Player Experience Using Consumer-Grade Electroencephalography

A growing body of literature has emerged that demonstrates the potential of neurogaming platforms for inter- facing with well-known video games. With the recent convergence of advances in consumer electronics, ubiquitous computing, and wearable sensor technologies real-time monitoring of neurocognitive and affective states can be studied in an objective manner. Whilst establishing the optimal relation among frequency bands, task engagement, and arousal states is a goal of neurogaming, a standardized method has yet to be established. Herein we aimed to test classifiers within the same context, group of participants, feature extraction methods, and protocol. Given the emphasis upon neurogaming, a commercial-grade electroencephalographic (EEG; Emotiv EPOC) headset was used to collect signals from 30 participants. The EEG data was then filtered to get separate frequency bands to train cognitive-affective classifiers with three classification techniques: Support Vector Machines (SVM), Naive Bayes (NB), and k-Nearest Neighbors (kNN). Results revealed that the NB classifier was the most robust classifier for identifying negative (e.g., character death) game-based events. The identification of general gameplay events is best identified using kNN and the Beta band. Results from this study suggest that a combination of classifiers is preferable over selection of a single classifier.

Deep Reinforcement Learning for General Game Playing

General Game Playing agents are required to play games they have never seen before simply by looking at a formal description of the rules of the game at runtime. Previous successful agents have been based on search with generic heuristics, with almost no work done into using machine learning. Recent advances in deep reinforcement learning have shown it to be successful in some two-player zero-sum board games such as Chess and Go. This work applies deep reinforcement learning to General Game Playing, extending the AlphaZero algorithm and finds that it can provide competitive results.

Introduction to Logic Programming

Logic Programming is a style of programming in which programs take the form of sets of sentences in the language of Symbolic Logic. Over the years, there has been growing interest in Logic Programming due to applications in deductive databases, automated worksheets, Enterprise Management (business rules), Computational Law, and General Game Playing. book introduces Logic Programming theory, current technology, and popular applications. this volume, we take an innovative, model-theoretic approach to logic programming. We begin with the fundamental notion of datasets, i.e., sets of ground atoms. Given this fundamental notion, we introduce views, i.e., virtual relations; and we define classical logic programs as sets of view definitions, written using traditional Prolog-like notation but with semantics given in terms of datasets rather than implementation. We then introduce actions, i.e., additions and deletions of ground atoms; and we define dynamic logic programs as sets of action definitions. addition to the printed book, there is an online version of the text with an interpreter and a compiler for the language used in the text and an integrated development environment for use in developing and deploying practical logic programs. This is a book for the 21st century: presenting an elegant and innovative perspective on logic programming. Unlike other texts, it takes datasets as a fundamental notion, thereby bridging the gap between programming languages and knowledge representation languages; and it treats updates on an equal footing with datasets, leading to a sound and practical treatment of action and change. – Bob Kowalski, Professor Emeritus, Imperial College London In a world where Deep Learning and Python are the talk of the day, this book is a remarkable development. It introduces the reader to the fundamentals of traditional Logic Programming and makes clear the benefits of using the technology to create runnable specifications for complex systems. – Son Cao Tran, Professor in Computer Science, New Mexico State University Excellent introduction to the fundamentals of Logic Programming. The book is well-written and well-structured. Concepts are explained clearly and the gradually increasing complexity of exercises makes it so that one can understand easy notions quickly before moving on to more difficult ideas. – George Younger, student, Stanford University

XML-Based Video Game Description Language

This paper presents the XML-based Video Game Description Language (XVGDL), a new language for specifying Video games which is based on the Extensible Markup Language (XML). The proposal is portable and extensible, and allows games to not only be defined at engine level but also includes specific features that can lead the game design process whilst simultaneously reducing the gap between game specification and its corresponding game implementation. XVGDL is as generic as possible, making it possible to describe different genres of games. This paper focuses on presenting the basis of the language. The paper describes the syntax as well as the components of XVGDL, and provides examples of their use. Defining games via XML structures provides all the advantages of the management of XML files and opens up interesting lines of research. Our proposal provides a number of novel features. So, XVGDL game definitions can be managed as any other XML file, which means that it can be automatically handled by any XML file management software. Another interesting feature is that XVGDL can specify game components (e.g., game Artificial), in-game processes (e.g., the procedural generation of maps) or in-game events (e.g., the checking of the conditions to end a game match) via the association with external (possibly non-XML) files. Moreover, XVGDL files can be easily validated as any XML file what means that validations against a particular Document Type Definition (DTD) or XML Schema Definition (XSD) are possible. In addition, the paper presents a first prototype implementation of a (text-based) interpreter that allows XVGDL game specifications as a playable game to be executed. This tool not only validates our proposal but also represents a first step towards smoothing the path to obtaining an executable version of a game from its game specification.

General Game Playing with Imperfect Information

General Game Playing is a field which allows the researcher to investigate techniques that might eventually be used in an agent capable of Artificial General Intelligence. Game playing presents a controlled environment in which to evaluate AI techniques, and so we have seen an increase in interest in this field of research. Games of imperfect information offer the researcher an additional challenge in terms of complexity over games with perfect information. In this article, we look at imperfect-information games: their expression, their complexity, and the additional demands of their players. We consider the problems of working with imperfect information and introduce a technique called HyperPlay, for efficiently sampling very large information sets, and present a formalism together with pseudo code so that others may implement it. We examine the design choices for the technique, show its soundness and completeness then provide some experimental results and demonstrate the use of the technique in a variety of imperfect-information games, revealing its strengths, weaknesses, and its efficiency against randomly generating samples. Improving the technique, we present HyperPlay-II, capable of correctly valuing information-gathering moves. Again, we provide some experimental results and demonstrate the use of the new technique revealing its strengths, weaknesses and its limitations.

Inductive general game playing

General game playing (GGP) is a framework for evaluating an agent’s general intelligence across a wide range of tasks. In the GGP competition, an agent is given the rules of a game (described as a logic program) that it has never seen before. The task is for the agent to play the game, thus generating game traces. The winner of the GGP competition is the agent that gets the best total score over all the games. In this paper, we invert this task: a learner is given game traces and the task is to learn the rules that could produce the traces. This problem is central to inductive general game playing (IGGP). We introduce a technique that automatically generates IGGP tasks from GGP games. We introduce an IGGP dataset which contains traces from 50 diverse games, such as Sudoku, Sokoban, and Checkers. We claim that IGGP is difficult for existing inductive logic programming (ILP) approaches. To support this claim, we evaluate existing ILP systems on our dataset. Our empirical results show that most of the games cannot be correctly learned by existing systems. The best performing system solves only 40% of the tasks perfectly. Our results suggest that IGGP poses many challenges to existing approaches. Furthermore, because we can automatically generate IGGP tasks from GGP games, our dataset will continue to grow with the GGP competition, as new games are added every year. We therefore think that the IGGP problem and dataset will be valuable for motivating and evaluating future research.