General Video Game Artificial Intelligence Research Articles

Playtesting: What is Beyond Personas

We present two approaches to improve automated playtesting. First, we propose developing persona, which allows a persona to progress to different goals. In contrast, the procedural persona is fixed to a single goal. Second, a human playtester knows which paths she has tested before, and during the consequent tests, she may test different paths. However, Reinforcement Learning (RL) agents disregard these previous paths. We propose a novel methodology that we refer to as Alternative Path Finder (APF). We train APF with previous paths and employ APF during the training of a RL agent. APF modulates the reward structure of the environment while preserving the agent's goal. When evaluated, the agent generates a different trajectory that achieves the same goal. We use the General Video Game Artificial Intelligence and VizDoom frameworks to test our proposed methodologies. We use Proximal Policy Optimization RL agent during experiments. First, we compare the playtest data generated by developing and procedural persona. Our experiments show that developing persona provides better insight into the game and how different players would play. Second, we present the alternative paths found using APF and argue why traditional RL agents cannot learn those paths.

Rolling Horizon Evolutionary Algorithms for General Video Game Playing

Game-playing evolutionary algorithms, specifically rolling horizon evolutionary algorithms (RHEA), have recently managed to beat the state of the art in win rate across many video games. However, the best results in a game are highly dependent on the specific configuration of modifications introduced over several papers, each adding additional parameters to the core algorithm. Furthermore, the best previously published parameters have been found from only a few human-picked combinations, as the possibility space has grown beyond exhaustive search. This article presents the state of the art in RHEA, combining all modifications described in the literature, as well as new ones. We then use a parameter optimizer, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -tuple bandit evolutionary algorithm, to find the best combination of parameters in 20 games from the general video game Artificial Intelligence (AI) framework. Furthermore, we analyze the algorithm’s parameters and some interesting combinations revealed through the optimization process. Finally, we find new state of the art solutions on several games by automatically exploring the large parameter space of RHEA.

Automated Video Game Testing Using Synthetic and Humanlike Agents

In this article, we present a new methodology that employs tester agents to automate video game testing. We introduce two types of agents - synthetic and humanlike - and two distinct approaches to create them. Our agents are derived from Sarsa and Monte Carlo tree search (MCTS) but focus on finding defects, while traditional game-playing agents focus on maximizing game scores. The synthetic agent uses test goals generated from game scenarios, and these goals are further modified to examine the effects of unintended game transitions. The humanlike agent uses test goals extracted by our proposed multiple greedy-policy inverse reinforcement learning (MGP-IRL) algorithm from tester trajectories. MGP-IRL captures multiple policies executed by human testers. We use our agents to produce test sequences, and run the game with these sequences. At each run, we use an automated test oracle to check for bugs. We analyze the proposed method in two parts-we compare the success of humanlike and synthetic agents in bug finding, and we evaluate the similarity between humanlike agents and human testers. We collected 427 trajectories from human testers using the General Video Game Artificial Intelligence (GVG-AI) framework and created three games with 12 levels that contain 45 bugs. Our experiments reveal that humanlike and synthetic agents compete with human testers' bug finding performances. Moreover, we show that MGP-IRL increases the humanlikeness of agents while improving the bug finding performance.

Co-Generation of Game Levels and Game-Playing Agents

Open-endedness, a longstanding cornerstone of artificial life research, is the ability of systems to generate potentially unbounded ontologies of increasing novelty and complexity. Engineering generative systems displaying at least some degree of this ability is a goal with clear applications to procedural content generation in games. The Paired Open-Ended Trailblazer (POET) algorithm, heretofore explored only in a biped walking domain, is a coevolutionary system that simultaneously generates environments and agents that can solve them. This paper introduces a POET-Inspired Neuroevolutionary System for KreativitY (PINSKY) in games, which co-generates levels for multiple video games and agents that play them. This system leverages the General Video Game Artificial Intelligence (GVGAI) framework to enable co-generation of levels and agents for the 2D Atari-style games Zelda and Solar Fox. Results demonstrate the ability of PINSKY to generate curricula of game levels, opening up a promising new avenue for research at the intersection of procedural content generation and artificial life. At the same time, results in these challenging game domains highlight the limitations of the current algorithm and opportunities for improvement.

Exception-Tolerant Hierarchical Knowledge Bases for Forward Model Learning

This article provides an overview of the recently proposed forward model approximation framework for learning games of the general video game artificial intelligence (GVGAI) framework. In contrast to other general game-playing algorithms, the proposed agent model does not need a full description of the game but can learn the game's rules by observing game state transitions. Based on hierarchical knowledge bases, the forward model can be learned and revised during game-play, improving the accuracy of the agent's state predictions over time. This allows the application of simulation-based search algorithms and belief revision techniques to previously unknown settings. We show that the proposed framework is able to quickly learn a model for dynamic environments in the context of the GVGAI framework.

General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms

General video game playing aims at designing an agent that is capable of playing multiple video games with no human intervention. In 2014, the General Video Game Artificial Intelligence (GVGAI) competition framework was created and released with the purpose of providing researchers a common open-source and easy-to-use platform for testing their artificial intelligence (AI) methods with potentially infinity of games created using the video game description language (VGDL). The framework has been expanded into several tracks during the last few years to meet the demands of different research directions. The agents are required either to play multiple unknown games with or without access to game simulations, or to design new game levels or rules. This survey paper presents the VGDL, the GVGAI framework, existing tracks, and reviews the wide use of GVGAI framework in research, education, and competitions five years after its birth. A future plan of framework improvements is also described.

Fun and games at IEEE WCCI 2016, Vancouver, Canada

Following UCL spin-out DeepMind's success at beating the world Go champion, there was very much a flavour of artificial intelligence (AI) in the air. For example Deep Learning was the topic of Juergen Schmidhuber's invited plenary talk and some of the competitions, for example, Diego Perez and Simon Lucas' General Video Game AI Competition (winner Tom Vodopivec shown in Figure 9). David Fogel president of Natural Selection Inc., gave an impressive lecture open to the general public (see Figure 1). He covered the story of how he and Kumar Chellapilla evolved a competition level checkers (draughts) player. He talked about taking it out into the real world to play people in online competitions. He related the difference in his opponents online behaviour when playing as David 1101 or when playing as Blondie242.

일반 비디오 게임 플레이 인공지능을 위한 GreedyUCB1기반 몬테카를로 트리 탐색

Generally, the existing Artificial Intelligence (AI) systems were designed for specific purposes and their capabilities handle only specific problems. Alternatively, Artificial General Intelligence can solve new problems as well as those that are already known. Recently, General Video Game Playing the game AI version of General Artificial Intelligence, has garnered a large amount of interest among Game Artificial Intelligence communities. Although video games are the sole concern, the design of a single AI that is capable of playing various video games is not an easy process. In this paper, we propose a GreedyUCB1 algorithm and rollout method that were formulated using the knowledge from a game analysis for the Monte-Carlo Tree Search game AI. An AI that used our method was ranked fourth at the GVG-AI (General Video Game-Artificial Intelligence) competition of the IEEE international conference of CIG (Computational Intelligence in Games) 2014.