Artificial Intelligence Player Research Articles

Implementing the AlphaZero algorithm for Connect Four: A deep reinforcement learning approach

The realm of board games presents a challenging domain for the application of artificial intelligence (AI), given their vast state-action space and inherent complexity. This paper explores the development of a proficient AI for Connect Four using DeepMind's AlphaZero algorithm. The algorithm employs a policy-value network for concurrent prediction of action probabilities and state values, and Monte Carlo Tree Search (MCTS) for decision-making, guided by the policy-value network. Through extensive self-play and data augmentation, our AI learns without the need for explicit prior knowledge. Our experiment demonstrated that the AI player showed significant capability in playing Connect Four, exhibiting strategic decision-making that sometimes-surpassed human performance. These results underline the potential of deep reinforcement learning in advancing AI performance in complex board games.

Artificial Intelligence Gamers Based on Deep Reinforcement Learning

This study investigates the design and implementation of Artificial Intelligence (AI) game players based on deep reinforcement learning, offering a novel approach to autonomous decision-making and strategy acquisition in intelligent games. Initially, the fundamental principles and algorithms of deep reinforcement learning are introduced, along with the fusion of deep learning and reinforcement learning. Subsequently, existing research is reviewed, and the pros and cons of current methodologies are examined, highlighting the underlying issues and challenges. The utilization of AI players in mainstream games is then introduced, and the influence of AI players on contemporary games is analyzed. Through this analysis of AI players in mainstream games, the strengths and weaknesses of current AI players are identified, and recommendations for optimizing them are provided. This study holds significant implications for guiding the design and development of intelligent game players, while also enriching the application of deep reinforcement learning within the gaming domain.

A Formal Representation for Intelligent Decision-Making in Games

The study of intelligent game-playing has gained tremendous attention in the past few decades. The recent development of artificial intelligence (AI) players (e.g., the Go player AlphaGo) has made intelligent game-playing even more prominent in both academia and industry. The performance of state-of-the-art AI players benefits greatly from machine learning techniques, based on which, players can make estimations and decisions even without understanding the games. Although AI machines show great superiority over humans in terms of data processing and complex computation, there remains a vast distance between artificial intelligence and human intelligence with respect to the abilities of context understanding and reasoning. In this paper, we explore the theoretical foundation of intelligent game-playing from a logical perspective. The proposed logic, by considering the computational limits in practical game-playing, drops the ideal assumptions in existing logics for the classical game model. We show that under logical framework, the basis of decision-making for agents in game scenarios can be formally represented and analyzed. Moreover, by characterizing the solutions of games, this logic is able to formalize players’ rational decision-making during practical game-playing.

Boosting human Insight by Cooperative AI: Foundations of Shannon-Neumann Logic

We present the logical foundation of an artificial intelligence (AI) capable of dealing with complex dynamic challenges, that would be very hard to handled using traditional approaches (e.g. predicate logic and deep learning). The AI is based on a cooperative questioning game, to boost insight. Insight gains are measured by information, probability, uncertainty (Shannon), as well as utility (von Neumann). The framework is a two-person cooperative iterated Q&A game, in which both players (human, AI agent) benefit (positive-sum): the human player gains insight and the AI player learns to improve its suggestions. Generally speaking, valuable insight is typically gained by asking ’good’ questions about the ’right’ topic, at the ’appropriate’ time and place: by posing insightful questions. In this study, we propose a logical and mathematical framework, for the meanings of ’good, right, appropriate’, within clearly-defined classes of human intentions.

Fighting-Game Gameplay Generation Using Highlight Cues

In this article, we propose a fighting-game artificial intelligence (AI) that selects its actions from the perspective of highlight generation using Monte Carlo tree search with three highlight cues in the evaluation function. The intended use of this proposed AI is to generate gameplay in live streaming platforms such as Twitch and YouTube where a large number of spectators watch gameplay to entertain themselves. A gameplay analysis and user study are conducted using FightingICE, a fighting-game platform in an international fighting-game AI competition. Results show that gameplay generated by two AI players of the proposed method has more promising characteristics than not only gameplay generated according to an existing method but also gameplay by the top two AI players in the 2019 competition and that it is more entertaining than the latter gameplay.

A Card Game for Collecting Human-Perceived Similarity Data of Artwork Images

In this paper, we propose a card game system named “Lottery” for collecting similarity data of artwork images. It is built based on the concept of game design called “Audience Participation Game With a Purpose (APGWAP),” which is to outsource computational steps in a given task to humans via a game that allows audiences to participate in a meaningful way. The game system is streamed on Twitch. In this game system, two Artificial Intelligence (AI) players match two cards that are most similar from a set of cards in their turn and discard them. The audiences can choose their roles between Assistant and Jury. Assistant helps the AI player of a given turn choose such a pair of cards, while Jury gives a score telling how similar a chosen pair is. In the experiments, different methods for providing rewards to audiences are investigated: equal reward, random reward, and performance-based reward. Experimental results show that the performance-based reward significantly promotes personal gratification and provides a better gaming experience on several other factors. In addition, it was found in our experiment that different groups of humans provided similar similarity scores for all the pairs of images, indicating that collecting promises reliable human data that are consistent through several trials is possible using our game system.

TAG: Terraforming Mars

Games and Artificial Intelligence (AI) have had a tight relationship for many years. A multitude of games have been used as environments in which AI players can learn to act and interact with others or the game mechanics directly; used as optimisation problems; used as generators of large amounts of data which can be analysed to learn about the game, or about the players; or used as containers of content which can be automatically generated by AI methods. Yet many of these environments have been very simple and limited in scope. We propose here a much more complex environment based on the boardgame Terraforming Mars, implemented as part of the Tabletop Games Framework: a very large and dynamic action space, hidden information, large amounts of content, resource management and high variability make this problem domain stand out in the current landscape and a very interesting problem for AI methods of multiple domains. We include results of baseline AI game-players in this game and in-depth analysis of the game itself, together with an exploration of problem complexity, challenges and opportunities.

Method for Constructing Artificial Intelligence Player With Abstractions to Markov Decision Processes in Multiplayer Game ofMahjong

In this article, we propose a method for constructing artificial intelligence (AI) player of Mahjong, which is a multiplayer imperfect information game. Since the size of the game tree is huge, constructing an expert-level AI player of Mahjong is challenging. We define multiple Markov decision processes (MDPs) as abstractions of Mahjong to construct effective search trees. We also introduce two methods of inferring state values of the Mahjong using these MDPs. We evaluated the effectiveness of our method using gameplays vis-à-vis the current strongest AI player.

The pace of artificial intelligence innovations: Speed, talent, and trial-and-error

• Investigating the pace of AI innovations has thus become increasingly relevant. Yet, few studies have managed to measure or depict this increasing velocity of innovations in AI. • A research framework and three measures, i.e., Average Time Interval (ATI), Innovation Speed (IS), and Update Speed (US), are proposed, to explore the pace of AI from the perspective of AI papers, AI players, and trial and error. The fine-grained analysis of the AI field enabled to portrait the pace of AI innovation and demonstrated that the proposed approach can be adopted to understand other fast-growing fields. • In 2019, more than 3 AI preprints were submitted to arXiv per hour, over 148 times faster than in 1994. Furthermore, there was one deep learning–related preprint submitted to arXiv every 0.87 hours in 2019, over 1,064 times faster than in 1994. • 5.26 new researchers entered into the field of AI each hour in 2019, more than 175 times faster than in the 1990s. • An updated version of the AI preprint was submitted to arXiv every 41 days, with around 33% of arXiv’s AI preprints being updated twice or more in 2019. It took about 0.2 year on average for AI preprints to get their first citations from 2014–2019, around 5 times faster than from 2000–2007. Innovations in artificial intelligence (AI) are occurring at speeds faster than ever witnessed before. However, few studies have managed to measure or depict this increasing velocity of innovations in the field of AI. In this paper, we combine data on AI from arXiv and Semantic Scholar to explore the pace of AI innovations from three perspectives: AI publications, AI players, and AI updates (trial and error). A research framework and three novel indicators, Average Time Interval (ATI), Innovation Speed (IS) and Update Speed (US), are proposed to measure the pace of innovations in the field of AI. The results show that: (1) in 2019, more than 3 AI preprints were submitted to arXiv per hour, over 148 times faster than in 1994. Furthermore, there was one deep learning–related preprint submitted to arXiv every 0.87 h in 2019, over 1064 times faster than in 1994. (2) For AI players, 5.26 new researchers entered into the field of AI each hour in 2019, more than 175 times faster than in the 1990s. (3) As for AI updates (trial and error), one updated AI preprint was submitted to arXiv every 41 days, with around 33 % of AI preprints having been updated at least twice in 2019. In addition, as reported in 2019, it took, on average, only around 0.2 year for AI preprints to receive their first citations, which is 5 times faster than 2000–2007. This swift pace in AI illustrates the increase in popularity of AI innovation. The systematic and fine-grained analysis of the AI field enabled to portrait the pace of AI innovation and demonstrated that the proposed approach can be adopted to understand other fast-growing fields such as cancer research and nano science.

Nature of Attractive Multiplayer Games: Case Study on China’s Most Popular Card Game—DouDiZhu

DouDiZhu, a multiplayer game with incomplete information, is the most popular card game in China. Although there are many DouDiZhu card games in the world, the specific characteristics of classical DouDiZhu card games are a harmonious combination of player numbers, player characters (landlords and peasants), deck numbers, and scoring systems. However, research on the complexity and attractiveness of DouDiZhu has not established. Therefore, in this paper, artificial intelligence (AI) players of different levels of DouDiZhu game were constructed for research, self-game simulation was conducted for DouDiZhu AI players, and game refinement measures were used to evaluate and identify the best Settings of the game. The results show that classical DouDiZhu provides the most complex game setup for all types of DouDiZhu AI players, while also clarifying its popularity.

Emulating Human Play in a Leading Mobile Card Game

Monte Carlo tree search (MCTS) has become a popular solution for game artificial intelligence (AI), capable of creating strong game playing opponents. However, the emergent playstyle of agents using MCTS is not necessarily human-like, believable or enjoyable. AI Factory Spades , currently the top rated Spades game in the Google Play store, uses a variant of MCTS to control AI allies and opponents. In collaboration with the developers, we showed in a previous study that the playstyle of human players significantly differed from that of the AI players. This paper presents a method for player modeling using gameplay data and neural networks that does not require domain knowledge, and a method of biasing MCTS with such a player model to create Spades playing agents that emulate human play whilst maintaining strong, competitive performance. The methods of player modeling and biasing MCTS presented in this study are applied to the commercial codebase of AI Factory Spades , and are transferable to MCTS implementations for discrete-action games where relevant gameplay data are available.

Artificial Intelligence and International Security: The Long View

AbstractHow will emerging autonomous and intelligent systems affect the international landscape of power and coercion two decades from now? Will the world see a new set of artificial intelligence (AI) hegemons just as it saw a handful of nuclear powers for most of the twentieth century? Will autonomous weapon systems make conflict more likely or will states find ways to control proliferation and build deterrence, as they have done (fitfully) with nuclear weapons? And importantly, will multilateral forums find ways to engage the technology holders, states as well as industry, in norm setting and other forms of controlling the competition? The answers to these questions lie not only in the scope and spread of military applications of AI technologies but also in how pervasive their civilian applications will be. Just as civil nuclear energy and peaceful uses of outer space have cut into and often shaped discussions on nuclear weapons and missiles, the burgeoning uses of AI in consumer products and services, health, education, and public infrastructure will shape views on norm setting and arms control. New mechanisms for trust and confidence-building measures might be needed not only between China and the United States—the top competitors in comprehensive national strength today—but also among a larger group of AI players, including Canada, France, Germany, India, Israel, Japan, Russia, South Korea, and the United Kingdom.

Learning atoms for materials discovery

Exciting advances have been made in artificial intelligence (AI) during recent decades. Among them, applications of machine learning (ML) and deep learning techniques brought human-competitive performances in various tasks of fields, including image recognition, speech recognition, and natural language understanding. Even in Go, the ancient game of profound complexity, the AI player has already beat human world champions convincingly with and without learning from the human. In this work, we show that our unsupervised machines (Atom2Vec) can learn the basic properties of atoms by themselves from the extensive database of known compounds and materials. These learned properties are represented in terms of high-dimensional vectors, and clustering of atoms in vector space classifies them into meaningful groups consistent with human knowledge. We use the atom vectors as basic input units for neural networks and other ML models designed and trained to predict materials properties, which demonstrate significant accuracy.

Usertesting Without the User

The use of human participants in game evaluation can be costly, time-consuming, and present challenges for constructing representative player samples. These challenges may be overcome by using computer-controlled agents in place of human users for certain stages of the usertesting process. This article explores opportunities and challenges in the use of behavioural modelling to create independent “user” agents driven by artificial intelligence (AI). We highlight the utility of imitating cognitive processes such as spatial reasoning, memory, and goal-oriented decision-making as a means to increase the viability of independent agents as a tool in usertesting. Specifically, we investigate the possible design and use of proxy AI “users” that mimic human navigational behaviour to assist in the evaluation of level designs. Ultimately, we propose that a configurable population of AI players can provide a data-rich supplement to current approaches in games user research.

Performance Evaluation Gaps in a Real-Time Strategy Game Between Human and Artificial Intelligence Players

Since 2010, annual StarCraft artificial intelligence (AI) competitions have promoted the development of successful AI players for complex real-time strategy games. In these competitions, AI players are ranked based on their win ratio over thousands of head-to-head matches. Although simple and easily implemented, this evaluation scheme may less adequately help develop more human-competitive AI players. In this paper, we recruited 45 human StarCraft players at different expertise levels (expert/medium/novice) and asked them to play against the 18 top AI players selected from the five years of competitions (2011-2015). The results show that the human evaluations of AI players differ substantially from the current standard evaluation and ranking method. In fact, from a human standpoint, there has been little progress in the quality of StarCraft AI players over the years. It is even possible that AI-only tournaments can lead to AIs being created that are unacceptable competitors for humans. This paper is the first to systematically explore the human evaluation of AI players, the evolution of AI players, and the differences between human perception and tournament-based evaluations. The discoveries from this paper can support AI developers in game companies and AI tournament organizers to better incorporate the perspective of human users into their AI systems.

An analysis of play style of advanced mahjong players toward the implementation of strong AI player

The studies of artificial intelligence (AI) on the game with perfect information has been very much advanced to have an ability to compete top-rate human players. In contrast, it is still difficult for AI to seek the best strategy of the facing situation in the games with imperfect information. In this type of the games, it is usually more effective for a player to adopt the strategy which match the other players’ strategies than to find an optimal strategy. In this study, we took mahjong as an example of complicated games with imperfect information. We propose a new method to classify opponents’ strategies in mahjong by analyzing play records of mahjong called ‘Haihu’ statistically. We find that the advanced mahjong players’ behaviors are classified into four patterns. We also validate out result by implementing simple AI programs.

The missing media: The procedural rhetoric of computer games

The discussion about the implications of new or digital media focuses mainly on ‘social’ media. This reduction is a conceptual shortcoming. In order to fully understand how digital media impact on society and the communication landscape, our conceptualisation needs to include new entertainment media, especially computer games. Interactivity here might mean interaction with the AI (artificial intelligence) or human players or both, but the crucial difference lies in the fact that the interaction takes place in a ‘world’ created by the software. Despite the discussion about ‘gamification’ in marketing and a tradition of game studies in the humanities, there have been few attempts to treat computer games not only as trivial culture, but as a “persuasive device”, as a way to shape public opinion. This article explores how the theory of procedural rhetoric, as outlined by Ian Bogost, enhances our understanding of this growing area.

WHAT CAN GAME AI TEACH US?

Artificial intelligence (AI) applications typically involve encoding expert knowledge in machine form to find optimal solutions for a given problem. However, this paper deals with the opposite process of extracting new and human-comprehensible insights from emergent AI behaviour. Some examples of useful game-related insights drawn from observing AI players in action are presented.

Towards a data‐driven approach to scenario generation for serious games

ABSTRACTSerious games have recently shown great potential to be adopted in many applications, such as training and education. However, one critical challenge in developing serious games is the authoring of a large set of scenarios for different training objectives. In this paper, we propose a data‐driven approach to automatically generate scenarios for serious games. Compared with other scenario generation methods, our approach leverages on the simulated player performance data to construct the scenario evaluation function for scenario generation. To collect the player performance data, an artificial intelligence (AI) player model is designed to imitate how a human player behaves when playing scenarios. The AI players are used to replace human players for data collection. The experiment results show that our data‐driven approach provides good prediction accuracy on scenario's training intensities. It also outperforms our previous heuristic‐based approach in its capability of generating scenarios that match closer to specified target player performance.Copyright © 2014 John Wiley & Sons, Ltd.