Reinforcement Learning In Games Research Articles

Reinforcement Learning in Game Industry—Review, Prospects and Challenges

This article focuses on the recent advances in the field of reinforcement learning (RL) as well as the present state–of–the–art applications in games. First, we give a general panorama of RL while at the same time we underline the way that it has progressed to the current degree of application. Moreover, we conduct a keyword analysis of the literature on deep learning (DL) and reinforcement learning in order to analyze to what extent the scientific study is based on games such as ATARI, Chess, and Go. Finally, we explored a range of public data to create a unified framework and trends for the present and future of this sector (RL in games). Our work led us to conclude that deep RL accounted for roughly 25.1% of the DL literature, and a sizable amount of this literature focuses on RL applications in the game domain, indicating the road for newer and more sophisticated algorithms capable of outperforming human performance.

A Survey of Text Games for Reinforcement Learning Informed by Natural Language

Abstract Reinforcement Learning has shown success in a number of complex virtual environments. However, many challenges still exist towards solving problems with natural language as a core component. Interactive Fiction Games (or Text Games) are one such problem type that offer a set of safe, partially observable environments where natural language is required as part of the Reinforcement Learning solution. Therefore, this survey’s aim is to assist in the development of new Text Game problem settings and solutions for Reinforcement Learning informed by natural language. Specifically, this survey: 1) introduces the challenges in Text Game Reinforcement Learning problems, 2) outlines the generation tools for rendering Text Games and the subsequent environments generated, and 3) compares the agent architectures currently applied to provide a systematic review of benchmark methodologies and opportunities for future researchers.

A modeling environment for reinforcement learning in games

Developing Non-Player Characters, i.e., game characters that interact with the game’s environment autonomously with flexibility to experiment different behavior configurations is not a trivial task. Traditionally, this has been done with techniques that limit the complexity of the behavior of Non-Player Characters (NPCs), as in the case of using Navigation Mesh (NavMesh) for navigation behaviors. For this problem, it has been shown that reinforcement learning can be more efficient and flexible than traditional techniques. However, integrating reinforcement learning into current game development tools is laborious, given that a great deal of experimentation and coding is required. For that, we have developed a modeling environment that integrates with a game development tool and allows the direct specification of reward functions and NPC agent components with maximum code reuse and automatic code generation.

Role‐based attention in deep reinforcement learning for games

AbstractReinforcement learning method that learns while interacting with the environment, relies heavily on the concept of state as the input to the policy and value function. In the task, the view of agent contains a lot of information, and it is difficult for the agent to learn to ignore the irrelevant information and focus on the key information. Inspired by recent work in attention models for computer vision, we present a role‐based attention model for reinforcement learning. The proposed model uses convolutional neural networks to generate soft attention maps, adding crucial role information in the task, forcing the agent to focus on important features and distinguish task‐related information. To validate the performance in complex problems, the proposed approach is evaluated in a challenging scenario, Football Academy in Google Research Football Environment, a newly released reinforcement learning environment with physics‐based three‐dimensional simulator. The experimental results demonstrate that agents using role‐based attention mechanism can perform better in football games.

Nonzero-Sum Game Reinforcement Learning for Performance Optimization in Large-Scale Industrial Processes.

This article presents a novel technique to achieve plant-wide performance optimization for large-scale unknown industrial processes by integrating the reinforcement learning method with the multiagent game theory. A main advantage of this technique is that plant-wide optimal performance is achieved by a distributed approach where multiple agents solve simplified local nonzero-sum optimization problems so that a global Nash equilibrium is reached. To this end, first, the plant-wide performance optimization problem is reformulated by decomposition into local optimization subproblems for each production index in a multiagent framework. Then, the nonzero-sum graphical game theory is utilized to compute the operational indices for each unit process with the purpose of reaching the global Nash equilibrium, resulting in production indices following their prescribed target values. The stability and the global Nash equilibrium of this multiagent graphical game solution are rigorously proved. The reinforcement learning methods are then developed for each agent to solve the nonzero-sum graphical game problem using data measurements available in the system in real time. The plant dynamics do not have to be known. Finally, the emulation results are given to show the effectiveness of the proposed automated decision algorithm by using measured data from a large mineral processing plant in Gansu Province, China.

Research on Micro-Grid Group Intelligent Decision Mechanism under the Mode of Block-Chain and Multi-Agent Fusion

This paper aims to study the problems of surplus interaction, poor real-time performance, and excessive processing of information in the micro-grid scheduling and decision-making process. Firstly, the micro-grid dual-loop mobile topology structure is designed by using the method of block-chain and multi-agent fusion, realizing the real-time update of the decision-making body. Secondly, on the basis of optimizing the decision-making body, a two-layer model of intelligent decision-making under the decentralized mechanism is established. Aiming at the upper model, based on the theory of block-chain consensus mechanism, this paper proposes an improved evolutionary game algorithm. The maximum risk-benefit in the decision-making process is the objective function, which realizes the evaluation and optimization of decision tasks. For the lower layer model, based on the block-chain distributed ledger theory, this paper proposes an improved hybrid game reinforcement learning algorithm, with the maximum controllable load participation as the objective function, and realizes the optimal configuration of distributed energy in the micro-grid. This paper reveals the rules of group intelligent decision making in micro-grid under multi-task. Finally, the effectiveness of the proposed algorithm is verified by using Beijing Jin-feng Energy Internet Park data.

Reports of the Workshops Held at the 2019 AAAI Conference on Artificial Intelligence

The workshop program of the Thirty‐Third AAAI Conference on Artificial Intelligence (AAAI‐19) was held in Honolulu, Hawaii, on Sunday and Monday, January 27 and 28, 2019. There were 16 workshops in the program: Affective Content Analysis: Modeling Affect‐in‐Action; Agile Robotics for Industrial Automation Competition; Artificial Intelligence for Cyber Security; Artificial Intelligence Safety; Dialog System Technology Challenge; Engineering Dependable and Secure Machine Learning Systems; Games and Simulations for Artificial Intelligence; Health Intelligence; Knowledge Extraction from Games; Network Interpretability for Deep Learning; Plan, Activity, and Intent Recognition; Reasoning and Learning for Human‐Machine Dialogues; Reasoning for Complex Question Answering; Recommender Systems Meet Natural Language Processing; Reinforcement Learning in Games; and Reproducible AI. This report contains brief summaries of all the workshops that were held.

Understanding the Reinforcement Learning

Artificial Intelligence has been a hot topic for a long time. It is a cross-discipline combined with many fields. Among them, machine learning plays the most important role. There are four core subjects in machine learning, supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. In this paper, we are going to talk about the reinforcement learning in the perspective of Markov Decision Process and Partially Observable Markov Decision Process, which are the core algorithms in reinforcement learning. Also, an example of Hearthstone is illustrated to show how to apply reinforcement learning in games for better understanding.

Nonconvergence to saddle boundary points under perturbed reinforcement learning

For several reinforcement learning models in strategic-form games, convergence to action profiles that are not Nash equilibria may occur with positive probability under certain conditions on the payoff function. In this paper, we explore how an alternative reinforcement learning model, where the strategy of each agent is perturbed by a strategy-dependent perturbation (or mutations) function, may exclude convergence to non-Nash pure strategy profiles. This approach extends prior analysis on reinforcement learning in games that addresses the issue of convergence to saddle boundary points. It further provides a framework under which the effect of mutations can be analyzed in the context of reinforcement learning.

The dynamics of generalized reinforcement learning

We consider reinforcement learning in games with both positive and negative payoffs. The Cross rule is the prototypical reinforcement learning rule in games that have only positive payoffs. We extend this rule to incorporate negative payoffs to obtain the generalized reinforcement learning rule. Applying this rule to a population game, we obtain the generalized reinforcement dynamic which describes the evolution of mixed strategies in the population. We apply the dynamic to the class of Rock–Scissor–Paper (RSP) games to establish local convergence to the interior rest point in all such games, including the bad RSP game.

The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

Urban traffic self-adaptive control problem is dynamic and uncertain, so the states of traffic environment are hard to be observed. Efficient agent which controls a single intersection can be discovered automatically via multiagent reinforcement learning. However, in the majority of the previous works on this approach, each agent needed perfect observed information when interacting with the environment and learned individually with less efficient coordination. This study casts traffic self-adaptive control as a multiagent Markov game problem. The design employs traffic signal control agent (TSCA) for each signalized intersection that coordinates with neighboring TSCAs. A mathematical model for TSCAs’ interaction is built based on nonzero-sum markov game which has been applied to let TSCAs learn how to cooperate. A multiagent Markov game reinforcement learning approach is constructed on the basis of single-agentQ-learning. This method lets each TSCA learn to update itsQ-values under the joint actions and imperfect information. The convergence of the proposed algorithm is analyzed theoretically. The simulation results show that the proposed method is convergent and effective in realistic traffic self-adaptive control setting.

Learning Companion Behaviors Using Reinforcement Learning in Games

Our goal is to enable Non Player Characters (NPC) in computer games to exhibit natural behaviors. The quality of behaviors affects the game experience especially in story-based games, which rely on player-NPC interactions. We used Reinforcement Learning to enable NPC companions to develop preferences for actions. We implemented our RL technique in BioWare Corp.’s Neverwinter Nights. Our experiments evaluate an NPC companion’s behaviors regarding traps. Our method enables NPCs to rapidly learn reasonable behaviors and adapt to changes in the game.

A fuzzy decision tree-based robust Markov game controller for robot manipulators

Two-player zero-sum Markov game framework offers an effective platform for designing robust controllers. In the Markov game-based learning, theoretical convergence of the learning process with the function approximator cannot be guaranteed. However, fusing Q-learning with decision tree (DT) function approximator has shown good learning performance and more reliable convergence. It scales better to larger input spaces with lower memory requirements, and can solve problems that are infeasible using table lookup. This motivates us to introduce DT function approximator in Markov game reinforcement learning (RL) framework. This approach works, though it deals with only discrete actions. In realistic applications, it is imperative to deal with continuous state?action spaces. In this paper, we propose Markov game framework for continuous state?action space systems using fuzzy DT as a function approximator. Simulation experiments on a two-link robot manipulator bring out the importance of the proposed structure in terms of better robust performance and computational efficiency.

Reinforcing motor re-training and rehabilitation through games: a machine-learning perspective

Reinforcing motor re-training and rehabilitation through games: a machine-learning perspective

Transient and asymptotic dynamics of reinforcement learning in games

Reinforcement learners tend to repeat actions that led to satisfactory outcomes in the past, and avoid choices that resulted in unsatisfactory experiences. This behavior is one of the most widespread adaptation mechanisms in nature. In this paper we fully characterize the dynamics of one of the best known stochastic models of reinforcement learning [Bush, R., Mosteller, F., 1955. Stochastic Models of Learning. Wiley & Sons, New York] for 2-player 2-strategy games. We also provide some extensions for more general games and for a wider class of learning algorithms. Specifically, it is shown that the transient dynamics of Bush and Mosteller's model can be substantially different from its asymptotic behavior. It is also demonstrated that in general—and in sharp contrast to other reinforcement learning models in the literature—the asymptotic dynamics of Bush and Mosteller's model cannot be approximated using the continuous time limit version of its expected motion.