Using provenance data and imitation learning to train human-like bots

Abstract

Nonplayer Characters are becoming more realistic in their actions and behav- iors because of the development of gaming technology and gamers’ increased demand for enhancements. While this progress is an exciting development, it has also become a major concern for game developers over the years, since players demand that NPCs look alike to other human players. Our major objective in this work is to make an NPC that satisfactorily mimics a player. This work proposes a method for training an NPC using imitation learning with the Generative Adversarial Imitation Learning framework to become similar to a human player. To simulate player behavior, our proposal trains agents using provenance data sets, cause-and-effect data mining, and the GAIL framework. The proposed model was developed to be universal and adaptable to different games. We validate our model using the DodgeBall game environment inside the Unity ML-Agents Toolkit for Unity Engine. Some players competed against our agent and found that our NPC was credible by observing his actions and behaviors. In this work, we present a new way of giving rewards compared to the model presented in the previous work. The tests and results found were also expanded, improving the validation of our model.