Modeling Decisions in Games Using Reinforcement Learning

Abstract

Reinforcement-learning (RL) algorithms have been used to model human decisions in different decision-making tasks. Recently, certain deep RL algorithms have been proposed; however, there is little research that compares deep RL algorithms with traditional RL algorithms in accounting for human decisions. The primary objective of this paper is to compare deep and traditional RL algorithms in a virtual environment concerning their performance, learning speed, ability to account for human decisions, and ability to extract features from the decision environment. We implemented traditional RL algorithms like imitation learning, Q-Learning, and a deep RL algorithm, DeepQ Learning, to train an agent for playing a platform jumper game. For model comparison, we collected human data from 15 human players on the platform jumper game. As part of our evaluation, we also increased the speed of the moving platform in the jumper game to test how humans and model agents responded to the changing game conditions. Results showed that DeepQ approach took more training episodes than the traditional RL algorithms to learn the gameplay. However, the DeepQ algorithm could extract features directly from images of gameplay; whereas, other algorithms had to be fed the extracted features. Furthermore, conventional algorithms performed more human-like in a slow version of the game; however, the DeepQ algorithm performed more humanlike in the fast version of the game.