A framework for designing Reinforcement Learning agents with Dynamic Difficulty Adjustment in single-player action video games

Abstract

Dynamic Difficulty Adjustment (DDA) within video games aims to avoid frustration or boredom. This paper provides the first framework for designing a Reinforcement Learning (RL) agent with DDA for single-player action video games. The framework includes the definitions of states, actions, and rewards. We propose a 2Q-table system that can provide a better winning/losing ratio and extend the duration of the rounds. We apply the framework to a use case study. We address the challenges that the design and implementation of RL agents with DDA for single-player action video games might present, such as (i) large and/or continuous action–state spaces, (ii) an appropriate definition of the rewards for achieving a correct DDA, (iii) learning from each player online from limited samples and (iv) in an arcade shooter video game. The two evaluations performed (with computer-driven and human players) show that the paper’s goals are met since players face personalized challenges according to their playing skills.