Diversifying dynamic difficulty adjustment agent by integrating player state models into Monte-Carlo tree search

Abstract

Game developers have employed dynamic difficulty adjustment (DDA) in designing game artificial intelligence (AI) to improve players’ game experience by adjusting the skill of game agents. Traditional DDA agents depend on player proficiency only to balance game difficulty, and this does not always lead to improved enjoyment for the players. To improve game experience, there is a need to design game AIs that consider players’ affective states. Herein, we propose AI opponents that decide their next actions according to a player’s affective states, in which the Monte-Carlo tree search (MCTS) algorithm exploits the states estimated by machine learning models referencing in-game features. We targeted four affective states to build the model: challenge, competence, valence, and flow. The results of our user study demonstrate that the proposed approach enables the AI opponents to play automatically and adaptively with respect to the players’ states, resulting in an enhanced game experience.