A mixed-initiative design framework for procedural content generation using reinforcement learning

Abstract

Currently, there are a significant and growing number of games and players. Creating digital games becomes a challenging task, as manual game development is costly and time-consuming. A technique known as procedural content generation (PCG) can potentially reduce both the time and production costs of games. It is feasible to automate the creation process by utilizing artificial intelligence techniques and PCG, assisting game designers in their tasks. PCG is not a novel concept, and there is a diverse range of algorithms aimed at automatically generating content in games. However, a significant number of these techniques do not incorporate artificial intelligence. This paper introduces the PCGRLPuzzle framework used to generate procedural scenarios through Reinforcement Learning agents trained with the Policy Proximal Optimization algorithm. The process of building scenarios poses a challenging problem due to the existence of an exponential number of possibilities. The framework employs a mixed-initiative design, where humans and computers collaborate to create levels for 2D dungeon crawler games. We apply this framework to generate levels for three different games and analyze the results based on their expressive range, evaluating linearity and lenience. The conducted experiments demonstrate that utilizing Reinforcement Learning in conjunction with Procedural Content Generation and mixed-initiative enables the generation of highly diverse levels.