Scene Exploration Method of Coin Collection Game Based on Deep Reinforcement Learning Algorithm

Abstract

In game AI control, sparse-reward environment exploration has been a hotspot for agent control. At present, agents based on traditional deep reinforcement learning are unable to learn appropriate policies when faced with complex continuous action space under sparse-reward. Recently, intrinsic reward stimulation has become a promising direction to solve the sparse-reward exploration problem. But in the complex environment, the agent will still get the local optimal policy. To solve the above problems, this paper designed a coin collection game environment. Based on the Proximal Policy Optimization with Random Network Distillation, paper proposed a method for agents to explore sparse-reward environment. In this paper, custom reward function gives extrinsic rewards to the agent. 3D-sensing rays allow agent to observe environment states. By using Long Short-Term Memory (LSTM) network, previous states and current state are fused to estimate the state value. Through comparative experiments, the results show that the proposed method significantly improves the maximum reward convergence rate, and agent obtains the global optimal policy. The agent can explore the scene, and finish the goal more effectively.