Performance Analysis and Improvement of Deep Q Network in Pommerman Agent Based on Compound Training Method

Abstract

With the rapid development of the game industry, various intelligent agents are created and then applied in video games. Many of these kinds of agents are trained with reinforcement learning methods as well as deep-learning methods. However, under any circumstances, the agent-based on deep learning method requires a relatively high cost in computation resources. To this end, we propose a compound method with a relatively low cost to improve the training effect of pommerman agents. We separate the training into two stages. The first stage focuses on training the robot for survival with imitation learning methods. The second stage focuses on athletics by utilizing Deep Q-Network and reinforcement learning. Experiments show that the agent-based two-stage compound training method can win 4% more games than other agents in actual pommerman games. Meanwhile, judging from the 5-times increase in average game length before and after pre-training, the training process becomes more efficient and effective applying compound training methods.