Improving the Performance of Deep Q-learning in Games Pong and Ms. Pacman

Abstract

Unlike TD-gammon architecture, deep Q-learning algorithm combines deep learning and reinforcement learning, and it has achieved outstanding results in many Atari games. This study primarily focuses on two games, Pong and Ms. Pacman. It explores several approaches to improve the performance of deep Q-network (DQN). Based on the data obtained, while DQN displays a high-level performance in the simple Atari game Pong, it struggles a bit when learning the more complex game Ms. Pacman, leading to diverged loss. This under-performance may be partly explained by the shorter training time than the original paper due to limited computational resource. Given sufficient time of training and exploring, the model is believed to eventually converge once it identifies the optimal combination of hyperparameters.