Optimizing Agent Training with Deep Q-Learning on a Self-Driving Reinforcement Learning Environment

Abstract

Nowadays, there are many Deep Q-Learning (DQL) variants and Reinforcement Learning (RL) algorithms that have outperformed the original DQL algorithm using a simple Deep Q-Network (DQN) architecture on several RL applications. In this paper, we present an environment-oriented adapted DQN implementation on the OpenAI Gym CarRacing self-driving environment where a driver agent successfully learns difficult control policies from raw pixels using a deep convolutional neural network. By implementing multiple optimizations and environment-specific adaptations on the original DQL algorithm, we were able to improve on some of the limitations of the base DQN algorithm significantly on this platform. Our trained agent was therefore able to produce successful results and obtain higher convergence scores while being more time efficient and using less computational resources. Obtaining a promising average score over one hundred consecutive trials, the results achieved suggest that the proposed adapted DQN agent improves substantially on the base DQL algorithm. Our agent surpassed Double Deep Q-Network (DDQN) and multiple other approaches in the CarRacing RL environment.