Continuous Control in Car Simulator with Deep Reinforcement Learning

Abstract

Deep reinforcement learning (DRL), which can be trained without abundant labeled data required in supervised learning, plays an important role in autonomous vehicle researches. According to action space, DRL can be further divided into two classes: discrete domain and continuous domain. In this work, we focus on continuous steering control since it's impossible to switch among different discrete steering values at short intervals in reality. We first define the steering smoothness to quantify the degree of continuity. Then we propose a new penalty in reward shaping. We carry experiments based on Deep Deterministic Policy Gradient (DDPG) and Asynchronous Advantage Actor Critic (A3C), which are the state of the art in continuous domain. Results show that the proposed penalty improves the steering smoothness with both algorithms.