A Deep Deterministic Policy Gradient Approach for Vehicle Speed Tracking Control With a Robotic Driver

Abstract

In performance tests, replacing humans with robotic drivers has many advantages, such as high efficiency and high security. To realize the vehicle speed tracking control with a robotic driver, this article proposes a novel deep reinforcement learning (DRL) approach based on deep deterministic policy gradient (DDPG). Specifically, the design of the approach includes state space, action space, reward function, and control algorithm. Then, to shorten the training time, the proposed approach utilizes the basic fundamental relationship between vehicle speed and pedal opening to intervene in network exploration. Furthermore, to solve speed fluctuations in low-speed sections, the replay buffer is optimized by adding weighted training samples. Experiments are conducted on fifteen cars, and results show that the proposed algorithm can effectively control the vehicle speed. Generally, it only needs three or four episodes of training to meet the requirements. Compared with the Segment-PID method, the proposed method has a smoother speed and fewer overbound times. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article was motivated by the algorithm of deep reinforcement learning (DRL), which can be applied to automatic control including vehicle speed tracking control with a robotic driver. With the complexity, the delay of the vehicle model, and the dead zone of the pedal, it is difficult for existing classic control theories to achieve satisfactory vehicle speed tracking. This article analyzes the mathematical model based on the DRL theory and the vehicle dynamic model, then builds the speed tracking controller based on the deep deterministic policy gradient (DDPG) algorithm, and makes corresponding optimizations for the problems in the experiment. Through the control software integrated with the DDPG algorithm, the operator can easily realize the training and testing of the neural network, which greatly reduces the time and cost of vehicle testing and is of great significance to the emission certification of new vehicles.