A Deep Reinforcement Learning Algorithm Suitable for Autonomous Vehicles: Double Bootstrapped Soft-Actor–Critic-Discrete

Abstract

With the rapid advancement of modern society, autonomous systems have been broadly applied in people’s daily lives. Under the guidance of this trend, autonomous vehicles have gradually become popular. However, due to some adverse factors(such as insufficient computing force and limited communication bandwidth) in edge computing scenarios and the lack of autonomous decision-making ability, the safety of autonomous vehicles is not enough. It is a good solution to use a deep reinforcement learning(DRL) algorithm, which combines deep learning(DL) and reinforcement learning(RL), to provide a fast convergence speed and an appropriate decision-making ability. In this paper, based on Soft-Actor-Critic(SAC) and Soft-Actor-Critic-Discrete(SAC-D), we propose a Double Bootstrapped Soft-Actor-Critic-Discrete(DBSAC-D) algorithm. By introducing bootstrap, the ability to explore in action space is enhanced, the value of each action is accurately judged, the convergence process is accelerated and the consumption of the computing force is reduced. In addition, we also propose a novel sampling strategy, which balances the novelty and importance of the sampled data, and improves the training value of the sampled data to the network model. The experimental results show that our proposed algorithm achieves good performances in several traffic scenes and has a fast convergence speed.