A Multi-sensing Input and Multi-constraint Reward Mechanism Based Deep Reinforcement Learning Method for Self-driving Policy Learning

Abstract

Planning and decision-making of autonomous driving is an active and challenging topic currently. Deep reinforcement learning-based approaches seek to solve the problem in an end-to-end manner, but generally require a large amount of sample data and confronted with high dimensionality of input data and complex models, which lead to slow convergence and cannot learn effectively with noisy data. Most of deep reinforcement learning-based approaches use a sample reward function. Due to the complicated and volatile traffic scenarios, these approaches cannot satisfy the driving policy requirement. To address the issues, a multi-sensing and multi-constraint reward function (MSMC-SAC) based deep reinforcement learning method is proposed. The inputs of the proposed method include front-view image, point cloud from LiDAR, as well as the bird's-eye view generated from the perception results. The multi-sensing input is first passed to an encoding network to obtain the representation in latent space and then forward to a SAC-based learning module. A multiple rewards function considering various constraints, such as the error of transverse-longitudinal distance and heading angle, smoothness, velocity, and the possibility of collision, is designed. The performance of the proposed method in different typical traffic scenarios is validated with CARLA [1]. The effects of multiple reward functions are compared. The simulation results show that the presented approach can learn the driving policies in many complex scenarios, such as straight ahead, passing the intersections, and making turning, and outperforms against the existing typical deep reinforcement learning methods.