Collaborative Optimization of Energy Management Strategy and Adaptive Cruise Control Based on Deep Reinforcement Learning

Abstract

Hybrid electric vehicles (HEVs) have great prospects in reducing fossil fuel consumption, and adaptive cruise control (ACC) technology provides safe and convenient travel for drivers. The fusion of the two technologies can theoretically improve the safety, comfort, and fuel economy of vehicles. Hence, the energy management strategy (EMS) of Prius, a typical HEV configuration, is studied under the car-following scenario. This optimization problem involves complex systems, inconsistent objectives, and stringent constraints, which may be challenging to conventional algorithms. Therefore, a novel deep deterministic policy gradient (DDPG)-based ecological driving strategy (DDPG-ECO) is proposed and the weights of multiple objectives are analyzed to optimize the training results. The extensive simulation experiment compares the effects of Ornstein-Uhlenbeck action noise (OUAN) and soft-max action noise (SAN), which act on the acceleration action. Simulations under different driving cycles show that the fuel economy of DDPG-ECO can achieve more than 90% of dynamic programming (DP)-based methods on the conditions of ensuring car-following performances.