Deep Reinforcement Learning Based PHEV Energy Management With Co-Recognition for Traffic Condition and Driving Style

Abstract

As one of the key technologies of plug-in hybrid electric vehicle (PHEV), the energy management strategy (EMS) is interactively influenced by driving style, traffic condition as well as vehicle operation state. In this paper, a novel twin delayed deep deterministic police gradient (TD3) algorithm based EMS integrating co-recognition for driving style and traffic condition is proposed, aiming to improve the generalization ability of EMS in various application scenarios with superior energy saving performance and higher self-learning efficiency. In particular, the TD3 based energy management architecture combining delayed policy update and smooth regularization technologies is studied to achieve simultaneous improvement for PHEV energy efficiency and strategy convergence speed. Furthermore, the traffic conditions are recognized by fuzzy C-means method, while the local minimum problem is effectively avoided by incorporating simulated annealing (SA) and genetic algorithm (GA). Sequentially, the driving styles are decoupled from recognized traffic condition, which are further recognized as three typical styles. The comparison results of the proposed strategy with several representative deep reinforcement learning based EMSs indicate that the TD3 based EMS outperforms DDQN and DDPG based EMSs in terms of convergence speed and energy saving performance. With considering the recognized traffic condition and driving style, the energy efficiency of TD3 based EMS is further improved with an ideal robustness to various driving cycles.