An Improved Energy Management Strategy for Hybrid Electric Vehicles Integrating Multistates of Vehicle-Traffic Information

Abstract

This study aims to answer the key question for hybrid electric vehicles (HEVs) on how to manage the power flow in HEVs with recent intelligent and electrified upgrades in automotive industries. The new energy management strategy (EMS) needs to fuse both the physic and cyber systems, reflecting the dynamic vehicle system in the physical layer, as well as taking full advantage of the outside information in the cyber layer. Given that, this article proposes the cyber-physical system (CPS)-based EMS using deep reinforcement learning (DRL) in two different types of vehicles [hybrid electric bus (HEB) and Prius]. Under the proposed framework, exploratory training is carried out for the EMS which is mediated by DRL algorithms, expert prior knowledge and multistate of traffic information. Then, the prior valid knowledge trained by HEB is applied to Prius based on deep transfer learning, accelerating the new EMS convergence and ensuring the same initial parameters of the two vehicles' deep neural networks. Moreover, the cyber information is decoupled from the vehicle itself, for the first time being visualized for comparative analysis. The results show a significant improvement by considering traffic states (TS) and using dynamic programming (DP) as a benchmark with 6.94% fuel economy improvement for deep deterministic policy gradients (DDPG) test results and 8.12% for deep Q-learning (DQL), respectively. The decoupling analysis distinguished the effect of various TS for the HEB and Prius due to their different characteristics in vehicle service, driving style, and vehicle structures, which further verifies the effectiveness of the proposed EMS.