Energy Management Strategy Combining Double Deep Q‐Networks and Demand Torque Prediction for Connected Hybrid Electric Vehicles

Abstract

This paper proposes an energy management strategy (EMS) based on double‐deep Q‐Networks (DDQN) with demand torque prediction (DTP) to optimize the fuel consumption of hybrid electric vehicles (HEVs) by online utilizing vehicle‐to‐vehicle (V2V) information. The main framework of the EMS is designed as DDQN, combining Q‐learning with the deep neural network to realize real‐time training and online optimization of the torque distribution between the internal combustion engine and the electric motor, and given that the model training easily falls into local minima and overestimation, two different Q‐networks are designed to decouple action selection and target evaluation. Meanwhile, BP‐network predicts future demand torque using the ego vehicle speed, front vehicle speed, and distance between the two vehicles to reduce the burden of model training due to the introduction of traffic information. The predicted demand torque is introduced into the DDQN‐based EMS as states, together with the distance traveled in the driving cycle and the current information on HEV. The effectiveness and adaptability to actual driving cycles of the proposed strategy are verified by comparison with DDQN‐based EMS without DTP and rule‐based EMS. The results show that the fuel consumption using DDQN‐based EMS without DTP is reduced by 4.1% compared with rule‐based EMS. The introduction of demand torque prediction resulted in a further 3.9% reduction in fuel consumption. © 2023 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.