Pre-optimization-assisted deep reinforcement learning-based energy management strategy for a series-parallel hybrid electric truck

Abstract

P2-P3 series-parallel hybrid electric vehicles (HEVs) feature intricate topologies with multiple power sources and multiple working modes, posing a challenge for developing effective energy management strategies (EMSs). This paper introduces a framework that combines deep reinforcement learning (DRL) with pre-optimized energy management to address this challenge. Considering the characteristics of HEVs, the framework incorporates the pre-optimized equivalent motor and pre-optimized mode selection rule into a deep deterministic policy gradient (DDPG) loop. The pre-optimization of the equivalent motor aims to reduce the energy management task’s dimensionality by equating two motors as the equivalent motor, while the pre-optimized mode selection rule systematically integrates mode selection and energy allocation into the EMS. The study assumes HEVs operate in connected urban environments and incorporates middle-horizon traffic information to enhance EMS performance. Additionally, the study integrates the frequent engine start-stop characteristics of HEVs into EMS design. Simulation results demonstrate that the fuel economy of the proposed EMSs in this study ranges from 87.2% to 90.7% of the DP-based EMS. The inclusion of an engine start-stop penalty significantly reduces the number of engine start-stops without compromising fuel economy.