Hierarchical Rewarding Deep Deterministic Policy Gradient Strategy for Energy Management of Hybrid Electric Vehicles

Abstract

Electrified vehicles are important for reducing emissions and will dominate road transportation in the future. An energy management strategy is critical for hybrid electric vehicles to manage the use of on-board energy resources. This paper proposes a novel double-layer energy management strategy based on hierarchical rewarding DDPG (HR-DDPG) for a heavy-duty hybrid electric vehicle (HEV) to improve energy efficiency adaptively to various scenarios. The proposed hierarchical rewarding structure with two reward functions can guide the DDPG agent to explore the optimal policy more efficiently and deeply and is well-suited to achieve targeted adjustment according to the vehicle operating modes to cope with rapidly changing scenarios. The optimality and adaptability of the new strategy are evaluated by standard and real-world driving data. The results show that the proposed strategy leads to an improvement of 8.11% in energy efficiency compared with the conventional DDPG strategy, and its energy-saving performance can reach up to 93.87% of the capacity of a dynamic programming (DP) based energy management strategy on the comprehensive driving condition II.