Energy Management Strategy for Fuel Cell/Battery/Ultracapacitor Hybrid Electric Vehicles Using Deep Reinforcement Learning With Action Trimming

Abstract

As for fuel cell hybrid electric vehicle equipped with battery (BAT) and ultracapacitor (UC), its dynamic topology structure is complex and different characteristics of three power sources induce challenges in energy management for fuel economy, power sources lifespan, and dynamic performance of the vehicle. In this paper, an energy management strategy (EMS) based on a hierarchical power splitting structure and deep reinforcement learning (DRL) is proposed. In the higher layer strategy of the proposed EMS, the UC is employed to supply peak power and recover braking energy through the adaptive filter based on fuzzy control. Then, the integrated DRL and equivalent consumption minimization strategy framework is proposed to optimize the power allocation of fuel cell (FC) and BAT in the lower layer, to ensure the highly efficient operation of FC and reduce hydrogen consumption. And the action trimming based on heuristic technique is proposed to further restrain the adverse effect of sudden peak power on FC lifespan. The simulation results show the proposed EMS can make the output of FC smoother, improve its working efficiency to alleviate the stress of BAT, and increase by 14.8% compared with the Q-learning strategy in fuel economy under WLTP driving cycle. Meanwhile, the obtained results under UDDSHDV show fuel economy of the proposed EMS can reach dynamic programming (DP) benchmark level of 89.7 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> .