A novel health-aware deep reinforcement learning energy management for fuel cell bus incorporating offline high-quality experience

Abstract

Data-driven intelligent energy management strategy (EMS) helps to further improve the performance and efficiency of fuel cell hybrid electric bus (FCHEB). However, most deep reinforcement learning (DRL) algorithms suffer from disadvantages such as overestimation and poor training stability, which limit the optimization effectiveness of the strategy. In addition, DRL-based EMSs tend to achieve good control only for the set optimization objectives and cannot be generalized to optimization objectives beyond the reward function. To solve the above problems, a novel health-aware DRL energy management for FCHEB is proposed in this paper. Firstly, based on the actual collected city bus driving cycles, a large amount of high-quality learning experience containing health-aware information is obtained through an advanced model predictive control strategy. Secondly, the state-of-the-art Twin Delayed Deep Deterministic policy gradient (TD3) algorithm is combined with offline high-quality learning experience to address the inherent shortcomings during the “cold-start phase” and to enhance the generalization capability of the proposed strategy. Finally, validation results showed that the proposed EMS improves training efficiency by 61.85% and fuel economy by 7.45%, extends fuel cell life by 4% and battery life by 19.4% compared to the conventional TD3-based EMS.