Multi-objective reinforcement learning-based energy management for fuel cell vehicles considering lifecycle costs

Abstract

To balance the hydrogen consumption of fuel cell vehicle (FCV), the durability of the fuel cell (FC), and the life of the power battery (PB) to further reduce the whole lifecycle costs of FCV. A multi-objective reinforcement learning-based (MORL-based) energy management strategy (EMS) is proposed in this research. First, the composition mechanism of the FCV lifecycle costs is analyzed, and the equivalent hydrogen consumption model, FC durability degradation model, and PB life decay model are established; Then, a three-dimensional reward function is constructed by integrating the objectives of equivalent hydrogen consumption, FC durability degradation, and PB life decay. And the penalty terms coupled with the decay factors are introduced into the reward function to satisfy the mutual constraint characteristics between the PB and the FC system to ensure the stability of the MORL-based EMS; In addition, the prioritized experience replay technology is introduced into the MORL-based EMS to improve the learning efficiency and convergence of traditional deep Q network (DQN) algorithm; After that, the evaluation and target network of the embedded dueling network are introduced to solve the multi-objective overestimation problem encountered in the training process by generalizing the behavior learning in the presence of similar value behaviors; Finally, the performance of MORL-based EMS and DQN-based EMS is compared by numerical simulation under various driving cycles. The results show that the MORL-based EMS proposed in this paper has better convergence ability, adaptability, and lower lifecycle costs than the DQN-based EMS. In addition, the lifecycle costs of the MORL-based EMS can achieve a 99.2% control effect of the dynamic programming-based EMS.