Dyna algorithm-based reinforcement learning energy management for fuel cell hybrid engineering vehicles

Abstract

Fuel cell hybrid engineering vehicles utilize green energy with high energy conversion efficiency. Optimizing the utilization of hybrid energy storage systems is crucial for improving energy efficiency and promoting energy conservation. This study introduces an innovative energy management framework that employs reinforcement learning to regulate a fuel cell/battery hybrid engineering vehicle. This novel framework facilitates the system achieve optimal energy management, even when confronted with demanding and ever-changing operational scenarios. The proposed strategy integrates the concepts of the Dyna algorithm and the deep Q-network (DQN) algorithm. A working conditions predictive model is developed using the gated recurrent unit (GRU) neural network. The aim of this predictive model is to generate a virtual depiction of the environment and accelerate the process of model updating. The results from simulations and hardware-in-the-loop test illustrate that the predictive model exhibits effective prediction capabilities, enabling the development of an accurate environmental model even when limited sample data is available. The proposed strategy exhibits superior performance compared to the DQN and Q-Learning algorithms regarding adaptability, real-time optimization, training efficiency, convergence speed, and energy conservation. When interacting with the environment, our proposed learning algorithm exhibits a faster convergence rate. Moreover, this strategy achieves a fuel economy of 95 %, as determined by the dynamic programming-based method. The model-based deep reinforcement learning approach is practical and does not require extensive training. An added benefit of this energy management architecture is its use of a more accurate world model.