Deep Reinforcement Learning-Based Energy Management for Liquid Hydrogen-Fueled Hybrid Electric Ship Propulsion System

Abstract

This study proposed a deep reinforcement learning-based energy management strategy (DRL-EMS) that can be applied to a hybrid electric ship propulsion system (HSPS) integrating liquid hydrogen (LH2) fuel gas supply system (FGSS), proton-exchange membrane fuel cell (PEMFC) and lithium-ion battery systems. This study analyzed the optimized performance of the DRL-EMS and the operational strategy of the LH2-HSPS. To train the proposed DRL-EMS, a reward function was defined based on fuel consumption and degradation of power sources during operation. Fuel consumption for ship propulsion was estimated with the power for balance of plant (BOP) of the LH2 FGSS and PEMFC system. DRL-EMS demonstrated superior global and real-time optimality compared to benchmark algorithms, namely dynamic programming (DP) and sequential quadratic programming (SQP)-based EMS. For various operation cases not used in training, DRL-EMS resulted in 0.7% to 9.2% higher operating expenditure compared to DP-EMS. Additionally, DRL-EMS was trained to operate 60% of the total operation time in the maximum efficiency range of the PEMFC system. Different hydrogen fuel costs did not affect the optimized operational strategy although the operating expenditure (OPEX) was dependent on the hydrogen fuel cost. Different capacities of the battery system did not considerably change the OPEX.