Reinforcement learning based power management integrating economic rotational speed of turboshaft engine and safety constraints of battery for hybrid electric power system

Abstract

Hybrid electric power system (HEPS) with turboshaft engine is a promising solution for the land and air vehicle, and the power management strategy (PMS) is the key to obtaining better performance of HEPS. In this paper, a reinforcement learning (RL)-based PMS integrating economic rotational speed (ERS) of turboshaft engine and safety constraints-based variable action space (SC-VAS) approach is proposed. First, an efficient algorithm based on the turbine performance map for calculating ERS is proposed, with low complexity which is 5.5% of the conventional algorithm. Second, based on the ERS feature, the SC-VAS approach is presented to further optimize the action space to prevent the discharging/charging power and state of charge of the battery from violating the safety constraints. Comparison results show that with no violation of constraints of battery, the convergence speed of RL agent incorporating the SC-VAS approach increases by 5 times, and the size of the optimized Q table decreases to 21.9% of that of the basic Q table. The proposed PMS with the ERS feature and SC-VAS approach can bring a 4.29% reduction in the fuel consumption under an air-land driving condition. Moreover, the results of the hardware-in-the-loop experiment demonstrate the real-time performance of the proposed strategy.