Optimal power-split of hybrid energy storage system using Pontryagin’s minimum principle and deep reinforcement learning approach for electric vehicle application

Abstract

The battery supercapacitor hybrid energy storage system (HESS) based electric vehicles (EVs) require an efficient online energy management system (EMS) to enhance the battery life. The indirect optimal control method, like Pontryagin’s minimum principle (PMP), is gaining attention for its inherent instantaneous optimization and computational simplicity. However, determining the costate, non-casual variable of PMP for a given driving scenario is challenging and significantly affects the optimal solution of minimizing the battery degradation. In this study, we propose an online hybrid EMS by combining PMP and deep reinforcement learning (RL) to precisely predict the optimal costate and simultaneously minimize the battery degradation in a battery supercapacitor HESS assisted EV. Also, we propose an analytical approach to estimate the initial costate to create the costate action space of RL, which could also be used to estimate the initial costate guess for similar applications. We validate the performance of the proposed EMS under driving cycle uncertainties and untrained driving cycles via simulation. The results demonstrate the effectiveness of deep RL for optimal costate estimation to satisfy the charge sustainability of the supercapacitor. The predicted costate maintains the supercapacitor state-of-charge in the desired range while minimizing battery current for uncertain driving profiles. Further, we have compared the battery energy depletion in the proposed EMS and an offline optimal EMS to study the feasibility of the proposed intelligent EMS. The proposed EMS outperforms the offline EMS, achieving a significant improvement of 900 charge–discharge cycles.