High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning

Abstract

As a hybrid electric vehicle (HEV) key control technology, intelligent energy management strategies (EMSs) directly affect fuel consumption. Investigating the robustness of EMSs to maximize the advantages of energy savings and emission reduction in different driving environments is necessary. This article proposes a soft actor-critic (SAC) deep reinforcement learning (DRL) EMS for hybrid electric tracked vehicles (HETVs). Munchausen reinforcement learning (MRL) is adopted in the SAC algorithm, and the Munchausen SAC (MSAC) algorithm is constructed to achieve lower fuel consumption than the traditional SAC method. The prioritized experience replay (PER) is proposed to achieve more reasonable experience sampling and improve the optimization effect. To enhance the “cold start” performance, a dynamic programming (DP)-assisted training method is proposed that substantially improves the training efficiency. The proposed method optimization result is compared with the traditional SAC and deep deterministic policy gradient (DDPG) with PER through the simulation. The result shows that the proposed strategy improves both fuel consumption and possesses excellent robustness under different driving cycles.