The application of machine learning-based energy management strategy in a multi-mode plug-in hybrid electric vehicle, part II: Deep deterministic policy gradient algorithm design for electric mode

Abstract

Machine learning (ML)-based methods have attracted great attention in the multi-objective optimization problems, which is the key challenge in the energy management strategy (EMS) of the multi-power hybrid system. Our recently published research in this journal verified the effectiveness and feasibility of a Deep Deterministic Policy Gradient (DDPG)-based EMS in the charge-sustaining (CS) stage of a multi-mode plug-in hybrid vehicle (PHEV). However, the application of ML-based-EMS in the charge-depletion (CD) stage and the regenerative braking mode of PHEV are still missing. This study proposes a discrete-continuous hybrid actions-based hierarchical EMS to optimally distribute the dual-motor driving force in battery electric driving and regenerative braking. In the upper layer of EMS, DDPG is trained to learn the torque distribution principles of dual-motor operation to achieve better energy efficiency without losing dynamic performance. Meanwhile, the total recoverable braking torque is also determined by the upper layer EMS considering the braking demand, mechanical and electrical braking system conditions, vehicle safety, and the provisions of law. In the lower level of EMS, the driving mode is determined under the guidance of energy consumption optimization. The verified results show that the proposed EMS outperforms other deep reinforcement learning (DRL)-based hierarchical and non-hierarchical EMSs.