Double Deep Reinforcement Learning-Based Energy Management for a Parallel Hybrid Electric Vehicle With Engine Start–Stop Strategy

Abstract

Committed to optimizing the fuel economy of hybrid electric vehicles (HEVs), improving the working conditions of the engine, and promoting research on deep reinforcement learning (DRL) in the field of energy management strategies (EMSs), this article first proposed a DRL-based EMS combined with a rule-based engine start–stop strategy. Moreover, considering that both the engine and the transmission are controlled components, this article developed a novel double DRL (DDRL)-based EMS, which uses a deep Q-network (DQN) to learning the gear-shifting strategy and uses a deep deterministic policy gradient (DDPG) to control the engine throttle opening, and the DDRL-based EMS realizes the multiobjective synchronization control by different types of learning algorithms. After off-line training, the simulation result of the online test shows that the fuel consumption gaps of the proposed DRL- and DDRL-based EMSs are −0.55% and 2.33% compared to that of the deterministic dynamic programming (DDP)-based EMS by overcoming some inherent flaws of DDP, respectively. The computational efficiency has been significantly improved, and the average output time per action is 0.91 ms. Therefore, the control strategy that combines learning- and rule-based controls and the multiobjective control strategies both have the potential to ensure optimization and real-time efficiency.