Enabling cross-type full-knowledge transferable energy management for hybrid electric vehicles via deep transfer reinforcement learning

Abstract

Deep reinforcement learning (DRL) now represents an emerging artificial intelligence technology to develop energy management strategies (EMSs) for hybrid electric vehicles (HEVs). However, developing specific DRL-based EMSs for different HEVs is currently a laborious task. Therefore, we design a transferable optimization framework crossing types of HEVs to expedite the development of DRL-based EMSs. In this framework, a novel enhanced twin delayed deep deterministic policy gradient (E-TD3) algorithm is first formulated, and then a deep transfer reinforcement learning (DTRL) method is designed by incorporating transfer learning into DRL. After that, a full-knowledge transfer method based on E-TD3 and DTRL is innovatively proposed. To assess the efficacy of the designed method, an E-TD3-based EMS of a light HEV is pre-trained to be the existing source EMS whose all learned knowledge is then transferred to be reused for a hybrid electric bus (HEB) to facilitate the acquisition of the target new EMS. Simulation results demonstrate that, in the designed transferable framework, the development cycle of the HEB's EMS can be shortened by 90.38 % and the fuel consumption can be saved by 6.07 %. This article provides a practical method to reuse existing DRL-based EMSs for the rapid development of new EMSs across HEV types.