A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle

Abstract

Energy management strategy (EMS) has a huge impact on the energy efficiency of hybrid electric vehicles (HEVs). Recently, fast-growing number of studies have applied different deep reinforcement learning (DRL) based EMS for HEVs. However, a unified performance review benchmark is lacking for most popular DRL algorithms. In this study, 13 popular DRL algorithms are applied as HEV EMSs. The reward performance, computation cost, and learning convergence of different DRL algorithms are discussed. In addition, HEV environments are modified to fit both discrete and continuous action spaces. The results show that the stability of agent during the learning process of continuous action space is more stable than discrete action space. In the continuous action space, SAC has the highest reward, and PPO has the lowest time cost. In discrete action space, DQN has the lowest time cost, and FQF has the highest reward. The comparison among SAC, FQF, rule-based, and equivalent consumption minimization strategies (ECMS) shows that DRL EMSs run the engine more efficiently, thus saving fuel consumption. The fuel consumption of FQF is 10.26% and 5.34% less than Rule-based and ECMS, respectively. The contribution of this paper will speed up the application of DRL algorithms in the HEV EMS application.