A soft actor-critic-based energy management strategy for electric vehicles with hybrid energy storage systems

Abstract

Listen

Translate article

With the rapid development of machine learning, deep reinforcement learning (DRL) algorithms have been widely applied to energy management strategies (EMSs) of hybrid vehicles recently. However, current DRL algorithms show the drawbacks of slower convergence rate, brittle training stability, and dissatisfactory optimization effects. In this research, a new DRL algorithm, i.e. the soft actor-critic (SAC) is applied to the EMS of an electric vehicle (EV) with a hybrid energy storage system (HESS). Particularly, the knowledge extracted from the dynamic programming (DP), which is regarded as the benchmark for control methods, is adopted to improve the control performance of the SAC-based EMS and the parallel computing is applied to accelerate the training process. Results of this research indicate that the SAC-based EMS decreases the HESS energy loss by 8.75% and 6.09% compared to the deep Q-network (DQN) and deep deterministic policy gradient (DDPG)-based EMSs respectively while it narrows the gap with the DP-based EMS to 5.19%. Additionally, as the same actor-critic framework, the convergence rate of the SAC-based EMS is faster than that of the DDPG-based EMS by 205.66%. Furthermore, the adaptability validation results present that the SAC-based EMS outperforms the DQN and DDPG-based EMSs in energy saving up to 32.24%.