Energy management of hybrid electric bus based on deep reinforcement learning in continuous state and action space

Abstract

Energy management is a fundamental task in hybrid electric vehicle community. Efficient energy management of hybrid electric vehicle is challenging owning to its enormous search space, multitudinous control variables and complicated driving conditions. Most existing methods apply discretization to approximate the continuous optimum in real driving conditions, which results in relatively low performance with the discretization error and curse of dimensionality. We introduce a novel energy management strategy with a deep reinforcement learning framework Actor-Critic to address these challenges. Actor-Critic uses a deep neural network, named as actor network, to directly output continuous control signals. Another deep neural network, named as critic network, evaluates the control signals generated by the actor network.The actor and critic neural network are trained by reinforcement learning from self-play in a continuous action space. Several comprehensive experiments are conducted in this paper, the proposed method surpasses discretization-based strategies by directly optimizing in the continuous space, which improves energy management performance while blackucing computation load. The simulation results indicate that the AC achieve the optimal energy distribution in comparison with the discretization-based strategies, especially surpassing the existing baseline DP by 5.5%, 2.9%, 9.5% in CTUDC, WVUCITY and WVUSUB in one-tenth of the computational cost.