Energy management for a power-split hybrid electric bus via deep reinforcement learning with terrain information

Abstract

Due to the high mileage and heavy load capabilities of hybrid commercial vehicles, energy management becomes crucial in improving their fuel economy. In this paper, terrain information is systematically integrated into the energy management strategy for a power-split hybrid electric bus based on a deep reinforcement learning approach: the deep deterministic policy gradient algorithm. Specially, this energy management method is improved and capable of searching optimal energy management strategies in a discrete-continuous hybrid action space, which, in this work, consists of two continuous actions for the engine and four discrete actions for powertrain mode selections. Additionally, a Critic network with dueling architecture and a pre-training stage ahead of the reinforcement learning process are combined for efficient strategy learning with the adopted algorithm. Assuming the current terrain information was available to the controller, the deep reinforcement learning based energy management strategy is trained and tested on different driving cycles and simulated terrains. Simulation results of the trained strategy show that reasonable energy allocation schemes and mode switching rules are learned simultaneously. Its fuel economy gap with the baseline strategy using dynamic programming is narrowed down to nearly 6.4% while reducing the times of engine starts by around 76%. Further comparisons also indicate approximately 2% promotion in fuel economy is contributed by the incorporation of terrain information in this learning-based energy management. The main contribution of this study is to explore the inclusion of terrain information in a learning-based energy management method that can deal with large hybrid action spaces.