Cooperative energy management and eco-driving of plug-in hybrid electric vehicle via multi-agent reinforcement learning

Abstract

The advanced cruise control system has expanded the energy-saving potential of the hybrid electric vehicle (HEV). Despite this, most energy-saving researches for HEV either only optimize the energy management strategy (EMS) or integrate eco-driving through a hierarchically optimized assumption that optimizes EMS and eco-driving separately. Such kinds of approaches may lead to sub-optimal results. To fill this gap, we design a multi-agent reinforcement learning (MARL) based optimal energy-saving strategy for HEV, achieving a cooperative control on the powertrain and car-following behaviors to minimize the energy consumption and keep a safe following distance simultaneously. Specifically, a plug-in HEV model is regarded as the research object in this paper. Firstly, the HEV energy management problem in the car-following scenario is decomposed into a multi-agent cooperative task into two subtasks, each of which can conduct interactive learning through cooperative optimization. Secondly, the energy-saving strategy is designed, called the independent soft actor–critic, which consists of a car-following agent and an energy management agent. Finally, the performance of velocity tracking and energy-saving are validated under different driving cycles. In comparison to the state-of-the-art hierarchical model predictive control (MPC) strategy, the proposed MARL method can reduce fuel consumption by 15.8% while ensuring safety and comfort.