Multiobjective Intelligent Energy Management for Hybrid Electric Vehicles Based on Multiagent Reinforcement Learning

Abstract

This paper proposes a multi-objective energy management strategy (EMS) based on multi-agent reinforcement learning (MARL) for a hybrid electric vehicle equipped with an engine-generator set (EGS) and a hybrid energy storage system (HESS, consisting of a battery and ultracapacitor). First, besides improving fuel economy, maintaining battery state of charge (SOC), reducing battery degradation and constraint on ultracapacitor SOC are also taken into consideration, formulating multi-objective energy management. Then, the problem is solved using MARL which combines game theory and reinforcement learning. In this framework, EGS and HESS are viewed as two intelligent agents respectively, and their interactions are described as a general-sum stochastic game. Following the principle of MARL, the two agents can learn the optimal control policy which guarantees the Nash equilibrium of multiple objectives, thus achieving a satisfactory balance among them. In the simulation, the MARL-based EMS is compared with single-agent reinforcement learning (SARL) which ignores the relations of different agents, and dynamic programming (DP) which integrates various targets into a single cost function with weight coefficients. The simulation results verify the superiority of the proposed EMS in optimizing multiple objectives.