Reinforcement Learning Based on Equivalent Consumption Minimization Strategy for Optimal Control of Hybrid Electric Vehicles

Abstract

Hybrid electric vehicles, operated by engines and motors, require an energy management strategy to achieve competitive fuel economy performance. The equivalent consumption minimization strategy is a well-known algorithm that can be employed for the energy management of hybrid electric vehicles, based on the concept of the equivalent cost of fossil fuels and electric battery energy. However, in the equivalent consumption minimization strategy approach, a parameter called the equivalent factor should be determined to obtain the optimal control policy. In this study, reinforcement learning based approaches are proposed to determine the equivalent factor. First, we show that the equivalent factor can be indirectly extracted from the reinforcement learning results, using the control action from reinforcement learning for the specific driving cycle. In addition, a novel approach that combines reinforcement learning and the equivalent consumption minimization strategy is proposed, where the equivalent factor is determined based on the interaction between the reinforcement learning agent and driving environment, while the control input is decided by the equivalent consumption minimization strategy based on the determined equivalent factor. A model-based reinforcement learning method is used, and the proposed method is validated for vehicle simulation using a parallel hybrid electric vehicle. The simulation results show that the proposed method can achieve a near-optimal solution, which is close to the global solution obtained with the dynamic programming approach (96.7% compared to dynamic programming result in average), and improved performance of 4.3% in average compared with the existing adaptive equivalent consumption minimization strategy.