Intelligent Energy Management Strategy Based on an Improved Reinforcement Learning Algorithm With Exploration Factor for a Plug-in PHEV

Abstract

An intelligent energy management strategy (EMS) based on an improved Reinforcement Learning (RL) algorithm is developed to enhance the adaptability of the EMS and to further improve the fuel efficiency of a Plug-in Parallel Hybrid Electric Vehicle (PHEV). Both the numerical model and the energy management strategy of a plug-in PHEV are described. The improved RL with Q-learning algorithm is implemented to acquire the optimal control strategies for improving fuel economy. The Markov Chain is employed to calculate the Transition Probability Matrix of the required power. A Kullback-Leibler (KL) divergence rate is designed to activate the update of EMS, when a new corresponding driving cycle is expected. An Exploration Factor (EF) is proposed to overcome the disadvantages of the normal RL algorithm in convergence rate and reward cost evaluation. The diverse KL divergence rates are examined to seek optimal solutions. The normal-RL strategy, rule-based strategy, and dynamic programming strategy are implemented as benchmark strategies to verify the effectiveness of the proposed strategy. The validation results indicate that the improved RL algorithm with EF makes it possible to promote the EMS capable of significantly improving the energy efficiency of a plug-in PHEV.