Self-supervised reinforcement learning-based energy management for a hybrid electric vehicle

Abstract

Reinforcement learning is a new research hotspot in the energy management strategy. At present, some problems in the application of reinforcement learning to energy management control still exist, including sparse reward, convergence speed, generalization ability, etc. This paper proposes a self-supervised reinforcement learning method based on a Deep Q-learning approach for fuel-saving optimization of a plug-in hybrid electric vehicle (PHEV). First, a detailed vehicle powertrain model of the Prius is built. Second, we use the self-supervised learning model to enrich the reward function. The reward function consists of two parts: internal and external rewards. Finally, to prevent the self-supervised model from falling into the “self-good” situation, a reinforcement learning calibration method is proposed. The vehicle exploration method is more effective because of the enrichment of the reward function. Furthermore, following the characteristics of self-supervised learning, we have also constructed a new driving cycle to verify the generalization ability. Results show that our proposed deep reinforcement learning method based on self-supervised and learning calibration realizes faster training convergence and lower fuel consumption than the traditional policy, and its fuel economy can reach approximately the global optimum under our new driving cycle. • The method used in this can enhance the vehicle's ability to explore the environment. • Internal rewards were constructed using self-supervised learning methods. • The algorithm has generalization ability in energy management control strategy. • The reward function is corrected with the reinforcement learning calibration.