Reinforcement Learning-Based Energy Management System Enhancement Using Digital Twin for Electric Vehicles

Abstract

Compared to conventional engine-based powertrains, electrified powertrain exhibit increased energy efficiency and reduced emissions, making electrification a key goal for the automotive industry. For a vehicle with hybrid energy storage system, its performance and lifespan are substantially affected by the energy management system. Reinforcement learning-based methods are gaining popularity in vehicle energy management, but most of the literature in this area focus on pure simulation while hardware implementation is still limited. This paper introduces the digital twin methodology to enhance the Q-learning-based energy management system for battery and ultracapacitor electric vehicles. The digital twin model can exploit the bilateral interdependency between the virtual model and the actual system, which improves the control performance of the energy management system. The physical model is established based on a hardware-in-the-loop simulation platform. In addition, battery degradation is also considered for prolonging the battery lifespan to reduce the operating cost. The validation results of the trained reinforcement learning agent illustrate that the digital twin-enhanced Q-learning energy management system improves the energy efficiency by 4.36% and the battery degradation is reduced by 25.28%.