Incentive learning-based energy management for hybrid energy storage system in electric vehicles

Abstract

Deep reinforcement learning has emerged as a promising candidate for online optimal energy management of multi-energy storage vehicles. However, how to ensure the adaptability and optimality of the reinforcement learning agent under realistic driving conditions is still the main bottleneck. To enable the reinforcement learning agent to efficiently learn the optimal power allocation strategies under diverse driving conditions, this paper proposes an incentive learning-based energy management strategy for battery-supercapacitor electric vehicles to minimize the battery capacity loss cost and power loss cost. First, an incentive reward function based on supercapacitor state-of-charge and vehicle acceleration is proposed for proximal policy optimization-based energy management strategy, which can stimulate the agent to learn for optimal power allocation policy under high load power conditions quickly. Second, a random sampling-based velocity transfer probability surface is constructed for pre-training to guarantee strategy optimality under unfamiliar driving cycles. Third, the generalized advantage estimation and layer normalization of neural networks are incorporated to improve the learning convergence. Results show that the proposed method can reduce the above costs by 5.8%–13.8% and 11.7%–38.8% compared with existing deep reinforcement learning methods under the pre-training driving cycle and test driving cycles, respectively, which yields closer results to offline dynamic programming.