Confidence-aware reinforcement learning for energy management of electrified vehicles

Abstract

The reliability of data-driven techniques, such as deep reinforcement learning (DRL) frequently diminishes in scenarios beyond their training environments. Despite DRL-based energy management strategies (EMS) having gained great popularity in optimizing the energy economy of electrified vehicles (EVs), their performance degradation in untrained contexts has not received adequate attention. This study presents a confidence-aware EMS designed to mitigate this problem and thereby enhance the overall EMS functionality. Firstly, a deep ensemble model-based uncertainty evaluation method is developed for devising a confidence assessment mechanism to measure the reliability of DRL actions. On this basis, a confidence-aware DRL-based strategy is proposed, wherein a knowledge-driven approach replaces DRL actions in instances of low confidence, aiming to improve overall performance. For validation, a fuel cell EV with complex energy flow was used as the testbed, and our proposed EMS was trained with the aim of optimizing fuel cell system energy consumption, battery longevity, and capacity maintenance. Both the confidence mechanism and the proposed EMS were evaluated using real-world driving profiles. Results suggest the established confidence mechanism accurately represents the DRL’s performance across different situations. In addition, the proposed EMS outperforms existing DRL-based EMS by 4.0% in hydrogen economy without compromising other objectives. The comprehensive architecture of the proposed amalgamation of data-driven and knowledge-driven methodologies can be effectively tailored to analogous energy management problems, thereby contributing to advancements in related fields.