Multi-objective explainable smart dispatch for integrated energy system based on an explainable MO-RL method

Abstract

With the rapid development of renewable energy penetration and the transition to high-efficiency, low-carbon-footprint energy systems, renewable-based heat-electricity integrated energy systems (RHE-IES) have become a viable method. The dispatch and operation scheduling of the RHE-IES is more complex than before because a well-operated RHE-IES should be economically favorable, environmentally friendly, and feasible with respect to security constraints. However, it is not easy to balance these trade-offs with uncertainties. In this work, a multi-objective hierarchical deep reinforcement learning (H-DRL) method is proposed to address the dispatch of RHE-IES, and it is also explainable due to the design of the algorithmic structure, the decoupling of rewards, and the analysis of feature importance. First, the dispatch and operation scheduling problem of the RHE-IES is formulated considering economic, environmental, and security constraints and is converted to a Markov decision process. Second, a novel H-DRL method is proposed with a multi-critic, single-actor structure, where each objective is handled by a designated deep critic network with decoupled rewards, and the action value functions of different objectives are hierarchically optimized by the actor network. H-DRL can be trained in a comprehensive environment offline and provide fast online decisions considering the uncertainties of renewable generation. Third, a shapley additive explanations method is carried out utilizing the explainable nature of the H-DRL structure and the loss function design. The simulation results and case studies show the effectiveness of the proposed method in terms of balancing the tradeoffs of different objectives, overcoming uncertainties and enhancing AI interpretability.