Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

Abstract

The coordinated optimization problem of the electricity-gas-heat integrated energy system (IES) has the characteristics of strong coupling, non-convexity, and nonlinearity. The centralized optimization method has a high cost of communication and complex modeling. Meanwhile, the traditional numerical iterative solution cannot deal with uncertainty and solution efficiency, which is difficult to apply online. For the coordinated optimization problem of the electricity-gas- heat IES in this study, we constructed a model for the distributed IES with a dynamic distribution factor and transformed the centralized optimization problem into a distributed optimization problem in the multi-agent reinforcement learning environment using multi-agent deep deterministic policy gradient. Introducing the dynamic distribution factor allows the system to consider the impact of changes in real-time supply and demand on system optimization, dynamically coordinating different energy sources for complementary utilization and effectively improving the system economy. Compared with centralized optimization, the distributed model with multiple decision centers can achieve similar results while easing the pressure on system communication. The proposed method considers the dual uncertainty of renewable energy and load in the training. Compared with the traditional iterative solution method, it can better cope with uncertainty and realize real- time decision making of the system, which is conducive to the online application. Finally, we verify the effectiveness of the proposed method using an example of an IES coupled with three energy hub agents.