Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems

Abstract

With the integration of renewable energy, pumped storage, and new energy storage into multi-area integrated energy systems, the generation control of multi-area integrated energy systems is facing serious challenges. A differential evolution variable parameter vector multi-agent emotional deep Q network is proposed to increase the frequency regulation accuracy and convergence speed of multi-area integrated energy systems. The proposed control framework enhances the performance of artificial emotion by differential evolution and adaptive to the environment; the learning rates and action values of two deep Q networks are emotionalized separately by adaptive artificial emotion based on differential evolution; the action values of two deep Q networks are employed to generate commands for smart generation control through vector control. The proposed control framework is calculated in two-area and four-area integrated energy systems with China Southern Power Grid as the background. The numerical calculation results verify the best control performance and fastest convergence speed of the proposed control framework. The frequency deviations of the two cases are reduced by at least 7.44 % and 8.37 %, respectively; the convergence speed of the control framework in the two cases is increased by at least 2.70 % and 0.84 %, respectively; the power generation costs of the two cases are reduced by at least 305,370.9 $ and 460,186.6 $, respectively; the carbon emissions of the two cases are reduced by at least 10,940 kg and 11,610 kg, respectively.