Multi-agent deep reinforcement learning for joint dynamic conservation voltage reduction and Q-sharing in inverter-based autonomous microgrids

Abstract

Conservation voltage reduction (CVR) is implemented in power systems to mitigate power consumption in steady-state time scales. We propose dynamic-scale CVR (DCVR) as a potent solution to provide cost-effective frequency support in inverter-interfaced (micro) grids. The proposed DCVR reduces the voltage profile in dynamics and consequently power reduction helps to maintain instant production-consumption balance for dynamic frequency support. However, DCVR implementation in autonomous MGs (AMGs) faces critical control and stability problems. DCVR is implemented by controlling the voltage, which is a local variable, and makes it difficult to be realized through a decentralized structure. Besides, preserving accurate reactive power sharing (Q-sharing) through conventional droop controllers while employing the DCVR is another critical concern. In this light, this paper proposes a novel artificial intelligence (AI)-based decentralized control structure to implement the DCVR in AMGs for tackling the existing issues. The multi-agent deep reinforcement learning (DRL) model, with a deep Q-network (DQN) algorithm, is adopted to address the grid instabilities and inaccurate Q-sharing issues that arise due to incorporating the DCVR. The proposed method is able to handle the nonlinearity and complexity of the system while maintaining proper dynamic performance and AMG stability. Simulation results in MATLAB/Simulink prove the effectiveness of the proposed control method.