Residential demand response online optimization based on multi-agent deep reinforcement learning

Abstract

Demand side management in the context of smart grids exploits the value stream of load flexibility, promoting stable and economic operation of the power grid. To address multi-uncertainties such as device characteristics, distributed renewable generation, and residential electricity demand, this paper proposes a model-free framework based on multi-agent deep reinforcement learning to achieve efficient online optimization of demand response strategies. Firstly, the demand side management is formulated as a partially observable Markov game. Then, a multi-agent soft actor-critic algorithm is proposed, combining a temporal electricity price feature extraction technology to improve learning efficiency. Reward correction mechanism is designed to avoid new charging and discharging peaks in the demand response. Finally, case studies demonstrate the effectiveness of the proposed method in reducing the electricity cost and aggregate peak demand of residents. The proposed method reduces the daily average cost by 21.40%, 8.09%, and 5.88% compared to MAPPO, MADDPG, and SUMADRL, and exhibits the highest training efficiency and scalability against the benchmarks.