Deep Reinforcement Learning for Direct Load Control in Distribution Networks

Abstract

Direct load control enables load aggregators in distribution networks to remotely curtail customers’ appliances during peak time periods. This paper proposes a direct load control algorithm for residential customers, while accounting for the uncertainties in the customers’ discomfort from curtailing their demand as well as the operational constraints imposed by the distribution network. We model the load control problem as a Markov decision process (MDP). Solving such an MDP is challenging due to the ac power flow equations and the unknown dynamics of the system states (i.e., price, demand, and customer’s discomfort). We develop a deep reinforcement learning algorithm based on the actor-critic method that enables the load aggregator to consider the distribution network constraints and the consequences of its past decisions to update the neural network parameters for the policy and value function without any knowledge of the system dynamics. Simulations are performed on an IEEE 85-bus test feeder with 59 households. Results show that the load aggregator learns to reduce the peak load by 16.7%, while taking into account the distribution network constraints. Also, the customers’ cost is decreased by 26.6% on average; thereby reaching a win-win outcome.