Sample Efficient Reinforcement Learning With Domain Randomization for Automated Demand Response in Low-Voltage Grids

Abstract

Automateddemand response programs are being increasingly used to address voltage and congestion issues on the low-voltage distributed grid due to rapid proliferation of distributed energy resources and demand electrification. Data-driven methods, such as reinforcement learning (RL), can help realize these solutions in practice. However, these algorithms have their own limitations, including high sample complexity and limited capability to generalize in nonstationary settings. In this article, using actual data from residential buildings and distribution grids in Belgium and The Netherlands, we investigate the limits of state-of-the-art RL-based controllers in both centralized and decentralized settings. We also show that it is possible to considerably improve the performance of these RL controllers by making use of domain randomisation and transfer learning. With the proposed method, it is not necessary to have a high fidelity simulation of the system under consideration and we demonstrate this by considering varying degrees of misspecification. Our results show that the proposed technique improves on naively posed RL controllers even when the simulation model is misspecified, and helps minimize grid violations and energy losses, while avoiding the need for costly exploration. The proposed controllers can also address communication and single-point-of-failure issues.