Scalable Designs for Reinforcement Learning-Based Wide-Area Damping Control

Abstract

This article discusses how techniques from reinforcement learning (RL) can be exploited to transition to a model-free and scalable wide-area oscillation damping control of power grids. We present two control architectures with distinct features. Performing full-dimensional RL control designs for any practical grid would require an unacceptably long learning time and result in a dense communication architecture. Our designs avoid the curse of dimensionality by employing ideas from model reduction. The first design exploits time-scale separation in the generator electro-mechanical dynamics arising from coherent clustering, and learns a controller using both electro-mechanical and non-electro-mechanical states while compensating for the error in incorporating the latter through the RL loop. The second design presents an output-feedback approach enabled by a neuro-adaptive observer using measurements of only the generator frequencies. The controller exhibits an adaptive behavior that updates the control gains whenever there is a notable change in the loads. Theoretical guarantees for closed-loop stability and performance are provided for both designs. Numerical simulations are shown for the IEEE 68-bus power system model.