Stochastic Optimal CPS Relaxed Control Methodology for Interconnected Power Systems Using Q-Learning Method

Abstract

This paper presents the application and design of a novel stochastic optimal control methodology based on the Q-learning method for solving the automatic generation control (AGC) under the new control performance standards (CPS) for the North American Electric Reliability Council (NERC). The aims of CPS are to relax the control constraint requirements of AGC plant regulation and enhance the frequency dispatch support effect from interconnected control areas. The NERC’s CPS-based AGC problem is a dynamic stochastic decision problem that can be modeled as a reinforcement learning (RL) problem based on the Markov decision process theory. In this paper, the Q-learning method is adopted as the RL core algorithm with CPS values regarded as the rewards from the interconnected power systems; the CPS control and relaxed control objectives are formulated as immediate reward functions by means of a linear weighted aggregative approach. By regulating a closed-loop CPS control rule to maximize the long-term discounted...