Supervised assisted deep reinforcement learning for emergency voltage control of power systems

Abstract

The increasing complexity of power systems makes existing deep reinforcement learning-based emergency voltage control methods face challenges in learning speed and data utilization efficiency. Meanwhile, the accumulated data containing expert experience and domain knowledge has not been fully utilized to improve the performance of the deep reinforcement learning methods. To address the above issues, a novel hybrid emergency voltage control method that combines expert experience and machine intelligence is proposed in this paper. Specifically, the expert experience in the off-line demonstration is extracted through a behavioral cloning model and the deep reinforcement learning method is applied to discover and learn new knowledge autonomously. A special supervised expert loss is designed to utilize the pre-trained behavioral cloning model to assist the self-learning process. The demonstration is dynamically updated during the training process such that the behavioral cloning model and the deep reinforcement learning model can facilitate each other continuously. Experiments are conducted on the open-source RLGC platform to validate the performance and the experimental results show that the proposed method can effectively improve the learning speed and the applicability of the model to different test situations.