A novel controllable bias reinforcement learning method for distributed automatic generation control integrated with large-scale electric vehicles

Abstract

Due to the introduction of large-scale renewable energy resources and flexible load access, strong random disturbances will bring to the current integrated energy system. The traditional control methods fall short in adequately addressing the frequency regulation needs of distributed systems. From the perspective of distributed automatic generation control (AGC), an optimal coordinated control method is proposed based on reinforcement learning. The proposed method is developed to mitigate the estimation bias of the action-values via the minimum action-value from multiple action-value estimates. Further, the bias of action-value estimates can be adjusted from positive to negative to enhance the control accuracy. Then this method uses behaviour utility trace to achieve adaptive adjustment of the exploration rate, preventing repetitive and unbalanced exploration. Simulation experiments have been performed on the two-area LFC model integrated with electric vehicles and the multi-area integrated energy system model to verify the effectiveness of the proposed control method.