Power Delivery Capability Improvement of Voltage Source Converters in Weak Power Grid Using Deep Reinforcement Learning with Continuous Action

Abstract

Voltage source converters (VSCs) are widely used in integrating renewable energy sources (RESs), electric vehicles, and energy storage systems to the utility grid. Conventionally, VSCs are controlled using decoupled vector technique with a designed proportional-integral (PI) controller. However, increasing penetration of renewable energy sources leads to a weak grid condition that is characterized by low short circuit ratio (SCR), low inertia, and poor reactive power control. Recent studies have demonstrated the limitations of the applicability of the conventional vector control-based PI method to VSC control. Specifically, the active power transfer capability of the VSC in weak grid conditions is limited to a portion of the converter-rated power. Also, the phase lock loop (PLL) losses grid synchronization as the VSC transfers maximum power under weak grid conditions. To mitigate these problems, a reinforcement learning (RL) based VSC control scheme is proposed. In this paper, a small-signal model of the VSC is developed, and the impact of the grid strength as well as the performance of the phase lock loop (PLL), is investigated. The effectiveness of the proposed controller is compared against the conventional PI-based vector control method in MATLAB/Simulink platform. The proposed control shows a better performance such that under very weak grid conditions, the PLL remains stable as the VSC transfer maximum power.