Reinforcement Q-Learning Based Current Control for Grid-Connected Three-Phase Inverters with Unknown System Dynamics

Abstract

This paper proposes a Q-learning-based state feedback suboptimal controller to solve the current control problem of three-phase grid-connected LCL coupled inverters with unknown circuit parameters. In practice, the circuit parameters of the inverters will change obviously for reasons such as calculation errors, external environment and operation aging, which makes the dynamics of the inverter system become unknown. With the circuit’s model of the LCL inverters and the reference current dynamic, an augmented system is constructed and a discounted performance function is formulated as the optimal objective of current control which transform it into a H∞ tracking control. Using Q-learning algorithm with model-free characteristics, a current controller is proposed in which an iterative reinforcement learning (RL) algorithm is embedded. Simulations are presented to verify the valid of the proposed control scheme, where especially the results shows that it can keep excellent control performance under the condition of inverter parameter mutation and grid voltage distortion.