Based on Q-Learning Optimal Tracking Control Schemes for Linear Itô Stochastic Systems With Markovian Jumps

Abstract

In this brief, a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning algorithm is designed to solve the optimal tracking control problem (OTCP) for stochastic system with Markovian jump. The problem is formulated by stochastic augmented system composed of the dynamic system and the reference trajectory system and then using ideas from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning to provide a solution. Namely, a critic neural network (NN) is used to estimate the optimal cost function while an actor NN is used to approximate the optimal controller. Moreover, both the requirements of transition probability and system matrix are avoided via the designed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning algorithm. Finally, we apply the designed algorithm in a single-area power system to track continuous sinusoidal waveforms, and the simulation result is provided to show the effectiveness and applicability of the algorithm.