Optimal control for continuous-time Markov jump singularly perturbed systems : A hybrid reinforcement learning scheme

Abstract

This article discusses the adaptive optimal control problem for continuous-time Markov jump singularly perturbed systems with unknown system dynamics. First, the subsystems transformation technique is introduced to reconstruct the system with stochastic jump characteristics, yielding a set of parallel subsystems. Next, under the framework of reinforcement learning, an offline model-based hybrid iteration algorithm is developed to approximate the solution of the full-order coupled algebraic Riccati equations. Following this, to escape from the system model constraints, an online model-free hybrid iteration algorithm is introduced and relevant convergence proof is given. Compared to traditional value iteration and policy iteration algorithms, the hybrid iteration algorithm has ideal convergence rate and eliminates the requirement of initial stabilizing control policy. At the end, the availability of the online algorithm is demonstrated by means of an operational amplifier circuit model as a simulation example.