RL-based adaptive control for a class of non-affine uncertain stochastic systems with mismatched disturbances

Abstract

This paper investigates the reinforcement learning (RL) adaptive tracking control design problem for a class of mismatched stochastic nonlinear systems with non-affine structure. The stochastic system studied in this paper is more generally representative due to the presence of non-affine inputs, internal uncertainties, and mismatched external disturbances. Firstly, in order to solve the non-affine structure of stochastic systems, an extended stochastic differential equation is constructed. Based on the actor–critic framework, by generating reinforcement signals, the network is stimulated to evolve more quickly towards the desired direction, while compensating internal uncertainties and induced uncertainties in the stochastic system. Furthermore, aiming at the approximation errors and external disturbances, while satisfying stochastic stability, the adaptive laws for disturbances boundary estimator are established with the usage of higher powers of tracking errors. As a result, a novel non-affine adaptive tracking controller has been proposed by integrating RL, disturbances boundary estimation, and dynamic surface method. Through the stability analysis, it is proved that all closed-loop signals are bounded in probability, and the system output converges to a small neighborhood near the desired trajectory. Numerical simulations demonstrate the effectiveness and superiority of the proposed controller.