Online Reinforcement Learning-Based Adaptive Tracking Control of an Unknown Unmanned Surface Vehicle with Input Saturations

Abstract

In this paper, a novel online reinforcement learning-based optimal tracking control scheme is proposed for an unmanned surface vehicle (USV) in the presence of input saturations. To be specific, the saturation function can be expressed as a combination of a hyperbolic tangent function function and a bounded function which is encapsulated into lumped tracking error dynamics. An auxiliary design system is introduced to compensate for the nonlinear term arising from the input saturations. In order to derive a practically optimal solution, the exponent index function is defined to ensure that the long-term cumulative reward is bounded, moreover, the NNs-based actor-critic reinforcement learning framework is built to recursively approximate the totally optimal policy and cost function. Eventually, the closed-loop system stability and tracking accuracy can be guaranteed by theoretical analysis, subject to optimal cost. Simulation results and comprehensive comparisons on a prototype USV demonstrate remarkable effectiveness and superiority.