Adaptive Optimal Tracking Control for Uncertain Unmanned Surface Vessel via Reinforcement Learning

Abstract

This paper presents a reinforcement-learning (RL)-based adaptive optimal tracking control scheme for unmanned surface vessel (USV) in the presence of modeling uncertainties and time-varying disturbances. By backstepping technique, the virtual and actual controls are designed as the optimal solutions of the corresponding sub-systems such that the overall optimization control is realized for the USV system. To improve the robustness on modeling uncertainties and time-varying disturbances, we employ the neural network (NN) approximation to approximate the modeling uncertainties and construct a disturbance observer to compensate for the external disturbances. An adaptive optimal control algorithm is presented by employing the actor-critic RL algorithm. The theoretical analysis shows that the desired optimized performance can be obtained via Lyapunov stability theory. Simulation results demonstrate the effectiveness of the proposed adaptive optimal control scheme.