Reinforcement learning-based moving-target enclosing control for an unmanned surface vehicle in multi-obstacle environments

Abstract

This paper investigates the moving-target enclosing problem in multi-obstacle environments for an unmanned surface vehicle (USV) with complex unknown factors, including target dynamics, vehicle dynamics, and disturbances. A reinforcement learning-based moving-target enclosing control scheme is proposed to ensure collision-free behavior and bolster the enclosing capability. Specifically, an extended state observer is deployed to estimate the target dynamics. Leveraging the estimated data and control obstacle functions, a virtual safety control law is formulated to dynamically harmonize obstacle avoidance and target enclosing control. Then, a novel controller is constructed to track this control law utilizing a Nussbaum-type function in conjunction with actor–critic neural networks (NNs). The actor and critic NNs are employed to approximate unknown dynamics encapsulating vehicle dynamics and disturbances, and value function, respectively. The Nussbaum-type function is embedded to adaptively identify unknown inertia mass, regulating the control input of the USV with the actor NN online. The proposed scheme effectively decouples obstacle avoidance and target enclosing control and only relies on the measurable variables. A rigorous theoretical analysis is further employed to ensure the closed-loop stability of the USV system. Eventually, Simulations are demonstrated to validate the effectiveness and superiority of the proposed scheme for the USV in multi-obstacle environments.