Optimal formation tracking control based on reinforcement learning for multi-UAV systems

Abstract

This paper investigates the problem of optimal formation tracking control for multi-UAV systems with model uncertainty and external disturbances. Firstly, by combining the sliding mode method and a neural network, an adaptive sliding mode controller is derived that counteracts the effects of modeling uncertainty and external disturbance. Subsequently, the optimal formation tracking control problem of the original system is then converted to the optimal control problem of a nominal system, and an actor–critic reinforcement learning framework is built using adaptive neural network identifiers to recursively approximate the total optimal policy and cost function. The Lyapunov analysis method shows that the stability of the closed-loop system and the convergence of the estimation weights for the actor–critic network are guaranteed. Additionally, a formation tracking using virtual experiment platform for multi-UAV systems are constructed based on the Robot Operating System (ROS) and Gazebo simulator. Finally, virtual-reality experiments is performed to demonstrate the effectiveness of the proposed control scheme.