Optimized Backstepping Tracking Control Using Reinforcement Learning for Quadrotor Unmanned Aerial Vehicle System

Abstract

In this article, an optimized tracking control scheme is studied for the quadrotor unmanned aerial vehicle (QUAV) system by combining both reinforcement learning (RL) and the backstepping technique. The RL aims to overcome the difficulty coming from solving the Hamilton–Jacobi–Bellman (HJB) equation, and it is performed via iterating both critic and actor each other, where the critic is for improving the control performance and the actor is for executing the control behavior. In mathematics, a QUAV system is composed of two connected subsystems that are, respectively, modeled by the translational and rotational dynamic equations, which are coupled via a rotation matrix; hence, the optimized tracking scheme is composed of two interconnected individual controls corresponding to the position and attitude, respectively. To achieve the two optimized position and attitude controls, the RL is constructed on the basis of the neural network (NN) approximation of the HJB equation’s solution by utilizing NN’s outstanding function approximation ability. Particularly, the position control is accomplished by introducing an intermediate control because the translational dynamic is an underactuated system. Since the proposed RL algorithm is significantly simple in comparison with the published methods, the optimized QUAV control can be easily executed in practical applications. Finally, the results are demonstrated by a Lyapunov stability analysis and a numerical simulation.