A novel model-free robust saturated reinforcement learning-based controller for quadrotors guaranteeing prescribed transient and steady state performance

Abstract

For the purpose of improving the performance of trajectory tracking for quadrotors with the control input saturation, a novel model-free saturated prescribed performance reinforcement learning framework is proposed in the presence of the model uncertainties, nonlinearities and external disturbances. In this paper, saturation functions are employed to deal with input saturation, and the actuator's saturation nonlinearity is compensated by an intelligent method to decrease the saturation effects. Moreover, the prescribed performance control is utilized to ensure an adjustable transient and steady state response for the tracking errors. Besides, adaptive robust controllers are introduced to handle the effects of external disturbances online. A novel controller is proposed in collaboration with a reinforcement learning method based on actor-critic neural networks. The actor neural network is employed to estimate nonlinearities, actuator saturation nonlinearity, and model uncertainties, and the critic neural network is applied to estimate the reinforcement signals, which regulates the control action of the actor neural network online. The proposed actor-critic-based control structure benefits from a model-free calculation and only depends on the measurable signals of the closed-loop system. This freedom from system dynamics leads to a significant low computational load for the controller and, therefore, the proposed control method is computationally cost-effective. The adaptive robust controllers and the proposed actor-critic structures are trained online, and the convergence behavior of their learning laws is investigated in the course of stability examination. For the proof of stability, Lyapunov's direct method is used to show that all error variables of the closed-loop nonlinear control system are uniformly ultimately bounded. Finally, simulations along with some quantitative comparisons verify the efficiency and usefulness of the proposed control scheme.