Integral reinforcement learning-based angular acceleration autopilot for high dynamic flight vehicles

Abstract

During the synthesis of acceleration autopilots for high dynamic flight vehicles (HDFV), autopilots with feedback of angular acceleration (AFAA) have become more perspective with stringent requirements on response speed and high maneuverability, compared with autopilots with feedback of angular rate (AFAR). Integral reinforcement learning (IRL) method has now proved to be an effective technique for adaptive optimal control of partially unknown nonlinear systems. In this paper, a novel data-driven IRL algorithm with “actor-critic” structure is proposed for HDFV utilizing AFAA. As an advanced model-free approach, actor-critic based IRL algorithm learns optimal behaviors by observing the real-time responses from the environment under the action of nonoptimal control policies. Instead of solving algebraic Riccati equation directly, the control policy updates online via the solution of proposed IRL Bellman equation with sensed quantities. Numerical simulation is carried out to validate the effectiveness of proposed online IRL-based angular acceleration autopilot for HDFVs. Besides, the tracking performance under different wave commands, the robustness against parameter uncertainties and the noise attenuation capacity between classical optimal tracking approach and proposed IRL method are analyzed for AFAR and AFAA, respectively. Simulation results show that, angular acceleration autopilot with proposed integral RL algorithm possesses better tracking performance against various disturbances.