Policy Gradient-based Integral Reinforcement Learning for Optimal Control Design of Nonaffine Morphing Aircraft Systems

Abstract

An online model-free optimal control design strategy is proposed for general continuous-time nonlinear nonaffine systems using policy gradient-based integral reinforcement learning. In the case of the nonaffine system such as a morphing wing aircraft considering the morphing parameters as control effectors, general nonlinear control design method cannot be applied and solving the Hamilton-Jacobi-Bellman equation analytically is difficult. The proposed online optimal control algorithm is constructed based on the actor-critic structure using Q-function and policy gradient scheme and the integral reinforcement learning approach is used to develop the actor-critic parameter estimation for the continuous-time system. The closed-loop stability analysis for the designed method is presented. Through the proposed method, the optimal controller can be designed for the general nonaffine system, which has an advantage in terms of a computational issue for the complex system. Note that the entire dynamic model is not required. Simulation results demonstrate the effectiveness of the proposed scheme.