Reinforcement Learning Based Aircraft Controller Enhanced By Gaussian Process Trim Finding

Abstract

Abstract This work presents mathematical and practical frameworks for designing deep deterministic policy gradient (DDPG) flight controllers for fixed-wing aircraft. The aim is to design reinforcement learning (RL) flight controllers and accelerate training by substituting the six degrees-of-freedom aircraft models with linear time-invariant (LTI) dynamic models. The initial validation flight tests of the DDPG RL flight controller exhibited poor performance. Post-flight test investigation revealed that the unsatisfactory performance of the RL flight controller could be attributed to the high reliance of the LTI model on accurate control trim values and the substantial errors observed in the predicted trim values generated by the engineering-level dynamic analysis software. A complementary real-time learning Gaussian process (GP) regression was designed to mitigate this critical shortcoming of the LTI-based RL flight controller. The GP estimates and updates the trim control surfaces using observed flight data. The GP regression method incorporates real-time corrections to the trim control surfaces to enhance the performance of the flight controller. Flight test validation was repeated, and the results show that the RL controller, bolstered by the GP trim-finding algorithm, can successfully control the aircraft with excellent tracking performance.