Optimal and Adaptive Control Design Using Recursive Least Square with a New Exponential Forgetting Factor

Abstract

AbstractThis paper presents a Reinforcement Learning application using a recursive least squares (RLS) with an exponential forgetting (EF) factor to solve the Discrete Linear Quadratic Regulator problem. Temporal Difference learning based RLS algorithm is implemented to find a kernel matrix of the action value function (or Q-function) approximated by neural network. Based on the EF RLS, a New Exponential Forgetting (New EF) factor algorithm is developed by adding a covariance term to the forgetting factor to prevent the estimator windup problem. Numerical simulations on a fixed-wing aircraft are performed to show the effectiveness of the new EF RLS.KeywordsReinforcement learningQ-LearningOptimal controlAdaptive controlRecursive least squaresForgetting factor