Optimal Adaptive Output Regulation of Uncertain Nonlinear Discrete-time Systems using Lifelong Concurrent Learning

Abstract

This paper addresses neural network (NN) based optimal adaptive regulation of uncertain nonlinear discrete-time systems in affine form using output feedback via lifelong concurrent learning. First, an adaptive NN observer is introduced to estimate both the state vector and control coefficient matrix, and its NN weights are adjusted using both output error and concurrent learning term to relax the persistency excitation (PE) condition. Next, by utilizing an actor-critic framework for estimating the value functional and control policy, the critic network weights are tuned via both temporal different error and concurrent learning schemes through a replay buffer. The actor NN weights are tuned using control policy errors. To attain lifelong learning for performing effectively during multiple tasks, an elastic weight consolidation term is added to the critic NN weight tuning law. The state estimation, regulation, and the weight estimation errors of the observer, actor and critic NNs are demonstrated to be bounded when performing tasks by using Lyapunov analysis. Simulation results are carried out to verify the effectiveness of the proposed approach on a Vander Pol Oscillator. Finally, extension to optimal tracking is given briefly.