Neuro‐observer based online finite‐horizon optimal control for uncertain non‐linear continuous‐time systems

Abstract

In this study, within framework of adaptive dynamic programming (ADP), a neuro-observer based online optimal control solution is developed for the finite-horizon optimal control problem of uncertain non-linear continuous-time systems. First, a neuro-observer is designed to estimate system states from the uncertain system without knowledge of system drift dynamics. Then based on the observed states, an online finite-horizon optimal control scheme is proposed by a single critic network to approximate the solution of the time-varying Hamilton–Jacobi–Bellman (HJB) equation and obtain the optimal control. In the design, a novel observer–critic architecture is presented for implementing the control scheme by using two neural networks (NNs): an observer NN is used to learn the uncertain system dynamics and a critic NN is employed not only to approximate the solution of time-varying HJB equation, but also to ensure the terminal constraint is satisfied. Besides, an additional adjusting term is introduced by Lyapunov theory in critic NN weight tuning algorithm, which relaxes the requirement for an initial stabilising control. Moreover, uniform ultimate bounded stability of the closed-loop system is guaranteed via Lyapunov's direct method. Finally, simulation results are shown to demonstrate the effectiveness of the proposed approach.