Optimal control of affine nonlinear continuous-time systems using an online Hamilton-Jacobi-Isaacs formulation

Abstract

Solving the Hamilton-Jacobi-Isaacs (HJI) equation, commonly used in H ∞ optimal control, is often referred to as a two-player differential game where one player tries to minimize the cost function while the other tries to maximize it. In this paper, the HJI equation is formulated online and forward-in-time using a novel single online approximator (SOLA)-based scheme to achieve optimal regulation and tracking control of affine nonlinear continuous-time systems. The SOLA-based adaptive approach is designed to learn the infinite horizon HJI equation, the corresponding optimal control input, and the worst case disturbance. A novel parameter tuning algorithm is derived which not only achieves the optimal cost function, control input, and the disturbance, but also ensures the system states remain bounded during the online learning. Lyapunov methods are used to show that all signals are uniformly ultimately bounded (UUB) while ensuring the approximated signals approach their optimal values with small bounded error. In the absence of OLA reconstruction errors, asymptotic convergence to the optimal signals is demonstrated, and simulation results illustrate the effectiveness of the approach.