Finite-Horizon Near Optimal Design of Nonlinear Two-Player Zero-Sum Game in Presence of Completely Unknown Dynamics

Abstract

In this paper, a novel time-based finite-horizon near optimal design is proposed for nonlinear two-player zero-sum game with completely unknown system dynamics. First, an online neural network (NN) identifier is developed to learn the dynamics of the nonlinear continuous-time system. Then, a novel online NN-based approximator is proposed to estimate the time-varying solution, or referred to as value function, of the Hamilton–Jacobi–Isaacs equation in a forward-in-time manner. Subsequently, by utilizing the estimated time-varying value function from the NN-based approximator and system dynamics from the online NN identifier, the near optimal strategies for nonlinear two-player zero-sum game are derived. To deal with time-varying property of value function, a NN structure with constant weights and time-varying activation functions is considered, and a suitable NN update law is proposed based on normalized gradient descent approach. In order to ensure the stability and satisfy terminal cost at the fixed final time, two novel additional terms, one corresponding to terminal cost, and the other to stabilize the nonlinear two-player zero-sum game are introduced to the update law of the NN-based approximator. Eventually, standard Lyapunov theory is utilized to verify the uniformly ultimately boundedness of the closed-loop system, and the simulation results demonstrate the effectiveness of proposed scheme.