Stable value iteration for two-player zero-sum game of discrete-time nonlinear systems based on adaptive dynamic programming

Abstract

In this paper, a stable value iteration (SVI) algorithm is developed to solve discrete-time two-player zero-sum game (TP-ZSG) for nonlinear systems based on adaptive dynamic programming (ADP). In the SVI algorithm, both optimality and stability of nonlinear systems are considered with proofs given. First, an iterative ADP algorithm is presented to obtain the approximate optimal solutions by solving Hamilton–Jacobi–Isaacs (HJI) equation. Second, a range of the discount factor is shown, which guarantees HJI equation serving as a Lyapunov equation. Moreover, we prove that if the iteration number reaches a given number, then the iterative control inputs make the closed-loop system asymptotic stable. Third, in order to improve the practicability of the developed stability condition, a simple criteria is established based on Lyapunov stability theory. Neural networks (NNs) are used to approximate the system states, the value function, the control and disturbance inputs. Finally, simulation results are given to illustrate the performance of the developed optimal control method.