Accelerated Value Iteration for Nonlinear Zero-Sum Games with Convergence Guarantee

Abstract

In this paper, an accelerated value iteration (VI) algorithm is established to solve the zero-sum game problem with convergence guarantee. First, inspired by the successive over relaxation theory, the convergence rate of the iterative value function sequence is accelerated significantly with the relaxation factor. Second, the convergence and monotonicity of the value function sequence are analyzed under different ranges of the relaxation factor. Third, two practical approaches, namely the integrated scheme and the relaxation function, are introduced into the accelerated VI algorithm to guarantee the convergence of the iterative value function sequence for zero-sum games. The integrated scheme consists of the accelerated stage and the convergence stage, and the relaxation function can adjust the value of the relaxation factor. Finally, including the autopilot controller, the fantastic performance of the accelerated VI algorithm is verified through two examples with practical physical backgrounds.