Reinforcement Learning Based Solution to Two-player Zero-sum Game Using Differentiator

Abstract

In this paper, a synchronous adaptive learning algorithm based on reinforcement learning (RL) is proposed for the solution to twoplayer zero-sum games for partially-unknown systems. To approximate the unknown drift dynamics required to solve the Hamilton-Jacobi-Isaacs equation, one feasible method is to employ a first-order robust exact differentiator (RED) to obtain the estimations of the state derivatives, and then the estimation of the unknown drift dynamics can be obtained se- quentially with the known input disturbance dynamics. An actor- critic-disturbance neural network (NN) structure is established to approximate the optimal control policy, value function and disturbance policy, respectively. An online synchronous tuning algorithm is proposed for the three NNs applying the RL technique and the designed first-order RED. The proposed method can guarantee that the optimum can be reached in the worst case of disturbance and the closed-loop system can be stabilized by applying Lyapunov theorem. Finally, the effectiveness of the presented scheme is demonstrated by two linear and nonlinear simulation examples.