Trilayer Stackelberg game for nonlinear systems using adaptive dynamic programming

Abstract

This paper considers a trilayer Stackelberg game problem for nonlinear system with three players. A novel performance function is defined for each player, which depends on the coupling relationships with the other two players. The coupled Hamilton–Jacobi–Bellman (HJB) equations are built from the performance functions, and the optimal control polices of three players are obtained based on the Bellman’s principle of optimality. Because of the nonlinearity and coupling characteristics, a policy iteration (PI) algorithm with a three-layer decision-making framework is developed to online learn the coupled HJB equations. In order to implement the algorithm, we construct a critic-action neural network (NN) structure and design a NN approximation-based iteration algorithm. Finally, a simulation example is presented to verify the effectiveness of the proposed method.