Nonzero-sum Game Problem Research Articles

Open-loop and closed-loop local and remote stochastic nonzero-sum game with inconsistent information structure

In this paper, the open-loop and closed-loop local and remote stochastic nonzero-sum game (LRSNG) problem is investigated. Different from previous works, the stochastic nonzero-sum game problem under consideration is essentially a special class of two-person nonzero-sum game problem, in which the information sets accessed by the two players are inconsistent. More specifically, both the local player and the remote player are involved in the system dynamics, and the information sets obtained by the two players are different, and each player is designed to minimise its own cost function. For the considered LRSNG problem, both the open-loop and closed-loop Nash equilibrium are derived. The contributions of this paper are given as follows. Firstly, the open-loop optimal Nash equilibrium is derived, which is determined in terms of the solution to the forward and backward stochastic difference equations (FBSDEs). Furthermore, by using the orthogonal decomposition method and the completing square method, the feedback representation of the optimal Nash equilibrium is derived for the first time. Finally, the effectiveness of our results is verified by a numerical example.

Dynamic event-triggered robust optimal tracking control for multi-player nonzero-sum games with mismatched uncertainties and asymmetric constrained inputs

In this paper, a novel dynamic event-triggered robust optimal tracking control method is proposed for multi-player nonzero-sum game problem with mismatched uncertainties and asymmetric constrained inputs. First, the original asymmetric constrained-input multi-player system is transformed into a new multi-player system with symmetric constrained inputs implemented by the state-space transformation technique, and an augmented system is established based on the new system and the desired trajectory. Next, an auxiliary augmentation system is constructed to handle the robust tracking control problem. Then, a critic neural network with a novel modified weight updating law is established for each player to solve the event-triggered coupled Hamilton–Jacobi equations, which relaxes the restrictions of initial admissible control and persistence of excitation condition. Furthermore, efficient communication and computing are obtained by the newly established dynamic event-triggered mechanism, and the Zeno behavior can be excluded by introducing an exponential term. The uniformly ultimately bounded properties of all signals are proved. Finally, two numerical examples verify the effectiveness of the designed method.

A Stochastic Control Approach for Constrained Stochastic Differential Games with Jumps and Regimes

We develop an approach for two-player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different types of constraints, which lead us to find real-valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with the stochastic maximum principle technique. Our application is an example of cooperation between a bank and an insurance company, which is a popular, well-known business agreement type called Bancassurance.

Fuzzy logic nonzero-sum game-based distributed approximated optimal control of modular robot manipulators with human-robot collaboration

In this paper, a fuzzy logic nonzero-sum game-based distributed approximated optimal control scheme is presented for modular robot manipulators (MRMs) with human-robot collaboration (HRC) tasks. The MRM dynamic model is formulated by using joint torque feedback (JTF) technique. Based on the differential game strategy, the optimal control problem of HRC task-oriented MRM systems is transformed into a nonzero-sum game problem of multiple robotic subsystems. By taking advantage of the adaptive dynamic programming (ADP) algorithm, the distributed approximate optimal control policy under HRC tasks is developed by a novel fuzzy logic nonzero-sum game manner for solving the coupled Hamilton–Jacobian (HJ) equation. The trajectory tracking error under HRC task of the closed-loop MRM system is proved to be ultimately uniformly bounded (UUB) using the Lyapunov theory. Finally, experiment results have been presented, which demonstrate the advantage and effectiveness of the developed method.

An event-triggered neural critic technique for nonzero-sum game design with control constraints

In this paper, an event-triggered neural critic learning algorithm is investigated to address constrained nonzero-sum game problems with discrete-time nonaffine dynamics. First, in order to ensure the saturation independence of two controllers in the nonzero-sum game problem, we adopt two different boundaries to constrain them respectively. Then, a novel triggering condition is designed to reduce the update times of the controllers, which achieves the purpose of less calculation. It is emphasised that the triggering condition is established based on the iteration of the time-triggered mechanism. Meanwhile, we prove that the real cost function possesses a predetermined upper bound, which realises the cost guarantee of the controlled system. In addition, we prove that the closed-loop system using the developed algorithm is asymptotically stable and that the system state and the sampling state are uniformly ultimately bounded during the process of training neural networks. Finally, two simulation examples are conducted to demonstrate the effectiveness of the proposed algorithm.

Reinforcement learning based proportional–integral–derivative controllers design for consensus of multi-agent systems

This paper develops a novel Proportional–Integral–Derivative (PID) tuning method for multi-agent systems with a reinforced self-learning capability for achieving the optimal consensus of all agents. Unlike the traditional model-based and data-driven PID tuning methods, the developed PID self-learning method updates the controller parameters by actively interacting with unknown environment, with the outcomes of guaranteed consensus and performance optimization of agents. Firstly, the PID control-based consensus problem of multi-agent systems is formulated. Then, finding the PID gains is converted into solving a nonzero-sum game problem, thus an off-policy Q-learning algorithm with the critic-only structure is proposed to update the PID gains using only data, without the knowledge of dynamics of agents. Finally, simulations are given to verify the effectiveness of the proposed method.

Event-triggered integral reinforcement learning for nonzero-sum games with asymmetric input saturation

In this paper, an event-triggered integral reinforcement learning (IRL) algorithm is developed for the nonzero-sum game problem with asymmetric input saturation. First, for each player, a novel non-quadratic value function with a discount factor is designed, and the coupled Hamilton–Jacobi equation that does not require a complete knowledge of the game is derived by using the idea of IRL. Second, the execution of each player is based on the event-triggered mechanism. In the implementation, an adaptive dynamic programming based learning scheme using a single critic neural network (NN) is developed. Experience replay technique is introduced into the classical gradient descent method to tune the weights of the critic NN. The stability of the system and the elimination of Zeno behavior are proved. Finally, simulation experiments verify the effectiveness of the event-triggered IRL algorithm.

Adaptive Dynamic Programming for Control: A Survey and Recent Advances

This article reviews the recent development of adaptive dynamic programming (ADP) with applications in control. First, its applications in optimal regulation are introduced, and some skilled and efficient algorithms are presented. Next, the use of ADP to solve game problems, mainly nonzero-sum game problems, is elaborated. It is followed by applications in large-scale systems. Note that although the functions presented in this article are based on continuous-time systems, various applications of ADP in discrete-time systems are also analyzed. Moreover, in each section, not only some existing techniques are discussed, but also possible directions for future work are pointed out. Finally, some overall prospects for the future are given, followed by conclusions of this article. Through a comprehensive and complete investigation of its applications in many existing fields, this article fully demonstrates that the ADP intelligent control method is promising in today’s artificial intelligence era. Furthermore, it also plays a significant role in promoting economic and social development.

Discrete-Time Multi-Player Games Based on Off-Policy Q-Learning

In this paper, an off-policy game Q-learning algorithm is proposed for solving linear discrete-time non-zero sum multi-player game problems. Unlike the existing Q-learning methods for solving the Riccati equation by on-policy learning approaches for multi-player games, an off-policy game Q-learning method is developed for achieving the Nash equilibrium of multiple players. To this end, first, a non-zero sum game problem is formulated, and the value function and the Q-function defined according to each-player individual performance index are rigorously proved to be linear quadratic forms. Then, based on the dynamic programming and Q-learning methods, an off-policy game Q-learning algorithm is developed to find the control policies for multi-player games, such that the Nash equilibrium is reached under the learned control policies. The merit of this paper lies in that the proposed algorithm does not require the system model parameters to be known a priori and fully utilizes measurable data to learn the Nash equilibrium solution. Moreover, there is no bias of Nash equilibrium solution when implementing the proposed off-policy game Q-learning algorithm even though probing noises are added to control policies for maintaining the persistent excitation condition. While bias of the Nash equilibrium solution could be produced if on-policy game Q-learning is employed. This is another contribution of this paper.

A class of nonzero-sum investment and reinsurance games subject to systematic risks

Recently, there have been numerous insightful applications of zero-sum stochastic differential games in insurance, as discussed in Liu et al. [Liu, J., Yiu, C. K.-F. & Siu, T. K. (2014). Optimal investment of an insurer with regime-switching and risk constraint. Scandinavian Actuarial Journal 2014(7), 583–601]. While there could be some practical situations under which nonzero-sum game approach is more appropriate, the development of such approach within actuarial contexts remains rare in the existing literature. In this article, we study a class of nonzero-sum reinsurance-investment stochastic differential games between two competitive insurers subject to systematic risks described by a general compound Poisson risk model. Each insurer can purchase the excess-of-loss reinsurance to mitigate both systematic and idiosyncratic jump risks of the inter-arrival claims; and can invest in one risk-free asset and one risky asset whose price dynamics follows the famous Heston stochastic volatility model [Heston, S. L. (1993). A closed-form solution for options with stochastic volatility with applications to bond and currency options. Review of Financial Studies 6, 327–343]. The main objective of each insurer is to maximize the expected utility of his terminal surplus relative to that of his competitor. Dynamic programming principle for this class of nonzero-sum game problems leads to a non-canonical fixed-point problem of coupled non-linear integral-typed equations. Despite the complex structure, we establish the unique existence of the Nash equilibrium reinsurance-investment strategies and the corresponding value functions of the insurers in a representative example of the constant absolute risk aversion insurers under a mild, time-independent condition. Furthermore, Nash equilibrium strategies and value functions admit closed forms. Numerical studies are also provided to illustrate the impact of the systematic risks on the Nash equilibrium strategies. Finally, we connect our results to that under the diffusion-approximated model by proving explicitly that the Nash equilibrium under the diffusion-approximated model is an -Nash equilibrium under the general Poisson risk model, thereby establishing that the analogous Nash equilibrium in Bensoussan et al. [Bensoussan, A., Siu, C. C., Yam, S. C. P. & Yang, H. (2014). A class of nonzero-sum stochastic differential investment and reinsurance games. Automatica 50(8), 2025–2037] serves as an interesting complementary case of the present framework.

A class of non-zero-sum stochastic differential investment and reinsurance games

In this article, we provide a systematic study on the non-zero-sum stochastic differential investment and reinsurance game between two insurance companies. Each insurance company’s surplus process consists of a proportional reinsurance protection and an investment in risky and risk-free assets. Each insurance company is assumed to maximize his utility of the difference between his terminal surplus and that of his competitor. The surplus process of each insurance company is modeled by a mixed regime-switching Cramer–Lundberg diffusion approximation process, i.e. the coefficients of the diffusion risk processes are modulated by a continuous-time Markov chain and an independent market-index process. Correlation between the two surplus processes, independent of the risky asset process, is allowed. Despite the complex structure, we manage to solve the resulting non-zero sum game problem by applying the dynamic programming principle. The Nash equilibrium, the optimal reinsurance/investment, and the resulting value processes of the insurance companies are obtained in closed forms, together with sound economic interpretations, for the case of an exponential utility function.

A Variational Formula for Nonzero-Sum Stochastic Differential Games of FBSDEs and Applications

A nonzero-sum stochastic differential game problem is investigated for fully coupled forward-backward stochastic differential equations (FBSDEs in short) where the control domain is not necessarily convex. A variational formula for the cost functional in a given spike perturbation direction of control processes is derived by the Hamiltonian and associated adjoint systems. As an application, a global stochastic maximum principle of Pontryagin’s type for open-loop Nash equilibrium points is established. Finally, an example of a linear quadratic nonzero-sum game problem is presented to illustrate that the theories may have interesting practical applications and the corresponding Nash equilibrium point is characterized by the optimality system. Here the optimality system is a fully coupled FBSDE with double dimensions (DFBSDEs in short) which consists of the state equation, the adjoint equation, and the optimality conditions.

Nash equilibrium point for one kind of stochastic nonzero-sum game problem and BSDEs

In this Note, we deal with one kind of stochastic nonzero-sum differential game problem for N players. Using the theory of backward stochastic differential equations and Malliavin calculus, we give the explicit form of a Nash equilibrium point. To cite this article: J.-P. Lepeltier et al., C. R. Acad. Sci. Paris, Ser. I 347 (2009).

BSDEs and risk-sensitive control, zero-sum and nonzero-sum game problems of stochastic functional differential equations

We deal with the risk-sensitive control, zero-sum and nonzero-sum game problems of stochastic functional differential equations. Using backward stochastic differential equations we show the existence of an optimal control and, a saddle-point and an equilibrium point for respectively the zero-sum and nonzero-sum games.

Properties of Nash solutions of a two-stage nonzero-sum game

This paper contains exact expressions for the complete class of uncountably many globally optimal affine Nasb equilibrium strategies for a two-stage two-person nonzero-sum game problem with quadratic objective functionals and with dynamic information for beth players. Existence conditions for each of these Nash equilibrium solutions are derived and it is shown that a recursive Nash solution is not necessarily globally optimal. Cost-uniqueness property of the derived Nash strategies is investigated and it is proven that the game problem under consideration admits a unique Nash cost pair if and only if it can be made equivalent to either a team problem or a zero-sum game. It is also shown that existence conditions of a globally optimal Nash solution will be independent of the parameters characterizing the nonuniques of the Nash strategies only if the game problem can be made equivalent to a team problem.