Differential Game Problem Research Articles

Non-zero sum differential game for stochastic Markovian jump systems with partially unknown transition probabilities

This paper focuses on the non-zero sum differential game problem for Markovian jump systems with partially unknown transition probabilities. Firstly, a suboptimal control problem is studied by the free-connection weighting matrix method, and then the non-zero sum differential game problem is investigated on this basis. Several sufficient conditions for the existence of ε-suboptimal control strategy and ε-suboptimal Nash equilibrium strategies are provided, and their explicit expressions are designed. Moreover, the precise form for the upper bound of the cost function is also given. To facilitate the calculation, all conditions are converted into the corresponding equivalent linear matrix inequalities or bilinear matrix inequalities form. Finally, two numerical examples are utilized to demonstrate the effectiveness of the main results.

A game problem of pursuit-evasion with nth order differential equations describing players' dynamics

In this paper, we study a pursuit-evasion differential game problem in the Hilbert space Dynamics of countable number of pursuers and evader expressed as nth-order differential equations with geometric constraints on the control functions of the players. The game terminates at a given fixed time which is denoted by ϑ. The game's payoff is the infimum of the distances between the evader and pursuers at the time ϑ. According to the rule of the game, pursuers try to minimise the distance to the evader and the evader tries to maximises it. We found value of the game and constructed players' optimal strategies.

Linear-quadratic non-zero sum differential game for mean-field stochastic systems with asymmetric information

In this study, we consider a class of asymmetric information linear-quadratic non-zero sum stochastic differential game problems. The system dynamics are governed by a forward linear mean-field stochastic differential equation and the cost functional is quadratic. In order to facilitate some practical applications, the mean-field terms for the state process and control process are considered in both the system dynamics and cost functional. Using the classical calculus of variation and dual methods, the open-loop Nash equilibrium point can be expressed by introducing an auxiliary mean-field forward backward stochastic differential equation, which comprises one forward and two backward components. Due to some Riccati equations and ordinary differential equations that possess unique solutions, the Nash equilibrium point can also be represented in feedback form for several special cases under asymmetric information. The corresponding filtering equations are derived, and the existence and uniqueness of the solutions are proved. An investment problem in finance is discussed to demonstrate the good performance of the theoretical results.

Mixed Linear Quadratic Stochastic Differential Leader-Follower Game with Input Constraint

This paper investigates a mixed leader-follower differential games problem, where the model involves two players with the same hierarchy in decision making and each player has two controls which act as a leader and a follower, respectively. Specifically, we solve a follower problem with unconstrained controls and obtain the corresponding Nash equilibrium. Then a leader problem with constrained controls is tackled and a pair of optimal constrained controls are presented by a projection mapping. Furthermore, the control weights are allowed to be singular. In this case, we first investigate the uniform convexity of the cost functional whose corresponding states are fully-coupled forward-backward stochastic differential equation. After that, the minimizing sequence of solutions with non-degenerate control weights are constructed to study the weak convergence of the corresponding cost functionals. Finally, two examples are addressed for non-singular and singular cases, respectively.

Simple motion pursuit differential game problem of many players with integral and geometric constraints on controls function.

We study a simple motion pursuit differential game of many pursuers and one evader in a Hilbert space $l_{2}$. The control functions of the pursuers and evader are subject to integral and geometric constraints respectively. Duration of the game is denoted by positive number $\theta $. Pursuit is said to be completed if there exist strategies $u_{j}$ of the pursuers $P_{j}$ such that for any admissible control $v(\cdot)$ of the evader $E$ the inequality $\|y(\tau)-x_{j}(\tau)\|\leq l_{j}$ is satisfied for some $ j\in \{1,2, \dots\}$ and some time $\tau$. In this paper, sufficient conditions for completion of pursuit were obtained. Consequently strategies of the pursuers that ensure completion of pursuit are constructed.

Time-inconsistent linear-quadratic non-zero sum stochastic differential games with random jumps

ABSTRACT We study a kind of time-inconsistent linear-quadratic non-zero sum stochastic differential game problems with random jumps. The time-inconsistency arises from the presence of a quadratic term of the expected state and a state-dependent term, as well as the time-dependent weight of each term in the cost functionals. We define the time-consistent Nash equilibrium point for this kind of problems and establish a general sufficient condition for it through a flow of forward-backward stochastic differential equations with random jumps. In the situation of one-dimensional state and deterministic coefficients, a Nash equilibrium point is given explicitly by some flows of Riccati-like and linear ordinary differential equations. We apply the truncation method to obtain the existence and uniqueness of solutions to them for a specific case. Moreover, an investment and consumption problem is solved and some numerical examples are further provided to illustrate the application of our theoretic results.

On a class of non-zero-sum stochastic differential dividend games with regime switching

This paper investigates a class of non-zero-sum stochastic differential game problems between two insurance companies. The surplus process of each company is modeled by a Brownian motion where drift and volatility depend on the continuous-time Markov regime switching process. Both companies have the option of paying dividends. The objective is to maximize the expected discount utility of surplus relative to a reference point for each insurer, the gains brought by the insurer’s own dividend payout, and the losses incurred by the dividend payment of the competitor. The gains and losses are proportional to the amount of corresponding dividend payment. To find the optimal dividend policy, we relate this singular control problem to a stopping game. Further, we prove the link based on the verification theorem and show that value functions of the non-zero-sum stochastic differential problem can be derived by integrating value functions of a stopping game. Finally, we apply our results to the case with two regimes and the case without regime switching. Numerical examples are also provided.

Minimization of energy costs for UAV management in a conflict task

This article considers the method of developing an evader control strategy in the non-linear differential pursuit-evasion game problem. It is assumed that the pursuer resorts to the most probable control strategy in order to capture the evader and that at each moment the evader knows its own and the enemy’s physical capabilities. This assumption allows to bring the game problem down to the problem of a unilateral evader control, with the condition of reaching a saddle point not obligatory to be fulfilled. The control is realised in the form of synthesis and additionally ensures that the requirements for bringing the evader to a specified area with terminal optimization of certain state variables are satisfiedt. The solution of this problem will significantly reduce the energy losses for controlling an unmanned vehicle, the possible effect is to save 15-20 % of fuel with a probability of 0.98, to solve the problem of chasing the enemy.

Branching improved Deep Q Networks for solving pursuit-evasion strategy solution of spacecraft

<p style='text-indent:20px;'>With the continuous development of space rendezvous technology, more and more attention has been paid to the study of spacecraft orbital pursuit-evasion differential game. Therefore, we propose a pursuit-evasion game algorithm based on branching improved Deep Q Networks to obtain a space rendezvous strategy with non-cooperative target. Firstly, we transform the optimal control of space rendezvous between spacecraft and non-cooperative target into a survivable differential game problem. Next, in order to solve this game problem, we construct Nash equilibrium strategy and test its existence and uniqueness. Then, in order to avoid the dimensional disaster of Deep Q Networks in the continuous behavior space, we construct a TSK fuzzy inference model to represent the continuous space. Finally, in order to solve the complex and timeconsuming self-learning problem of discrete action sets, we improve Deep Q Networks algorithm, and propose a branching architecture with multiple groups of parallel neural Networks and shared decision modules. The simulation results show that the algorithm achieves the combination of optimal control and game theory, and further improves the learning ability of discrete behaviors. The algorithm has the comparative advantage of continuous space behavior decision, can effectively deal with the continuous space chase game problem, and provides a new idea for the solution of spacecraft orbit pursuit-evasion strategy.</p>

Robust optimal investment and reinsurance for an insurer with inside information

This paper studies a robust optimal investment–reinsurance problem for an insurer who possesses inside information on the financial market and the insurance business under model uncertainty. The insurer’s surplus process and the risky asset process in the financial market are assumed to be correlated jump diffusion processes with random coefficients. The inside information is modelled by a general random variable related to the future realizations of the surplus process and the risky asset process. Under the criterion of expected utility maximization, the optimal investment–reinsurance problem with inside information is transformed into an adapted stochastic control problem. By incorporating Knightian uncertainty into the model, we establish an adapted stochastic differential game problem with a nonstandard performance functional, which aims to select the robust optimal investment–reinsurance strategy for the insurer with inside information. The stochastic maximum principles are adopted to derive a characterization of the optimal strategy. Some interesting special cases are discussed and the corresponding optimal strategies are derived.

Event-triggered distributed zero-sum differential game for nonlinear multi-agent systems using adaptive dynamic programming

In this paper, to reduce the computational and communication burden, the event-triggered distributed zero-sum differential game problem for multi-agent systems is investigated. Firstly, based on the Minimax principle, an adaptive event-triggered distributed iterative differential game strategy is derived with an adaptive triggering condition for updating the control scheme aperiodically. Then, to implement this proposed strategy, the solution of coupled Hamilton–Jacobi–Isaacs​ (HJI) equation is approximated by constructing the critic neural network (NN). In order to further relax the restrictive persistent of excitation (PE) condition, a novel PE-free updating law is designed by using the experience replay method. Then, the distributed event-triggered nonlinear system is expressed as an impulsive dynamical system. After analyzing the stability, the developed strategy ensures the uniformly ultimately bounded (UUB) of all the closed-loop signals. Moreover, the minimal intersample time is proved to be lower bounded, which avoids the infamous Zeno behavior. Finally, the simulation results show that the number of controller update is reduced obviously, which saves the computational and communication resources.

Output feedback adaptive dynamic programming for linear differential zero-sum games

This paper addresses the problem of finding optimal output feedback strategies for solving linear differential zero-sum games using a model-free approach based on adaptive dynamic programming (ADP). In contrast to their discrete-time counterparts, differential games involve continuous-time dynamics and existing ADP approaches to their solutions require access to full measurement of the internal state. This difficulty is due to the fact that direct translation of the discrete-time output feedback ADP results requires derivatives of the input and output measurements, which is generally prohibitive in practice. This work aims to overcome this difficulty and presents a new embedded filtering based observer approach towards designing output feedback ADP algorithms for solving the differential zero-sum game problem. Two output feedback ADP algorithms based respectively on policy iteration and value iteration are developed. The proposed scheme is completely online in nature and works without requiring information of the system dynamics. In addition, this work also addresses the excitation bias problem encountered in output feedback ADP methods, which typically requires a discounting factor for its mitigation. We show that the proposed scheme is bias-free, and therefore, does not require a discounting factor. It is shown that the proposed algorithms converge to the solution obtained by solving the game algebraic Riccati equation. Two numerical examples are demonstrated to validate the proposed scheme.

Partial Information Stochastic Differential Games for Backward Stochastic Systems Driven by Lévy Processes

In this paper, we consider a partial information two-person zero-sum stochastic differential game problem, where the system is governed by a backward stochastic differential equation driven by Teugels martingales and an independent Brownian motion. A sufficient condition and a necessary one for the existence of the saddle point for the game are proved. As an application, a linear quadratic stochastic differential game problem is discussed.

A Differential Game Problem of Many Pursuers and One Evader in the Hilbert Space $$\ell_2$$

In this paper, we investigate a differential game problem of multiple number of pursuers and a single evader with motions governed by a certain system of first-order differential equations. The problem is formulated in the Hilbert space $$\ell_2,$$ with control functions of players subject to integral constraints. Avoidance of contact is guaranteed if the geometric position of the evader and that of any of the pursuers fails to coincide for all time t. On the other hand, pursuit is said to be completed if the geometric position of at least one of the pursuers coincides with that of the evader. We obtain sufficient conditions that guarantees avoidance of contact and construct evader’s strategy. Moreover, we prove completion of pursuit subject to some sufficient conditions. Finally, we demonstrate our results with some illustrative examples.

A Purusit Differential Game Problem on a Closed Convex Subset of a Hilbert Space

We study a pursuit differential game problem with finite number of pursuers and one evader on a nonempty closed convex subset of the Hilbert space l2. Players move according to certain first order ordinary differential equations and control functions of the pursuers and evader are subject to integral constraints. Pursuers win the game if the geometric positions of a pursuer and the evader coincide. We formulated and prove theorems that are concern with conditions that ensure win for the pursuers. Consequently, wining strategies of the pursuers are constructed. Furthermore, illustrative example is given to demonstrate the result.

Numerical Solution of Open-Loop Nash Differential Games Based on the Legendre Tau Method

In this paper, an efficient implementation of the Tau method is presented for finding the open-loop Nash equilibrium of noncooperative nonzero-sum two-player differential game problems with a finite-time horizon. Regarding this approach, the two-point boundary value problem derived from Pontryagin’s maximum principle is reduced to a system of algebraic equations that can be solved numerically. Finally, a differential game arising from bioeconomics among firms harvesting a common renewable resource is included to illustrate the accuracy and efficiency of the proposed method and a comparison is made with the result obtained by fourth order Runge–Kutta method.

Robust event-triggered game-based attitude control for on-orbit assembly

On-orbit assembly is important in future space mission. It is a key process to adjust the attitude of substructures before assembling them into the main structure. In this paper, an event-triggered robust differential game is developed for multiple microsatellites fixed on one substructure to control the attitude of the substructure subject to disturbance. By taking disturbance as a special player to game with coordinated microsatellites, the attitude regulation problem is formulated as a robust differential game problem so that each microsatellite can independently calculate its own worst-case control strategy to reject disturbance. Considering the limited computation ability, a set of coupled algebraic equations is derived to obtain analytical expressions of Nash equilibrium strategies of microsatellites. Simultaneously, an event triggering threshold is designed by using the state error to avoid continuous communication and computation of mulitple microsatellites, and Zeno behaviour can be effectively avoided. Numerical simulations verify the effectiveness and stability of the event-triggered robust differential game method for the attitude regulation of the substructure.

Differential game learning approach for multiple microsatellites takeover of the attitude movement of failed spacecraft

Takeover of the attitude control function of a failed spacecraft suffering from fuel exhaustion or actuator failures enables recycling of on-board valuable reusable payloads. Microsatellites can provide cost-efficient ways for the attitude takeover control through coordination. Differential games are used to study the individual optimal decision problem, where each player optimizes their local performance index function to obtain the control policy, and the games predefined global objective can be achieved. In this paper, the failed spacecraft attitude takeover control problem is transformed into a multi-microsatellite differential game problem. First, the multi-microsatellite differential game model is established, the performance index function is designed for each microsatellite, and the mathematical description of the multi-microsatellite differential game problem is realized. Second, the Hamilton-Jacobi (HJ) equations are provided and solved through the single neural network (NN) based policy iteration (PI) algorithm to learn the multi-microsatellite game equilibrium control strategies. Finally, numerical simulations are carried out to validate the effectiveness of the multi-microsatellite differential game learning method. The results have shown that the predefined global objective of the takeover of the attitude control of the failed spacecraft can be realized through the approximate game equilibrium control strategies of multiple microsatellites.

A kind of non-zero sum mixed differential game of backward stochastic differential equation

This paper is concerned with a non-zero sum mixed differential game problem described by a backward stochastic differential equation. Here the term “mixed” means that this game problem contains a deterministic control v_{1} of Player 1 and a random control process v_{2} of Player 2. By virtue of the classical variational method, a necessary condition and an Arrow’s sufficient condition for the mixed stochastic differential game problem are presented. A linear–quadratic mixed differential game problem is discussed, and the corresponding Nash equilibrium point is explicitly expressed by the solution of mean-field forward–backward stochastic differential equation. The most distinguishing feature, compared with the existing literature, is that the optimal state process of the linear–quadratic game satisfies a linear mean-field backward stochastic differential equation. Finally, a home mortgage and wealth management problem is given to illustrate our theoretical results.

Load-Frequency Control of Multi-Area Power System Based on the Improved Weighted Fruit Fly Optimization Algorithm

With the development and application of large-scale renewable energy sources, the electric power grid is becoming huge and complicated; one of the most concerning problems is how to ensure coordination between a large number of varied controllers. Differential games theory is used to solve the problem of collaborative control. However, it is difficult to solve the differential game problem with constraints by using conventional algorithm. Furthermore, simulation models established by existing research are almost linear, which is not conducive to practical engineering application. To solve the above problem, we propose a co-evolutionary algorithm based on the improved weighted fruit fly optimization algorithm (IWFOA) to solve a multi-area frequency collaborative control model with non-linear constraints. Simulation results show that the control strategy can achieve system control targets, and fully utilize the various characteristics of each generator to balance the interests of different areas. Compared with a co-evolutionary genetic algorithm and a collaborative multi-objective particle swarm optimization algorithm, the co-evolutionary algorithm based on the IWFOA has a better suppression effect on the frequency deviation and tie-line power deviation caused by the disturbance and has a shorter adjustment time.