Linear Quadratic Differential Games Research Articles

On the non-uniqueness of linear Markov perfect equilibria in linear-quadratic differential games: a geometric approach

On the non-uniqueness of linear Markov perfect equilibria in linear-quadratic differential games: a geometric approach

Policy iteration based cooperative linear quadratic differential games with unknown dynamics

This article investigates the Pareto optimality of infinite horizon cooperative linear quadratic (LQ) differential games by policy iteration technique where the system dynamics are partially or completely unknown. Firstly, the policy iteration algorithm for the approximate solutions of the corresponding algebraic Riccati equation (ARE) without any prior knowledge of the matrix parameters of the dynamic system is derived by collecting the input and state information of each player. Secondly, when the presented specific rank condition is satisfied, the convergence of the proposed algorithm is rigorously demonstrated by recursion. Moreover, the weighting approach is employed to obtain the Pareto optimal strategy and the Pareto optimal solutions on the basis of the convex optimization theory. Finally, simulation results are reported to verify the feasibility and correctness of the proposed theoretical results.

Linear Quadratic Nonzero-Sum Mean-Field Stochastic Differential Games with Regime Switching

Linear Quadratic Nonzero-Sum Mean-Field Stochastic Differential Games with Regime Switching

Existence of equilibrium in a dynamic supply chain game with vertical coordination, horizontal competition, and complementary goods

We consider supply chain competition and vertical coordination in a linear-quadratic differential game setting. In this setting, supply chains produce complementary goods and each of them includes a single manufacturer and a single retailer who coordinate their decisions through a revenue-sharing contract with a wholesale price and a fixed sales revenue share. We study a multiple leader-follower Stackelberg game where the manufacturers are the leaders and the retailers are the followers. Competition occurs at both levels of the supply chains. Retailers play Nash and compete in price; manufacturers also play Nash but they compete in choosing their production capacities by exploiting the equilibrium price decisions made by the retailers. We show that open-loop Nash equilibria exist when the manufacturers only receive a wholesale price (there are no longer exploiting the equilibrium price decision made by the retailers, however). When the manufacturers receive both a wholesale-price and a share of the retailers’ sales revenues, equilibria generally no longer exist. The non-existence of an equilibrium stems from the fact that the manufacturers’ instant profits are discontinuous functions of their production capacities. This discontinuity leads to a major technical difficulty in that one cannot apply standard optimal control approaches to study the equilibria of the dynamic game. Our results illustrate the possibility that competition between supply chains might not be sustainable when they sell complementary products and rely on a revenue-sharing agreement.

Model-Based Online Adaptive Inverse Noncooperative Linear-Quadratic Differential Games via Finite-Time Concurrent Learning

Model-Based Online Adaptive Inverse Noncooperative Linear-Quadratic Differential Games via Finite-Time Concurrent Learning

Identification methods for ordinal potential differential games

This paper introduces two new identification methods for linear quadratic (LQ) ordinal potential differential games (OPDGs). Potential games are notable for their benefits, such as the computability and guaranteed existence of Nash Equilibria. While previous research has analyzed ordinal potential static games, their applicability to various engineering applications remains limited. Despite the earlier introduction of OPDGs, a systematic method for identifying a potential game for a given LQ differential game has not yet been developed. To address this gap, we propose two identification methods to provide the quadratic potential cost function for a given LQ differential game. Both methods are based on linear matrix inequalities (LMIs). The first method aims to minimize the condition number of the potential cost function’s parameters, offering a faster and more precise technique compared to earlier solutions. In addition, we present an evaluation of the feasibility of the structural requirements of the system. The second method, with a less rigid formulation, can identify LQ OPDGs in cases where the first method fails. These novel identification methods are verified through simulations, demonstrating their advantages and potential in designing and analyzing cooperative control systems.

Analytical game strategies for active UAV defense considering response delays

In the realm of aerial warfare, the protection of Unmanned Aerial Vehicles (UAVs) against adversarial threats is crucial. In order to balance the impact of response delays and the demand for onboard applications, this paper derives three analytical game strategies for the active defense of UAVs from differential game theory, accommodating the first-order dynamic delays. The targeted UAV executes evasive maneuvers and launches a defending missile to intercept the attacking missile, which constitutes a UAV-Missile-Defender (UMD) three-body game problem. We explore two distinct operational paradigms: the first involves the UAV and the defender working collaboratively to intercept the incoming threat, while the second prioritizes UAV self-preservation, with independent maneuvering away from potentially sacrificial engagements. Starting with model linearization and order reduction, the Collaborative Interception Strategy (CIS) is first derived via a linear quadratic differential game formulation. Building upon CIS, we further explore two distinct strategies: the Informed Defender Interception Strategy (IDIS), which utilizes UAV maneuvering information, and the Unassisted Defender Interception Strategy (UDIS), which does not rely on UAV maneuvering information. Additionally, we investigate the conditions for the existence of saddle point solutions and their relationship with vehicle maneuverability and response agility. The simulations demonstrate the effectiveness and advantages of the proposed strategies.

Non-cooperative games to control learned inverter dynamics of distributed energy resources

We propose a control scheme via a non-cooperative linear quadratic differential game to coordinate the inverter dynamics of Distributed Energy Resources (DERs) in a microgrid (MG). The MG can provide regulation services in support to the upper-level grid, in addition to serving its own load. The control scheme is designed for the MG to track a power reference, while each DER seeks to minimize its individual cost function subject to learned inverter dynamics and load perturbations. We use a nonlinear high-fidelity model developed by Sandia National Laboratories to learn inverter dynamics. We determine a Nash strategy for the DERs that uses state estimation of a Loop Transfer Recovery. Results show that the control scheme enables savings up to 9.3 to 208 times in the DERs objective cost functions and a time-domain response with no oscillations with up to 3 times faster settling times relative to using droop and PI control.

A method for Designing a Guidance Law Utilizing Linear Quadratic Forms based on Differential Games in 3-Dimensional Space

Consider a T-A-D game scenario in space: the target strives to evade the attacker, the defender intercepts the target, and the attacker must both avoid the target's counterattack and attempt to hit the target. It is assumed that all entities are intelligent, anticipating the next state of their adversaries and making the most suitable maneuvering strategy based on this prediction. This paper presents a 3-dimensional linear-quadratic differential game guidance law for target-attacker-defender scenarios, which is more intelligent than traditional guidance laws and is suitable for future intelligent environments.

Orbital Pursuit–Evasion–Defense Linear-Quadratic Differential Game

Orbital Pursuit–Evasion–Defense Linear-Quadratic Differential Game

Stochastic adaptive linear quadratic nonzero-sum differential games

This paper focuses on solving stochastic linear quadratic nonzero-sum differential games with completely unknown system matrices and long-time average costs. Firstly, for the case that the system matrices are known, we design a model-based value iteration (VI) algorithm and a model-based robust VI algorithm to solve the coupled algebraic Riccati equations (AREs), which are directly related to the construction of the feedback Nash equilibrium solution. Then, we present a model-free VI algorithm, based on which feedback control strategies for players are designed using only the data of states and inputs, without requiring the prior knowledge of the system matrices. Moreover, we show that the designed feedback control strategies make the closed-loop game system stable and reach the Nash equilibrium. Finally, a numerical simulation is given to demonstrate the effectiveness of the proposed method.

Closed-loop and open-loop equilibrium of a class time-inconsistent linear-quadratic differential games

Closed-loop and open-loop equilibrium of a class time-inconsistent linear-quadratic differential games

On the Upper Bound of Near Potential Differential Games

This letter presents an extended analysis and a novel upper bound of the subclass of Linear Quadratic Near Potential Differential Games (LQ NPDG). LQ NPDGs are a subclass of potential differential games, for which there is a distance between an LQ exact potential differential game and the LQ NPDG. LQ NPDGs exhibit a unique characteristic: The smaller the distance from an LQ exact potential differential game, the more closer their dynamic trajectories. This letter introduces a novel upper bound for this distance. Moreover, a linear relation between this distance and the resulting trajectory errors is established, opening the possibility for further application of LQ NPDGs.

Saddle-Point Equilibrium Strategy for Linear Quadratic Uncertain Stochastic Hybrid Differential Games Based on Subadditive Measures

This paper describes a kind of linear quadratic uncertain stochastic hybrid differential game system grounded in the framework of subadditive measures, in which the system dynamics are described by a hybrid differential equation with Wiener–Liu noise and the performance index function is quadratic. Firstly, we introduce the concept of hybrid differential games and establish the Max–Min Lemma for the two-player zero-sum game scenario. Next, we discuss the analysis of saddle-point equilibrium strategies for linear quadratic hybrid differential games, addressing both finite and infinite time horizons. Through the incorporation of a generalized Riccati differential equation (GRDE) and guided by the principles of the Itô–Liu formula, we prove that that solving the GRDE is crucial and serves as both a sufficient and necessary precondition for identifying equilibrium strategies within a finite horizon. In addition, we also acquire the explicit formulations of equilibrium strategies in closed forms, alongside determining the optimal values of the cost function. Through the adoption of a generalized Riccati equation (GRE) and applying a similar approach to that used for the finite horizon case, we establish that the ability to solve the GRE constitutes a sufficient criterion for the emergence of equilibrium strategies in scenarios extending over an infinite horizon. Moreover, we explore the dynamics of a resource extraction problem within a finite horizon and separately delve into an H∞ control problem applicable to an infinite horizon. Finally, we present the conclusions.

Attitude control of a 3-DoF quadrotor platform using a linear quadratic integral differential game approach

In this study, a linear quadratic integral differential game approach is applied to regulate and track the Euler angles for a quadrotor experimental platform using two players. One produces commands for each channel of the quadrotor and another generates the worst disturbance based on the mini-maximization of a quadratic criterion with integral action. For this purpose, first, the attitude dynamics of the platform are modeled and its parameters are identified based on the Nonlinear Least Squares Trust-Region Reflective method. The performance of the proposed controller is evaluated for regulation and tracking problems. The ability of the controller is also examined in the disturbance rejection. Moreover, the influence of uncertainty modeling is studied on the obtained results. Then, the performance of the proposed controller is compared with the classic Proportional Integral Derivative, Linear Quadratic Regulator, and Linear Quadratic Integral Regulator. The results demonstrate the effectiveness of the Game Theory on the Linear Quadratic Regulator approach when the input disturbance occurs.

Stochastic Adaptive Linear Quadratic Differential Games

Game theory is playing more and more important roles in understanding complex systems and in investigating intelligent machines with various uncertainties. As a starting point, we consider the classical two-player zero-sum linear-quadratic stochastic differential games, but in contrast to most of the existing studies, the coefficient matrices of the systems are assumed to be unknown to both players, and consequently it is necessary to study adaptive strategies of the players, which may be termed as adaptive games and which has rarely been explored in the literature. In this paper, by introducing a suitable information structure for adaptive games, we will show that a theory can be established on adaptive strategies that are designed based on both the certainty equivalence principle and the diminishing excitation technique. Under almost the same physical structure conditions as those in the traditional known parameters case, it is shown that the closed-loop adaptive game systems will be globally stable and asymptotically reach the Nash equilibrium.

Linear Quadratic Zero-Sum Differential Games With Intermittent and Costly Sensing

Linear Quadratic Zero-Sum Differential Games With Intermittent and Costly Sensing

Research on Enterprise Investment Decision Based on Linear Quadratic Jump Uncertainty Stochastic Differential Game

Research on Enterprise Investment Decision Based on Linear Quadratic Jump Uncertainty Stochastic Differential Game

Optimal strategies for large-scale pursuers against one evader: A mean field game-based hierarchical control approach

This paper proposes a two-level hierarchical control approach based on pursuit-evasion game and mean field game for the problem of large-scale pursuers with multi-population against a single evader, which implements that the evader is surrounded by pursuers. At the upper layer, we model the pursuit-evasion game between the centers of the pursuer populations and the single evader, which is formulated as a linear quadratic differential game (LQDG) to obtain the optimal control of each player. Then the optimal trajectories derived from the optimal controls are input to the lower layer as the reference trajectories. At the lower layer, we formulate the tracking of reference trajectories and terminal surrounding to the evader of large-scale pursuers with multi-population as a multi-population mean-field game (MPMFG), which solves the communication and computing difficulties caused by large-scale agents. Then, we derive the variational primal–dual formulation of the proposed MPMFG model and solve it with CA-Net, a coupled alternating neural network approach. Finally, simulation experiments are performed under various pursuit-evasion scenarios, and it is verified that the proposed game-based two-level hierarchical control approach is feasible and effective.

Measures for the Importance of Information Exchange in Linear Quadratic Differential Games Over Networks

The objective of this paper is to study and characterize the role and the importance of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">information</i> in achieving a feedback (Nash) equilibrium strategy in linear quadratic (LQ) differential games whenever the underlying players are distributed over a (physical or logic) network. It is assumed that each player should achieve a desired goal, quantified by an individual cost functional, in a competitive dynamic environment - captured by an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interconnection network</i> - by relying only on the information and data exchanged with other players according to a prescribed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">information network:</i> the objective of the paper is to establish the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">value</i> of such an information exchange pattern towards achieving a more favorable (social) equilibrium. Interestingly, it is not assumed that the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interconnection network</i> and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">information network</i> coincide. Moreover, since the ability of achieving a certain Nash equilibrium strategy may be lost even by removing a single communication link in the network - thus partially limiting the use of the metrics discussed above - in the second part of the paper we also consider the value of the information in forming approximate Nash equilibrium strategies, namely the so-called <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\varepsilon$</tex-math></inline-formula> -Nash equilibria. Finally, the newly defined metrics are corroborated - together with a few suggested constructive results to characterize such values - by means of numerical examples.