Linear Quadratic Differential Games Research Articles

Identification methods for ordinal potential differential games

This paper introduces two new identification methods for linear quadratic (LQ) ordinal potential differential games (OPDGs). Potential games are notable for their benefits, such as the computability and guaranteed existence of Nash Equilibria. While previous research has analyzed ordinal potential static games, their applicability to various engineering applications remains limited. Despite the earlier introduction of OPDGs, a systematic method for identifying a potential game for a given LQ differential game has not yet been developed. To address this gap, we propose two identification methods to provide the quadratic potential cost function for a given LQ differential game. Both methods are based on linear matrix inequalities (LMIs). The first method aims to minimize the condition number of the potential cost function’s parameters, offering a faster and more precise technique compared to earlier solutions. In addition, we present an evaluation of the feasibility of the structural requirements of the system. The second method, with a less rigid formulation, can identify LQ OPDGs in cases where the first method fails. These novel identification methods are verified through simulations, demonstrating their advantages and potential in designing and analyzing cooperative control systems.

Analytical game strategies for active UAV defense considering response delays

In the realm of aerial warfare, the protection of Unmanned Aerial Vehicles (UAVs) against adversarial threats is crucial. In order to balance the impact of response delays and the demand for onboard applications, this paper derives three analytical game strategies for the active defense of UAVs from differential game theory, accommodating the first-order dynamic delays. The targeted UAV executes evasive maneuvers and launches a defending missile to intercept the attacking missile, which constitutes a UAV-Missile-Defender (UMD) three-body game problem. We explore two distinct operational paradigms: the first involves the UAV and the defender working collaboratively to intercept the incoming threat, while the second prioritizes UAV self-preservation, with independent maneuvering away from potentially sacrificial engagements. Starting with model linearization and order reduction, the Collaborative Interception Strategy (CIS) is first derived via a linear quadratic differential game formulation. Building upon CIS, we further explore two distinct strategies: the Informed Defender Interception Strategy (IDIS), which utilizes UAV maneuvering information, and the Unassisted Defender Interception Strategy (UDIS), which does not rely on UAV maneuvering information. Additionally, we investigate the conditions for the existence of saddle point solutions and their relationship with vehicle maneuverability and response agility. The simulations demonstrate the effectiveness and advantages of the proposed strategies.

Non-cooperative games to control learned inverter dynamics of distributed energy resources

We propose a control scheme via a non-cooperative linear quadratic differential game to coordinate the inverter dynamics of Distributed Energy Resources (DERs) in a microgrid (MG). The MG can provide regulation services in support to the upper-level grid, in addition to serving its own load. The control scheme is designed for the MG to track a power reference, while each DER seeks to minimize its individual cost function subject to learned inverter dynamics and load perturbations. We use a nonlinear high-fidelity model developed by Sandia National Laboratories to learn inverter dynamics. We determine a Nash strategy for the DERs that uses state estimation of a Loop Transfer Recovery. Results show that the control scheme enables savings up to 9.3 to 208 times in the DERs objective cost functions and a time-domain response with no oscillations with up to 3 times faster settling times relative to using droop and PI control.

Orbital Pursuit–Evasion–Defense Linear-Quadratic Differential Game

Orbital Pursuit–Evasion–Defense Linear-Quadratic Differential Game

Stochastic adaptive linear quadratic nonzero-sum differential games

This paper focuses on solving stochastic linear quadratic nonzero-sum differential games with completely unknown system matrices and long-time average costs. Firstly, for the case that the system matrices are known, we design a model-based value iteration (VI) algorithm and a model-based robust VI algorithm to solve the coupled algebraic Riccati equations (AREs), which are directly related to the construction of the feedback Nash equilibrium solution. Then, we present a model-free VI algorithm, based on which feedback control strategies for players are designed using only the data of states and inputs, without requiring the prior knowledge of the system matrices. Moreover, we show that the designed feedback control strategies make the closed-loop game system stable and reach the Nash equilibrium. Finally, a numerical simulation is given to demonstrate the effectiveness of the proposed method.

Closed-loop and open-loop equilibrium of a class time-inconsistent linear-quadratic differential games

Closed-loop and open-loop equilibrium of a class time-inconsistent linear-quadratic differential games

On the Upper Bound of Near Potential Differential Games

This letter presents an extended analysis and a novel upper bound of the subclass of Linear Quadratic Near Potential Differential Games (LQ NPDG). LQ NPDGs are a subclass of potential differential games, for which there is a distance between an LQ exact potential differential game and the LQ NPDG. LQ NPDGs exhibit a unique characteristic: The smaller the distance from an LQ exact potential differential game, the more closer their dynamic trajectories. This letter introduces a novel upper bound for this distance. Moreover, a linear relation between this distance and the resulting trajectory errors is established, opening the possibility for further application of LQ NPDGs.

Saddle-Point Equilibrium Strategy for Linear Quadratic Uncertain Stochastic Hybrid Differential Games Based on Subadditive Measures

This paper describes a kind of linear quadratic uncertain stochastic hybrid differential game system grounded in the framework of subadditive measures, in which the system dynamics are described by a hybrid differential equation with Wiener–Liu noise and the performance index function is quadratic. Firstly, we introduce the concept of hybrid differential games and establish the Max–Min Lemma for the two-player zero-sum game scenario. Next, we discuss the analysis of saddle-point equilibrium strategies for linear quadratic hybrid differential games, addressing both finite and infinite time horizons. Through the incorporation of a generalized Riccati differential equation (GRDE) and guided by the principles of the Itô–Liu formula, we prove that that solving the GRDE is crucial and serves as both a sufficient and necessary precondition for identifying equilibrium strategies within a finite horizon. In addition, we also acquire the explicit formulations of equilibrium strategies in closed forms, alongside determining the optimal values of the cost function. Through the adoption of a generalized Riccati equation (GRE) and applying a similar approach to that used for the finite horizon case, we establish that the ability to solve the GRE constitutes a sufficient criterion for the emergence of equilibrium strategies in scenarios extending over an infinite horizon. Moreover, we explore the dynamics of a resource extraction problem within a finite horizon and separately delve into an H∞ control problem applicable to an infinite horizon. Finally, we present the conclusions.

Attitude control of a 3-DoF quadrotor platform using a linear quadratic integral differential game approach

In this study, a linear quadratic integral differential game approach is applied to regulate and track the Euler angles for a quadrotor experimental platform using two players. One produces commands for each channel of the quadrotor and another generates the worst disturbance based on the mini-maximization of a quadratic criterion with integral action. For this purpose, first, the attitude dynamics of the platform are modeled and its parameters are identified based on the Nonlinear Least Squares Trust-Region Reflective method. The performance of the proposed controller is evaluated for regulation and tracking problems. The ability of the controller is also examined in the disturbance rejection. Moreover, the influence of uncertainty modeling is studied on the obtained results. Then, the performance of the proposed controller is compared with the classic Proportional Integral Derivative, Linear Quadratic Regulator, and Linear Quadratic Integral Regulator. The results demonstrate the effectiveness of the Game Theory on the Linear Quadratic Regulator approach when the input disturbance occurs.

Stochastic Adaptive Linear Quadratic Differential Games

Game theory is playing more and more important roles in understanding complex systems and in investigating intelligent machines with various uncertainties. As a starting point, we consider the classical two-player zero-sum linear-quadratic stochastic differential games, but in contrast to most of the existing studies, the coefficient matrices of the systems are assumed to be unknown to both players, and consequently it is necessary to study adaptive strategies of the players, which may be termed as adaptive games and which has rarely been explored in the literature. In this paper, by introducing a suitable information structure for adaptive games, we will show that a theory can be established on adaptive strategies that are designed based on both the certainty equivalence principle and the diminishing excitation technique. Under almost the same physical structure conditions as those in the traditional known parameters case, it is shown that the closed-loop adaptive game systems will be globally stable and asymptotically reach the Nash equilibrium.

Linear Quadratic Zero-Sum Differential Games With Intermittent and Costly Sensing

Linear Quadratic Zero-Sum Differential Games With Intermittent and Costly Sensing

Research on Enterprise Investment Decision Based on Linear Quadratic Jump Uncertainty Stochastic Differential Game

Research on Enterprise Investment Decision Based on Linear Quadratic Jump Uncertainty Stochastic Differential Game

Optimal strategies for large-scale pursuers against one evader: A mean field game-based hierarchical control approach

This paper proposes a two-level hierarchical control approach based on pursuit-evasion game and mean field game for the problem of large-scale pursuers with multi-population against a single evader, which implements that the evader is surrounded by pursuers. At the upper layer, we model the pursuit-evasion game between the centers of the pursuer populations and the single evader, which is formulated as a linear quadratic differential game (LQDG) to obtain the optimal control of each player. Then the optimal trajectories derived from the optimal controls are input to the lower layer as the reference trajectories. At the lower layer, we formulate the tracking of reference trajectories and terminal surrounding to the evader of large-scale pursuers with multi-population as a multi-population mean-field game (MPMFG), which solves the communication and computing difficulties caused by large-scale agents. Then, we derive the variational primal–dual formulation of the proposed MPMFG model and solve it with CA-Net, a coupled alternating neural network approach. Finally, simulation experiments are performed under various pursuit-evasion scenarios, and it is verified that the proposed game-based two-level hierarchical control approach is feasible and effective.

Measures for the Importance of Information Exchange in Linear Quadratic Differential Games Over Networks

The objective of this paper is to study and characterize the role and the importance of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">information</i> in achieving a feedback (Nash) equilibrium strategy in linear quadratic (LQ) differential games whenever the underlying players are distributed over a (physical or logic) network. It is assumed that each player should achieve a desired goal, quantified by an individual cost functional, in a competitive dynamic environment - captured by an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interconnection network</i> - by relying only on the information and data exchanged with other players according to a prescribed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">information network:</i> the objective of the paper is to establish the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">value</i> of such an information exchange pattern towards achieving a more favorable (social) equilibrium. Interestingly, it is not assumed that the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interconnection network</i> and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">information network</i> coincide. Moreover, since the ability of achieving a certain Nash equilibrium strategy may be lost even by removing a single communication link in the network - thus partially limiting the use of the metrics discussed above - in the second part of the paper we also consider the value of the information in forming approximate Nash equilibrium strategies, namely the so-called <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\varepsilon$</tex-math></inline-formula> -Nash equilibria. Finally, the newly defined metrics are corroborated - together with a few suggested constructive results to characterize such values - by means of numerical examples.

On Linear-Quadratic Differential Games for Fractional-Order Systems

On Linear-Quadratic Differential Games for Fractional-Order Systems

Human Inspired Autonomous Intersection Handling Using Game Theory

Left turning for autonomous vehicles at intersections is challenging due to the various driving behaviors from different human drivers and the strong interaction between the autonomous vehicle and human traffic participants. This paper proposes a planning and decision making framework for intersection left-turning which considers the interaction between autonomous vehicles and human drivers as well as pedestrians to address this issue. The proposed framework considers interactions mathematically by formulating the problem as a linear quadratic differential game. Through solving the Nash equilibrium of the game, the autonomous vehicle is able to properly interact with surrounding traffic participants. Under the differential game framework, the accuracy of the interaction formulation is closely related to the behavior model of human drivers. Therefore, real-world human behavior is extracted and evaluated from naturalistic driving dataset to help establish more realistic modeling and estimation of various kinds of traffic participants, including aggressive, neutral and conservative traffic participants. The simulation results show that the autonomous vehicle is able to properly estimate the types of traffic participants by observing their behavior using the proposed technique. Then the autonomous vehicle behave according to the types of those traffic participants to enable interactive and human-like planning and decision making at intersections.

The Equivalence Conditions of Optimal Feedback Control-Strategy Operators for Zero-Sum Linear Quadratic Stochastic Differential Game with Random Coefficients

From the previous work, when solving the LQ optimal control problem with random coefficients (SLQ, for short), it is remarkably shown that the solution of the backward stochastic Riccati equations is not regular enough to guarantee the robustness of the feedback control. As a generalization of SLQ, interesting questions are, “how about the situation in the differential game?”, “will the same phenomenon appear in SLQ?”. This paper will provide the answers. In this paper, we consider a closed-loop two-person zero-sum LQ stochastic differential game with random coefficients (SDG, for short) and generalize the results of Lü–Wang–Zhang into the stochastic differential game case. Under some regularity assumptions, we establish the equivalence between the existence of the robust optimal feedback control strategy operators and the solvability of the corresponding backward stochastic Riccati equations, which leads to the existence of the closed-loop saddle points. On the other hand, the problem is not closed-loop solvable if the solution of the corresponding backward stochastic Riccati equations does not have the needed regularity.

Feedback Nash Equilibria in Differential Games With Impulse Control

We study a class of deterministic finite-horizon two-player nonzero-sum differential games where players are endowed with different kinds of controls. We assume that Player 1 uses piecewise-continuous controls, while Player 2 uses impulse controls. For this class of games, we seek to derive conditions for the existence of feedback Nash equilibrium strategies for the players. More specifically, we provide a verification theorem for identifying such equilibrium strategies, using the Hamilton-Jacobi-Bellman (HJB) equations for Player 1 and the quasi-variational inequalities (QVIs) for Player 2. Further, we show that the equilibrium number of interventions by Player 2 is upper bounded. Furthermore, we specialize the obtained results to a scalar two-player linear-quadratic differential game. In this game, Player 1's objective is to drive the state variable towards a specific target value, and Player 2 has a similar objective with a different target value. We provide, for the first time, an analytical characterization of the feedback Nash equilibrium in a linear-quadratic differential game with impulse control. We illustrate our results using numerical experiments.

Linear Quadratic Leader-Follower Stochastic Differential Games: Closed-Loop Solvability

Linear Quadratic Leader-Follower Stochastic Differential Games: Closed-Loop Solvability

Robust backward linear-quadratic differential game and team: A soft-constraint analysis

The robust design with model uncertainty is an extensively investigated research issue but traditionally focuses on forward stochastic system for which the initial condition is given as a priori. This paper makes a systematic continuation of this issue but in backward linear-quadratic context. That is, the state dynamics is evolved by a linear backward stochastic differential equation with terminal condition being specified, and cost functionals to be optimized are of quadratic forms. Three types of dynamic backward decisions, are introduced in both game and team setting in presence of drift uncertainty (disturbance). Specifically, in (Stackelberg) game setting, both cases for the follower or leader with unknown disturbance are discussed respectively; whereas in the team setting, the case with disturbance unknown to the leader is highlighted. For each of these three types, corresponding robust backward strategies are designed and associated optimal values for either leader or follower are given.