Linear Differential Games Research Articles

Inverse reinforcement learning methods for linear differential games

In this study, we considered the problem of inverse reinforcement learning or estimating the cost function of expert players in multi-player differential games. We proposed two online data-driven solutions for linear–quadratic games that are applicable to systems that fulfill a specific dimension criterion or whose unknown matrices in the cost function conform to a diagonal condition. The first method, which is partially model-free, utilizes the trajectories of expert agents to solve the problem. The second method is entirely model-free and employs the trajectories of both expert and learner agents. We determined the conditions under which the solutions are applicable and identified the necessary requirements for the collected data. We conducted numerical simulations to establish the effectiveness of the proposed methods.

Estimating cost function of expert players in differential games: A model-based method and its data-driven extension

In this paper, we introduce two algorithms for estimating the cost function of expert players engaged in optimal performance within linear continuous-time differential games. Initially, we propose a model-based algorithm, followed by its data-driven model-free extension. Both methods rely on optimal policy gains, obtained by observing Nash equilibrium trajectories of the expert players. The model-free method also utilizes the trajectories of the learner system. This method addresses the limitations found in existing model-free approaches, which may suffer from either high computational costs, limited applicability to specific systems, or both. The effectiveness of the proposed methods is demonstrated through numerical simulations.

Pursuit and Evasion Linear Differential Game Problems with Generalized Integral Constraints

In this paper, we study pursuit and evasion differential game problems of one pursuer/one evader and many pursuers/one evader, respectively, in the space Rn. In both problems, we obtain sufficient conditions that guarantee the completion of a pursuit and an evasion. We construct the players’ optimal strategies in both problems, and we estimate the possible distance that an evader can preserve from pursuers. Lastly, we illustrate our results via some numerical examples.

A differential game model for sponsored content

We consider a communication platform distinguished for its high-quality content, where advertising can take two different forms: traditional and sponsored (also known as native advertising in the marketing literature). Native advertising is a widely used marketing tool that aims to mimic the regular topics of the platform on which it is placed. Due to this striking resemblance, native advertising may be very effective, but at the same time, it may negatively influence the perceived credibility of the media outlet. In our model, a firm allocates investments to both traditional and native advertising on such a platform. Meanwhile, the media outlet must grapple with the trade-off between the profit accrued from publishing native advertising and the ensuing decline in credibility. We formalise this problem as a hierarchical infinite-time horizon linear state differential game, played à la Stackelberg, where the media outlet acts as the leader while the firm is the follower. Finally, we characterise a time-consistent open-loop equilibrium and obtain the conditions that make it optimal for the media outlet to accept native advertising.

On a Linear Differential Game of Pursuit with Integral Constraints in ℓ2

In this paper, we study the stability, controllability, and differential game of pursuit for an infinite system of linear ODEs in ℓ2. The system we consider has a special right-hand side, which is not diagonal and serves as a toy model for controllable system of infinitely many interacting points. We impose integral constraints on the control parameters. We obtain criteria for stability and null controllability of the system. Further, we construct a strategy for the pursuer that guarantees completion of the pursuit problem for the differential game. To prove controllability we use the so called Gramian operators.

Evasion in a linear differential game with many pursuers

Evasion in a linear differential game with many pursuers

On a Linear Differential Game in the Hilbert Space ℓ2

Two player pursuit evasion differential game and time optimal zero control problem in <i>ℓ</i><sup>2</sup> are considered. Optimal control for the corresponding zero control problem is found. A strategy for the pursuer that guarantees the solution for the pursuit problem is constructed.

SINGLE-TYPE PROBLEM OF IMPULSE PURSUIT OF AN OBJECT WITH CHANGING DYNAMICS

We consider a linear differential game with an impulse control of the first player. The capabilities of the first player are determined by the stock of resources that he can use to form his control. The control of the second player is subject to a geometric constraint. The vectograms of the players are described by the same ball with different time-dependent radii. It is believed that the second player at a time moment unknown in advance to the first player can change his dynamics once. The terminal set is a ball with a fixed radius. The goal of the first player is to lead the phase vector to the terminal set at a given time. The goal of the second player is the opposite. Necessary and sufficient conditions for a meeting the terminal set at the given time are found. The corresponding controls of the players, guaranteeing the achievement of their goals are constructed.

Sufficient Solvability Conditions for the Problem of Pursuit under an Impulse Action

The article considers a linear differential pursuit game under the condition that an integral constraint is imposed on the evader’s control and the pursuer uses an impulse control. These impulse actions on the object are made at predetermined time points, and the corresponding control is represented using the Dirac delta function. The article studies linear conflicts described by a system of ordinary differential equations whose trajectories have jumps at certain times. The terminal set is represented by a cylinder in an n-dimensional Euclidean space. The problem is solved using the resolving function method. The fact that the lower bound is reached is proved using the theory of support functions. As a result, the quasi-strategy is replaced by an almost stroboscopic strategy and a method for building this strategy is presented. An example of a nonlinear right-hand side is given.

On the Problem of Pursuing Two Coordinated Evaders in Linear Recurrent Differential Games

On the Problem of Pursuing Two Coordinated Evaders in Linear Recurrent Differential Games

A generalization of first direct method of pursuit for differential inclu-sions

To solve the problem of pursuit in linear differential games, L.S. Pontryagin suggested two direct methods. Direct methods are of great importance in the development of the theory of differential games and in control theory under the conditions of uncertainty. It turned out to be useful also in solving the problem of control synthesis. Pontryagin direct methods have proved themselves as an effective means for solving problems of pursuit- evasion and control. These use integrals, having a number of significant differences from the classical integral. One of the differences consists in the use of multivalued mapping. Pontryaginʼs second direct method, based on concept of the alternating integral, which has no analogs in integration of real function. In definition of alternating integral participate of integration of setvalued mappings and geometric difference (Minkovski difference) of sets. These operations make difficulties for computation of alternating integral. From this point of view, the integral used by the first direct method has a simpler construction. Therefore, the question naturally arises of generalization the first direct method of pursuit. In this paper it will be studied a generalization of the first direct method for pursuit games, being described by differential inclusions where F is a continuous multivalued mapping. This method will be called the modified first direct method of pursuit for differential inclusions. In particular, the class of stroboscopic strategies, the trajectory of the system are determined. For these classes games, it is proved that if the starting point belongs to the modified first integral (the integral from the multivalued mapping, which is present in the definition of the modified fist direct metod), then this is necessary and sufficient condition for completing the game in a fixed time instant in the class of stroboscobic strategies. The problem of computation this integral is important. In the present article it has also been proved that the union operations in the definition of the modified first integral can be narrowed down to the class of compact-valued mappings

A Scheme for Calculating Solvability Sets “Up to Moment” in Linear Differential Games

A Scheme for Calculating Solvability Sets “Up to Moment” in Linear Differential Games

ЛИНЕЙНАЯ ДИФФЕРЕНЦИАЛЬНАЯ ИГРА УДЕРЖАНИЯ С ПОЛОМКОЙ

A linear simple motion constraint differential game is considered. This game is considered from the part of the first player, who needs to keep the state of the system in a given convex terminal set throughout the game, despite the possible glitch and control of the second player. A glitch is understood as an instantaneous stop of the first player at a previously unknown point in time; after a certain time he will eliminate the glitch and will continue his motion. The player control vectograms are n-dimensional convex compacts that depend on time. To construct a u-stable bridge, the second method of L.S. Pontryagin is used. This is how a multi-valued mapping is constructed on the basis of the alternating integral of L.S. Pontryagin. After that, it is proved that the constructed mapping is a u-stable bridge for the game under consideration if a number of conditions are satisfied. At the end of the article, a simple example on the plane is considered, where the vectors of the players are circles centered at the origin and with a constant radius, while the radius of the circle of the first player is strictly greater than the second. In this example, a u-stable bridge is built according to the method proposed in the article, and an extremal strategy is found for the first player on the constructed u-stable bridge.

Strong convexity of reachable sets of linear systems

The reachable set on some time interval of a linear control system $x'\in Ax {+} U$, $x(0)=0$, is considered. A number of cases is examined when the reachable set is the intersection of some balls of fixed radius $R$ (that is, a strongly convex set of radius $R$). In some cases the radius $R$ is estimated from above. It turns out that strong convexity is fairly typical for this class of reachable sets in a certain sense. Among possible applications of this result are the possibility of constructing outer polyhedral approximation of reachable sets with better accuracy in the Hausdorff metric than in the general case, and applications to linear differential games and some optimization problems. Bibliography: 23 titles.

Emission taxation, green innovations and inverted-U aggregate R&D efforts in a linear state differential game

We revisit a well known differential Cournot game with polluting emissions, to propose a version of the model in which environmental taxation is levied on emissions rather than the environmental damage. This allows to attain strong time consistency under open-loop information, and yields two main results which can be summarized as follows: (i) to attain a fully green technology in steady state, the regulator may equivalently adopt an appropriate tax rate (for any given number of firms) or regulate market access (for any given tax rate); (ii) if the environmental damage depends on emissions only (i.e., not on industry output) then the aggregate green R&D effort takes an inverted-U shape, and the industry structure maximising aggregate green innovation also minimises individual and aggregate emissions. This calls for a coordination of environmental and merger regulation so as to create the industry structure most favourable to green innovation.

One Game Problem for Oscillatory Systems

The paper concerns the linear differential game of approaching a cylindrical terminal set. We study the case when classic Pontryagin’s condition does not hold. Instead, the modified considerably weaker condition, dealing with the function of time stretching, is used. The latter allows expanding the range of problems susceptible to analytical solution by the way of passing to the game with delayed information. Investigation is carried out in the frames of Pontryagin’s First Direct method that provides hitting the terminal set by a trajectory of the conflict-controlled process at finite instant of time. In so doing, the pursuer’s control, realizing the game goal, is constructed on the basis of the Filippov-Castaing theorem on measurable choice. The outlined scheme is applied to solving the problem of pursuit for two different second-order systems, describing damped oscillations. For this game, we constructed the function of time stretching and deduced conditions on the game parameters, ensuring termination of the game at a finite instant of time, starting from arbitrary initial states and under all admissible controls of the evader. Keywords: differential game, time-variable information delay, Pontryagin’s condition, Aumann’s integral, principle of time stretching, Minkowski’ difference, damped oscillations.

Linear evasion differential game of one evader and several pursuers with integral constraints

An evasion differential game of one evader and many pursuers is studied. The dynamics of state variables x_1,ldots , x_m are described by linear differential equations. The control functions of players are subjected to integral constraints. If x_i(t) ne 0 for all i in {1,ldots ,m} and t ge 0, then we say that evasion is possible. It is assumed that the total energy of pursuers doesn’t exceed the energy of evader. We construct an evasion strategy and prove that for any positive integer m evasion is possible.

On estimation of Hausdorff deviation of convex polygons in $\mathbb{R}^2$ from their differences with disks

We study a problem concerning the estimation of the Hausdorff deviation of convex polygons in $\mathbb R^2$ from their geometric difference with circles of sufficiently small radius. Problems with such a subject, in which not only convex polygons but also convex compacts in the Euclidean space $\mathbb R^n$ are considered, arise in various fields of mathematics and, in particular, in the theory of differential games, control theory, convex analysis. Estimates of Hausdorff deviations of convex compact sets in $\mathbb R^n$ in their geometric difference with closed balls in $\mathbb R^n$ are presented in the works of L.S. Pontryagin, his staff and colleagues. These estimates are very important in deriving an estimate for the mismatch of the alternating Pontryagin’s integral in linear differential games of pursuit and alternating sums. Similar estimates turn out to be useful in deriving an estimate for the mismatch of the attainability sets of nonlinear control systems in $\mathbb R^n$ and the sets approximating them. The paper considers a specific convex heptagon in $\mathbb R^2$. To study the geometry of this heptagon, we introduce the concept of a wedge in $\mathbb R^2$. On the basis of this notion, we obtain an upper bound for the Hausdorff deviation of a heptagon from its geometric difference with the disc in $\mathbb R^2$ of sufficiently small radius.

On the Problem of Pursuit of the Fractional Order in a Dynamic System with Nonlinear Controls

This article discusses the problem of expanding the results for fractional differential games and optimal control systems. Fractional order management allows ensuring the quality and safety of products, environmental process control. Solving the pursuit problem, we rely on the Pontryagin methods developed for linear differential games of entire orders, and we obtain new sufficient conditions for solving the pursuit problem.

Output feedback adaptive dynamic programming for linear differential zero-sum games

This paper addresses the problem of finding optimal output feedback strategies for solving linear differential zero-sum games using a model-free approach based on adaptive dynamic programming (ADP). In contrast to their discrete-time counterparts, differential games involve continuous-time dynamics and existing ADP approaches to their solutions require access to full measurement of the internal state. This difficulty is due to the fact that direct translation of the discrete-time output feedback ADP results requires derivatives of the input and output measurements, which is generally prohibitive in practice. This work aims to overcome this difficulty and presents a new embedded filtering based observer approach towards designing output feedback ADP algorithms for solving the differential zero-sum game problem. Two output feedback ADP algorithms based respectively on policy iteration and value iteration are developed. The proposed scheme is completely online in nature and works without requiring information of the system dynamics. In addition, this work also addresses the excitation bias problem encountered in output feedback ADP methods, which typically requires a discounting factor for its mitigation. We show that the proposed scheme is bias-free, and therefore, does not require a discounting factor. It is shown that the proposed algorithms converge to the solution obtained by solving the game algebraic Riccati equation. Two numerical examples are demonstrated to validate the proposed scheme.