Nonanticipative Strategies Research Articles

A single player and a mass of agents: A pursuit evasion-like game

We study a finite-horizon differential game of pursuit-evasion like, between a single player and a mass of agents. The player and the mass directly control their own evolution, which for the mass is given by a first order PDE of transport equation type. Using also an adapted concept of non-anticipating strategies, we derive an infinite dimensional Isaacs equation, and by dynamic programming techniques we prove that the value function is the unique viscosity solution on a suitable invariant subset of a Hilbert space.

State and Control Path-Dependent Stochastic Zero-Sum Differential Games: Viscosity Solutions of Path-Dependent Hamilton–Jacobi–Isaacs Equations

In this paper, we consider the two-player state and control path-dependent stochastic zero-sum differential game. In our problem setup, the state process, which is controlled by the players, is dependent on (current and past) paths of state and control processes of the players. Furthermore, the running cost of the objective functional depends on both state and control paths of the players. We use the notion of non-anticipative strategies to define lower and upper value functionals of the game, where unlike the existing literature, these value functions are dependent on the initial states and control paths of the players. In the first main result of this paper, we prove that the (lower and upper) value functionals satisfy the dynamic programming principle (DPP), for which unlike the existing literature, the Skorohod metric is necessary to maintain the separability of càdlàg (state and control) spaces. We introduce the lower and upper Hamilton–Jacobi–Isaacs (HJI) equations from the DPP, which correspond to the state and control path-dependent nonlinear second-order partial differential equations. In the second main result of this paper, we show that by using the functional Itô calculus, the lower and upper value functionals are viscosity solutions of (lower and upper) state and control path-dependent HJI equations, where the notion of viscosity solutions is defined on a compact κ-Hölder space to use several important estimates and to guarantee the existence of minimum and maximum points between the (lower and upper) value functionals and the test functions. Based on these two main results, we also show that the Isaacs condition and the uniqueness of viscosity solutions imply the existence of the game value. Finally, we prove the uniqueness of classical solutions for the (state path-dependent) HJI equations in the state path-dependent case, where its proof requires establishing an equivalent classical solution structure as well as an appropriate contradiction argument.

On guarantee optimization in control problem with finite set of disturbances

In this paper, we deal with a control problem under conditions of disturbances, which is stated as a problem of optimization of the guaranteed result. Compared to the classical formulation of such problems, we assume that the set of admissible disturbances is finite and consists of piecewise continuous functions. In connection with this additional functional constraint on the disturbance, we introduce an appropriate class of non-anticipative control strategies and consider the corresponding value of the optimal guaranteed result. Under a technical assumption concerning a property of distinguishability of the admissible disturbances, we prove that this result can be achieved by using control strategies with full memory. As a consequence, we establish unimprovability of the class of full-memory strategies. A key element of the proof is a procedure of recovering the disturbance acting in the system, which allows us to associate every non-anticipative strategy with a full-memory strategy providing a close guaranteed result. The paper concludes with an illustrative example.

Non-anticipative strategies in guarantee optimization problems under functional constraints on disturbances

For a dynamical system controlled under conditions of disturbances, a problem of optimizing the guaranteed result is considered. A feature of the problem is the presence of functional constraints on disturbances, under which, in general, the set of admissible disturbances is not closed with respect to the operation of “gluing up” of two of its elements. This circumstance does not allow to apply directly the methods developed within the differential games theory for studying the problem and, thus, leads to the necessity of modifying them appropriately. The paper provides a new notion of a non-anticipative control strategy. It is proved that the corresponding functional of the optimal guaranteed result satisfies the dynamic programming principle. As a consequence, so-called properties of $u$- and $v$-stability of this functional are established, which may allow, in the future, to obtain a constructive solution of the problem in the form of feedback (positional) controls.

Explicit Characterization of Feedback Nash Equilibria for Indefinite, Linear-Quadratic, Mean-Field-Type Stochastic Zero-Sum Differential Games with Jump-Diffusion Models

We consider the indefinite, linear-quadratic, mean-field-type stochastic zero-sum differential game for jump-diffusion models (I-LQ-MF-SZSDG-JD). Specifically, there are two players in the I-LQ-MF-SZSDG-JD, where Player 1 minimizes the objective functional, while Player 2 maximizes the same objective functional. In the I-LQ-MF-SZSDG-JD, the jump-diffusion-type state dynamics controlled by the two players and the objective functional include the mean-field variables, i.e., the expected values of state and control variables, and the parameters of the objective functional do not need to be (positive) definite matrices. These general settings of the I-LQ-MF-SZSDG-JD make the problem challenging, compared with the existing literature. By considering the interaction between two players and using the completion of the squares approach, we obtain the explicit feedback Nash equilibrium, which is linear in state and its expected value, and expressed as the coupled integro-Riccati differential equations (CIRDEs). Note that the interaction between the players is analyzed via a class of nonanticipative strategies and the “ordered interchangeability” property of multiple Nash equilibria in zero-sum games. We obtain explicit conditions to obtain the Nash equilibrium in terms of the CIRDEs. We also discuss the different solvability conditions of the CIRDEs, which lead to characterization of the Nash equilibrium for the I-LQ-MF-SZSDG-JD. Finally, our results are applied to the mean-field-type stochastic mean-variance differential game, for which the explicit Nash equilibrium is obtained and the simulation results are provided.

Relaxation of pursuit-evasion differential game and program absorption operator

We consider some natural relaxation of pursuit-evasion differential game. For two closed sets, which are parameters, similar guidance problem for $\varepsilon$-neighborhoods is being solved. We are interested in finding a minimal size of such neighborhoods, which allows player I successfully solve his guidance problem in the class of generalized non-anticipating strategies. To resolve above-mentioned differential game, a modification of Program Iterations Method is implemented. Size of the neighborhoods is found as a position function and it's defined by application of special iterative procedure further below. As a corollary, it is shown that desired function is a fixed point of the open-loop operator, which defines the procedure.

Релаксации игровой задачи сближения, связанные с альтернативой в дифференциальной игре сближения-уклонения

Differential game (DG) of guidance-evasion for a finite time interval is considered;as parameters, the target set (TS) and the set defining phase constraints (PC) are used.Player I interested in realization of guidance with TS under validity PC uses set-valuedquasistrategies (nonanticipating strategies) and Player II having opposite target uses strategieswith nonanticipating choice of correction instants and finite numbers of such instants.On informative level, the setting corresponds to alternative theorem of N. N. Krasovskii andA. I. Subbotin. For position not belonging to solvability set of Player I, determination ofthe least size of neighborhoods for set-parameters under that Player I guarantees guidance(under weakened conditions) is interested. In article, this scheme is supplemented by priorityelements in questions of TS attainment and PC validity; this is realized by special parameterdefining relation for sizes of corresponding neighborhoods. Under these conditions, a functionof the least size of TS neighborhood is defined by procedure used program iteration methodfor two variants. The above-mentioned function is fixed point for one of two used “program”operators. Special type of the quality functional for which values of the above-mentionedfunction coincide with values of the minimax-maximin games is established.

A Differential Game with Exit Costs

We study a differential game where two players separately control their own dynamics, pay a running cost, and moreover pay an exit cost (quitting the game) when they leave a fixed domain. In particular, each player has its own domain and the exit cost consists of three different exit costs, depending whether either the first player only leaves its domain, or the second player only leaves its domain, or they both simultaneously leave their own domain. We prove that, under suitable hypotheses, the lower and upper values are continuous and are, respectively, the unique viscosity solution of a suitable Dirichlet problem for a Hamilton–Jacobi–Isaacs equation. The continuity of the values relies on the existence of suitable non-anticipating strategies respecting the domain constraint. This problem is also treated in this work.

A differential game on Wasserstein space. Application to weak approachability with partial monitoring

Studying continuous time counterpart of some discrete time dynamics is now a standard and fruitful technique, as some properties hold in both setups. In game theory, this is usually done by considering differential games on Euclidean spaces. This allows to infer properties on the convergence of values of a repeated game, to deal with the various concepts of approachability, etc. In this paper, we introduce a specific but quite abstract differential game defined on the Wasserstein space of probability distributions and we prove the existence of its value. Going back to the discrete time dynamics, we derive results on weak approachability with partial monitoring: we prove that any set satisfying a suitable compatibility condition is either weakly approachable or weakly excludable. We also obtain that the value for differential games with nonanticipative strategies is the same that those defined with a new concept of strategies very suitable to make links with repeated games.

Nash equilibrium payoffs for non-zero-sum stochastic differential games without Isaacs condition

ABSTRACTWe mainly investigate the existence of the Nash equilibrium payoffs for non-zero-sum stochastic differential games without assuming Isaacs condition in this paper. Along the partition π of the time interval , we choose a suitable random non-anticipative strategy with delay to study our non-zero-sum stochastic differential game. We prove for the corresponding both zero-sum stochastic differential games without Isaacs condition the existence of the value functions. With the help of these value functions we give the characterization of the Nash equilibrium payoffs. This characterization allows to prove the existence of Nash equilibrium payoffs.

Infinite Horizon Differential Games with Asymmetric Information

This article is concerned with the existence of a value for a zero-sum differential game with an asymmetric information on initial state with an infinite horizon running cost. Before the game begins, an initial state is chosen randomly from a finite set and each player receives a private signal generated by the chosen initial state. The main result is that the game has a value with random non-anticipative strategies with delay and that its value function can be characterized as the unique bounded continuous viscosity solution of a suitable Hamilton–Jacobi–Isaacs equation.

Lion and Man Game and Fixed Point-Free Maps

This paper is dedicated to the pursuit-evasion game in which both players (Lion and Man) move in a metric space, have equal maximum speeds and complete information about the location of each other. We assume that evasion is successful if, for some initial positions of players, there exists a positive number p and an evader’s non-anticipative strategy guaranteeing that the distance between the players is always greater than p. We consider connection between successful evasion and such properties of the phase space as geodesics behavior and the existence of non-expanding fixed point-free self-maps.

Image feedback based optimal control and the value of information in a differential game

In this paper, we address pursuit-evasion problems in which the pursuer is a Differential Drive Robot (DDR) that attempts to capture an omnidirectional evader. From the Nash property it follows that if the evader deviates from its maximum potential speed then the capture time shall not increase for a pursuer that does not deviate from its Nash equilibrium motion strategy. However, it is not immediately clear how the pursuer could exploit that evader’s deviation from its maximum potential speed, which might correspond to situations where the evader’s capabilities may degrade with time, for example, battery depletion in an autonomous vehicle, or fatigue in an animal evader. This can be considered as a scenario of an evader in which the set of admissible controls varies with time. In the present paper we consider such scenario. In our first result, we propose an alternative strategy for the pursuer, which, for certain scenarios, further reduces the capture time compared to the strategy based on the maximum potential evader’s speed. In our second result, we show that, under non-anticipative strategies, a pursuer strategy that uses the instantaneous evader speed alone, does not always guarantee to improve the payoff for the pursuer, nor the capture of the evader. Hence, we conclude that the evader’s location is the relevant information for the pursuer to know. Later, we present vision-based control laws that implement the optimal pursuer strategy. The optimal pursuer strategy is characterized by a partition of the reduced space (a representation of the game in the pursuer’s body-attached coordinate system) in which each region maps to an optimal pursuer action. We consider the case for which the pursuer is equipped with an omnidirectional catadioptric camera. Finally, in our third result we show that the location of the evader on the image can be directly used by the pursuer to define its motion strategy, in spite of the distortion of the state space suffered on the image. That is, the pursuer is able to apply its motion strategy using the image without explicitly reconstructing the evader’s position. This approach is computationally efficient, and robust to occlusions and noise in the image.

Uncertain saddle point equilibrium differential games with non-anticipating strategies

In this paper, we investigate uncertain saddle point equilibrium differential games under uncertain environment. We propose an optimistic value game model and define the value function of the game by introducing the concept of non-anticipating strategy. We prove the continuity and dynamic programming property of the value function. Then we derive the uncertain Hamilton–Jacobi–Isaacs equation by the viscosity solution approach.

Existence of Value for a Differential Game with Incomplete Information and Revealing

This article is devoted to the existence of value for a zero-sum differential game with incomplete information on initial state with partial revealing of information and with a running cost. Before the game begins, an initial state is chosen from among several states, and both players do not know this initial state until the dynamic system reaches a fixed target during the game. The main result is the existence of value for the game defined with nonanticipative strategies with delay. Moreover, the value is the unique bounded continuous viscosity solution of a suitable Hamilton--Jacobi--Isaacs equation.

Program-Predictive Feedback Control for Systems with Evolving Dynamics

Abstract We consider a new class of problems of control under uncertainty in which uncertainty is described by a finite set of scenarios of evolution of some system parameters. We derive optimality conditions for non-anticipating strategies in terms of some nonstandard maximum principle which includes a new type of boundary value problem. These optimality conditions are used for design of the optimal discontinuous feedback control for such systems.

Zero-sum and nonzero-sum differential games without Isaacs condition

In this paper we study the zero-sum and nonzero-sum differential games with not assuming Isaacs condition. Along with the partition $\pi$ of the time interval $[0,T]$, we choose the suitable random non-anticipative strategy with delay to study our differential games with asymmetric information. Using Fenchel transformation, we prove that the limits of the upper value function $W^\pi$ and lower value function $V^\pi$ coincide when the mesh of partition $\pi$ tends to 0. Moreover, we give a characterization for the Nash equilibrium payoff (NEP, for short) of our nonzero-sum differential games without Isaacs condition, then we prove the existence of the NEP of our games. Finally, by considering all the strategies along with all partitions, we give a new characterization for the value of our zero-sum differential game with asymmetric information under some equivalent Isaacs condition.

How Do Contract Parameters Influence the Economics of Vehicle-to-Grid?

By modifying the charging patterns of a pool of electric vehicles (EVs), aggregators are able to provide services to the electric grid. The parameters that shape the agreement between aggregators and EV owners, such as plug-in duration and guaranteed driving range, affect the appeal of signing up with an aggregator. Our study examines how these contract parameters influence the profitability of vehicle pools. We use a bottom-up model that encompasses the entire planning problem faced by an aggregator. The model is applied to the German secondary reserve market using actual driving patterns and market data. Our results indicate that contract parameters influence EVs’ value to aggregators in surprising ways, partially contradicting findings in previous literature. In essence, high levels of plug-in durations or low levels of guaranteed driving for planned trips do not necessarily provide the highest profit. Furthermore, we find that realistic nonanticipative strategies lead to annual profits in the range of €167 to €125 per vehicle, which is in the lower range of previous studies. Our results are a starting point for a discussion about optimal contract parameters and the design of corresponding marketing strategies. The online appendix is available at https://doi.org/10.1287/msom.2016.0601 .

The Existence of Game Value for Path-dependent Stochastic Differential Game

In this paper we study a two-player zero-sum stochastic differential game for a path-dependent stochastic system under a recursive path-dependent cost functional. Due to the typical non-Markovian structure, the game value is a random field. Dividing the time horizontal into small intervals, we approximate the path-dependent game by a series of state-dependent games. We utilize the state-dependent viscosity solution theory to prove that under Isaacs' condition the game value exists. In our model, coefficients of diffusion of the system contain control and strategy, and could be degenerate The dimension of the state space is high. The existence of approximating Nash equilibrium ($\varepsilon$-saddle point) is given under the formula about nonanticipative strategy with delay.

How Do Product Parameters Influence the Economics of Vehicle-to-Grid?

By modifying the charging patterns of a pool of electric vehicles EVs, aggregators are able to provide services to the electric grid. The parameters that shape the agreement between aggregators and EV owners, such as plug-in duration and guaranteed driving range, affect the appeal of signing up with an aggregator. Our study examines how these contract parameters influence the profitability of vehicle pools. We use a bottom-up model that encompasses the entire planning problem faced by an aggregator. The model is applied to the German secondary reserve market using actual driving patterns and market data. Our results indicate that contract parameters influence EVs' value to aggregators in surprising ways, partially contradicting findings in previous literature. In essence, high levels of plug-in durations or low levels of guaranteed driving for planned trips do not necessarily provide the highest profit. Furthermore, we find that realistic nonanticipative strategies lead to annual profits in the range of €167 to €125 per vehicle, which is in the lower range of previous studies. Our results are a starting point for a discussion about optimal contract parameters and the design of corresponding marketing strategies. The online appendix is available at https://doi.org/10.1287/msom.2016.0601 .