Deterministic Games Research Articles

Optimal power schedule for distributed Kalman filtering under DoS attacks: a Stackelberg game strategy

This paper investigates the power schedule problem for a cyber-physical system under signal-to-interference-plus-noise ratio (SINR)-based denial-of-service (DoS) attacks. In contrast to the existing works where only a single channel or a multi-channel network with an estimation centre is considered, the interactive dynamic game process in a fully distributed wireless sensor network is concerned in this paper based on a distributed Kalman filtering framework. The aim of the sensors is to minimise the mean square error of the fusion filter from the perspective of infinite time domain power management. The attacker interferes the fusion channels to reach the opposite goal. The competitive relationship among the sensors and attacker is modelled as a general-sum deterministic game based on Stackelberg game. Taking into account the practical situation where each player does not have a good knowledge of the global information, a distributed reinforcement learning in groups algorithm based on Stackelberg strategy is proposed to learn the joint optimal strategy. In particular, we consider that the strategy space of each player is unknown to other players and can be infinitely expanded through the individual local observation information. Finally, the proposed results are verified by the simulation examples.

The Fan Theorem, its strong negation, and the determinacy of games

The Fan Theorem, its strong negation, and the determinacy of games

Path-dependent zero-sum deterministic games with intermediate Hamiltonians

Path-dependent zero-sum deterministic games with intermediate Hamiltonians

Arena-Independent Finite-Memory Determinacy in Stochastic Games

We study stochastic zero-sum games on graphs, which are prevalent tools to model decision-making in presence of an antagonistic opponent in a random environment. In this setting, an important question is the one of strategy complexity: what kinds of strategies are sufficient or required to play optimally (e.g., randomization or memory requirements)? Our contributions further the understanding of arena-independent finite-memory (AIFM) determinacy, i.e., the study of objectives for which memory is needed, but in a way that only depends on limited parameters of the game graphs. First, we show that objectives for which pure AIFM strategies suffice to play optimally also admit pure AIFM subgame perfect strategies. Second, we show that we can reduce the study of objectives for which pure AIFM strategies suffice in two-player stochastic games to the easier study of one-player stochastic games (i.e., Markov decision processes). Third, we characterize the sufficiency of AIFM strategies through two intuitive properties of objectives. This work extends a line of research started on deterministic games to stochastic ones.

Stochastic Games with Synchronization Objectives

We consider two-player stochastic games played on a finite graph for infinitely many rounds. Stochastic games generalize both Markov decision processes (MDP) by adding an adversary player, and two-player deterministic games by adding stochasticity. The outcome of the game is a sequence of distributions over the graph states, representing the evolution of a population consisting of a continuum number of identical copies of a process modeled by the game graph. We consider synchronization objectives, which require the probability mass to accumulate in a set of target states, either always, once, infinitely often, or always after some point in the outcome sequence; and the winning modes of sure winning (if the accumulated probability is equal to 1) and almost-sure winning (if the accumulated probability is arbitrarily close to 1). We present algorithms to compute the set of winning distributions for each of these synchronization modes, showing that the corresponding decision problem is PSPACE-complete for synchronizing once and infinitely often and PTIME-complete for synchronizing always and always after some point. These bounds are remarkably in line with the special case of MDPs, while the algorithmic solution and proof technique are considerably more involved, even for deterministic games. This is because those games have a flavor of imperfect information, in particular they are not determined and randomized strategies need to be considered, even if there is no stochastic choice in the game graph. Moreover, in combination with stochasticity in the game graph, finite-memory strategies are not sufficient in general.

Minimax Q-learning control for linear systems using the Wasserstein metric

Stochastic optimal control usually requires an explicit dynamical model with probability distributions, which are difficult to obtain in practice. In this work, we consider the linear quadratic regulator (LQR) problem of unknown linear systems and adopt a Wasserstein penalty to address the distribution uncertainty of additive stochastic disturbances. By constructing an equivalent deterministic game of the penalized LQR problem, we propose a Q-learning method with convergence guarantees to learn an optimal minimax controller.

Nim variants

Combinatorial game theory provides results for the class of two-player, deterministic games with perfect information. With the aim of generalizing this theory to the class of non-perfect information games in mind, we introduce and analyze three variants of the game of Nim. In these variants, the opponent only receives partial information on the move executed by the opponent. We model the variants as games in extensive form and compute Nash equilibria for different starting configurations. For one variant, this provides a full characterization of the game. For the other variants, we prove some partial and structural results, but a full characterization remains elusive.

Convergence of Discrete-Time Deterministic Games to Path-Dependent Isaacs Partial Differential Equations Under Quadratic Growth Conditions

Convergence of Discrete-Time Deterministic Games to Path-Dependent Isaacs Partial Differential Equations Under Quadratic Growth Conditions

Innovation and the persistence of monopoly under diseconomies of scope or scale

We participate in the lasting debate about the persistence of monopolies under technological change, by examining two deterministic games modelling innovation auctions. We highlight some novel aspects within such debate. If product innovation is at stake, the joint effect of diseconomies of scope and product differentiation may allow the entrant to acquire the innovation and give rise to a duopoly. Process innovation is analysed in a model with increasing marginal production costs to show that the innovating monopolist always uses both technologies by virtue of Jensen’s inequality, and this is sufficient but not necessary to preserve its monopoly power.

Deterministic Asymmetric-cost Differential Games for Energy Production with Production Bounds

Deterministic Asymmetric-cost Differential Games for Energy Production with Production Bounds

Almost sure exponential stability of two-strategy evolutionary games with multiplicative noise

In this paper, stochastic two-strategy evolutionary games in which the players are interfered by multiplicative noise are studied. We propose a stochastic replicator dynamic model to investigate the almost sure exponential stability (ASES) of general two-strategy games. Compared with the deterministic games and stochastic games with additive noise, our most interesting results concern the impact of multiplicative noise on the cooperation behavior in large well-mixed populations. We establish sufficient conditions on stochastic noise which can ensure the stable equilibrium points of general two-strategy games are indeed ASES. The stochastic Lyapunov framework is used to prove the stochastic stability, where finding the suitable Lyapunov function is the key and challenging in our case. Three different types of two-strategy game models (snowdrift games, hunt stag games and prisoner’s dilemmas) are simulated to verify our theoretical results.

Towards a systematic computational framework for modeling multi-agent decision-making at micro level for smart vehicles in a smart world

We propose a multi-agent based computational framework for modeling decision-making and strategic interaction at micro level for smart vehicles in a smart world. The concepts of Markov game and best response dynamics are heavily leveraged. Our aim is to make the framework conceptually sound and computationally practical for a range of realistic applications, including micro path planning for autonomous vehicles. To this end, we first convert the would-be stochastic game problem into a closely related deterministic one by introducing risk premium in the utility function for each individual agent. We show how the sub-game perfect Nash equilibrium of the simplified deterministic game can be solved by an algorithm based on best response dynamics. In order to better model human driving behaviors with bounded rationality, we seek to further simplify the solution concept by replacing the Nash equilibrium condition with a heuristic and adaptive optimization with finite look-ahead anticipation. In addition, the algorithm corresponding to the new solution concept drastically improves the computational efficiency. To demonstrate how our approach can be applied to realistic traffic settings, we conduct a simulation experiment: to derive merging and yielding behaviors on a double-lane highway with an unexpected barrier. Despite assumption differences involved in the two solution concepts, the derived numerical solutions show that the endogenized driving behaviors are very similar. We also briefly comment on how the proposed framework can be further extended in a number of directions in our forthcoming work, such as behavioral calibration using real traffic video data, and computational mechanism design for traffic policy optimization.

Solving singular evolution problems in sub-Riemannian groups via deterministic games

In this manuscript, we prove the existence of viscosity solutions to singular parabolic equations in Carnot groups. We develop the analysis by constructing appropriate deterministic games adapted to the algebraic and differential structures of Carnot groups. We point out that the proof of existence does not require a comparison principle and it is based on an Arzela-Ascoli-type theorem.
 For more information see https://ejde.math.txstate.edu/Volumes/2021/57/abstr.html

Study on Stochastic Differential Game Model in Network Attack and Defense

In recent years, evolutionary game theory has been gradually applied to analyze and predict network attack and defense for maintaining cybersecurity. The traditional deterministic game model cannot accurately describe the process of actual network attack and defense due to changing in the set of attack-defense strategies and external factors (such as the operating environment of the system). In this paper, we construct a stochastic evolutionary game model by the stochastic differential equation with Markov property. The evolutionary equilibrium solution of the model is found and the stability of the model is proved according to the knowledge of the stochastic differential equation. And we apply the explicit Euler numerical method to analyze the evolution of the strategy selection of the players for different problem situations. The simulation results show that the stochastic evolutionary game model proposed in this paper can get a steady state and obtain the optimal defense strategy under the action of the stochastic disturbance factor. In addition, compared with other kinds of literature, we can conclude that the return on security investment of this model is better, and the strategy selection of the attackers and defenders in our model is more suitable for actual network attack and defense.

Stability and Hopf Bifurcation of a Stochastic Cournot Duopoly Game in a Blockchain Cloud Services Market Driven by Brownian Motion

The blockchain technology has been rapidly popularizing and gaining a quick mileage in many domains owing to its innate nature, so there must be several tremendous market opportunities and uncertainties for the blockchain cloud service. We propose a Cournot duopoly game in a blockchain cloud services market for cyber-physical-social systems (CPSS). Introducing Brownian noise disturbance to the deterministic Cournot duopoly game, we construct a stochastic Cournot duopoly game in a blockchain cloud service market with uncertainty. We analyze the stochastic stability of the Cournot duopoly game by the Lyapunov exponent method, the Lyapunov function, and the singular boundary theory. Then we investigate the stochastic Hopf bifurcation of the Cournot duopoly game by the invariant measure theory. At last, we numerically validate these results by the probability density. These results of stability and Hopf bifurcation illustrate complex behaviors of the stochastic Cournot duopoly game in the blockchain cloud services market.

A deterministic game interpretation for fully nonlinear parabolic equations with dynamic boundary conditions

This paper is devoted to deterministic discrete game-theoretic interpretations for fully nonlinear parabolic and elliptic equations with nonlinear dynamic boundary conditions. It is known that the classical Neumann boundary condition for general parabolic or elliptic equations can be generated by including reflections on the boundary to the interior optimal control or game interpretations. We study a dynamic version of such type of boundary problems, generalizing the discrete game-theoretic approach proposed by Kohn-Serfaty (2006, 2010) for Cauchy problems and later developed by Giga-Liu (2009) and Daniel (2013) for Neumann type boundary problems.

On Equilibrium Properties of the Replicator\u2013Mutator Equation in Deterministic and Random Games

In this paper, we study the number of equilibria of the replicator–mutator dynamics for both deterministic and random multi-player two-strategy evolutionary games. For deterministic games, using Descartes’ rule of signs, we provide a formula to compute the number of equilibria in multi-player games via the number of change of signs in the coefficients of a polynomial. For two-player social dilemmas (namely the Prisoner’s Dilemma, Snow Drift, Stag Hunt and Harmony), we characterize (stable) equilibrium points and analytically calculate the probability of having a certain number of equilibria when the payoff entries are uniformly distributed. For multi-player random games whose pay-offs are independently distributed according to a normal distribution, by employing techniques from random polynomial theory, we compute the expected or average number of internal equilibria. In addition, we perform extensive simulations by sampling and averaging over a large number of possible payoff matrices to compare with and illustrate analytical results. Numerical simulations also suggest several interesting behaviours of the average number of equilibria when the number of players is sufficiently large or when the mutation is sufficiently small. In general, we observe that introducing mutation results in a larger average number of internal equilibria than when mutation is absent, implying that mutation leads to larger behavioural diversity in dynamical systems. Interestingly, this number is largest when mutation is rare rather than when it is frequent.

Iterated Function Systems with the Weak Average Contraction Conditions

This paper concerns the chaos game of random iterated function systems. We consider a random iterated function system IFS() generated by a finite family of Lipschitz maps on a compact ball of ℝn with the weak average contraction condition and show that it admits a quasi-attractor satisfying the deterministic chaos game. In particular, these properties are preserved under small perturbations of the iterated function system IFS() with respect to the Lipschitz topology.

A stochastic approach to approximate values in cooperative games

Computing additive values in cooperative games, like the Shapley value, is a hard task because, in general, it involves the summation of an exponential number of terms. We propose a new method, based on the stochastic approximation of deterministic games and sampling theory, to calculate a statistic estimate of these values and, at the same time, keeping under control estimation errors. We applied this technique to several well-known games and we show that in many cases we were able to improve previous results.

On Monte-Carlo tree search for deterministic games with alternate moves and complete information

We consider a deterministic game with alternate moves and complete information, of which the issue is always the victory of one of the two opponents. We assume that this game is the realization of a random model enjoying some independence properties. We consider algorithms in the spirit of Monte-Carlo Tree Search, to estimate at best the minimax value of a given position: it consists in simulating, successively, n well-chosen matches, starting from this position. We build an algorithm, which is optimal, step by step, in some sense: once the n first matches are simulated, the algorithm decides from the statistics furnished by the n first matches (and the a priori we have on the game) how to simulate the (n + 1)th match in such a way that the increase of information concerning the minimax value of the position under study is maximal. This algorithm is remarkably quick. We prove that our step by step optimal algorithm is not globally optimal and that it always converges in a finite number of steps, even if the a priori we have on the game is completely irrelevant. We finally test our algorithm, against MCTS, on Pearl’s game [Pearl, Artif. Intell. 14 (1980) 113–138] and, with a very simple and universal a priori, on the game Connect Four and some variants. The numerical results are rather disappointing. We however exhibit some situations in which our algorithm seems efficient.