Strategy Nash Equilibrium Research Articles

Stability analysis of networked control systems under DoS attacks in frequency domain via game theory strategy

In this paper, the stability analysis of networked control systems (NCSs) under Denial-of-Service (DoS) attacks is studied. Firstly, the stability analysis for NCSs free of DoS attacks is investigated. Considering the DoS attacks, the game theory is introduced to establish the non-cooperative game between defender and attacker of NCSs. At the same time, the stability of NCSs is analysed in frequency domain, and the signal-to-noise ratio (SNR) expression is given by using spectral decomposition technique and Nash equilibrium (NE) strategy. The packet dropout phenomenon caused by DoS attacks is described by Bernoulli distribution. By virtue of the NE strategy, the stability of NCSs can be optimised. In the obtained results, the influence of the traditional constraints and DoS attacks is illustrated. Finally, some numerical examples are given to show the effectiveness of the derived theoretical results.

Non-zero sum differential game for stochastic Markovian jump systems with partially unknown transition probabilities

This paper focuses on the non-zero sum differential game problem for Markovian jump systems with partially unknown transition probabilities. Firstly, a suboptimal control problem is studied by the free-connection weighting matrix method, and then the non-zero sum differential game problem is investigated on this basis. Several sufficient conditions for the existence of ε-suboptimal control strategy and ε-suboptimal Nash equilibrium strategies are provided, and their explicit expressions are designed. Moreover, the precise form for the upper bound of the cost function is also given. To facilitate the calculation, all conditions are converted into the corresponding equivalent linear matrix inequalities or bilinear matrix inequalities form. Finally, two numerical examples are utilized to demonstrate the effectiveness of the main results.

Distributed Task Offloading Optimization With Queueing Dynamics in Multiagent Mobile-Edge Computing Networks

Task offloading decision making plays a key role in enabling mobile-edge computing (MEC) technologies in Internet of Things (IoT). However, it meets the significant challenges arising from the stochastic dynamics of task queueing in the application layer and coupled wireless interference in the physical layer in a distributed multiagent network without any centralized communication and computing coordination. In this article, we investigate the distributed task offloading optimization problem with consideration of the upper layer queueing dynamics and the lower-layer coupled wireless interference. We first propose a new optimization model that aims at maximizing the expected offloading rate of multiple agents by optimizing their offloading thresholds. Then, we transform the problem into a game-theoretic formulation, which further leads to the design of a distributed best-response (DBR) iterative optimization framework. The existence of Nash equilibrium strategies in the game-theoretic model has been analyzed. For the individual optimization of each agent's threshold policy, we further propose a programming scheme by transforming a constrained threshold optimization into an unconstrained Lagrangian optimization (ULO). The individual ULO is integrated into the DBR framework to enable agents to cooperate and converge to a global optimum in a distributed manner. Finally, simulation results are provided to validate the proposed method and demonstrate its significant advantage over other existing distributed methods. The numerical results also show that the proposed method can achieve comparable performance to a centralized optimization method.

Production/inventory competition between firms with fixed-proportions co-production systems

We study the production/inventory competition problem between two manufacturers each adopting the co-production system to process a raw material into two products in fixed proportions that are substitutable between the manufacturers. We first consider the one-period problem where the demands for the two manufacturers’ products are uncertain and independent. The unmet demand of one manufacturer's product can be met by the other manufacturer's leftover stock of the same product, if available, and is lost otherwise. The manufacturers compete for the substitute demands by choosing their own purchase/processing quantities. We show the existence of the unique Nash equilibrium processing quantity, and examine the impact of competition by comparing the total equilibrium quantity of the game with that of the centralized-decision case. We then study the multi-period production/inventory competition problem in which the manufacturers make decisions in each period according to the initial inventory levels of the products and the rivals’ decisions. We show the existence and uniqueness of the Nash equilibrium strategy and that the equilibrium strategy has the simple base-stock structure. We then consider the case where the manufacturers’ demands are correlated. In addition, studying the case where the manufacturers are capacitated, we show that the unique Nash equilibrium exists and has a modified base-stock structure. We further generalize the results to the infinite-period case and the case with more than two products. Finally, numerical studies are conducted to investigate the impacts of the model parameters on the equilibrium outcomes and get some managerial insights from the analytical findings.

Nash equilibrium and group strategy consensus of networked evolutionary game with coupled social groups

This paper studies the existence of Nash equilibrium and group strategy consensus problem of networked evolutionary game with coupled social groups, which is formulated as a two-layer networked game. First, by the approach of semi-tensor product, the payoff functions of players are converted into the algebraic forms, and two necessary and sufficient conditions are derived for the existence of pure Nash equilibrium. Second, the strict game algebraic equation by the best response adjustment rule with group intelligence is established, by which the group strategy consensus problem is investigated. At last, an example is worked out to verify our conclusions.

Algorithm for Computing Approximate Nash Equilibrium in Continuous Games with Application to Continuous Blotto

Successful algorithms have been developed for computing Nash equilibrium in a variety of finite game classes. However, solving continuous games—in which the pure strategy space is (potentially uncountably) infinite—is far more challenging. Nonetheless, many real-world domains have continuous action spaces, e.g., where actions refer to an amount of time, money, or other resource that is naturally modeled as being real-valued as opposed to integral. We present a new algorithm for approximating Nash equilibrium strategies in continuous games. In addition to two-player zero-sum games, our algorithm also applies to multiplayer games and games with imperfect information. We experiment with our algorithm on a continuous imperfect-information Blotto game, in which two players distribute resources over multiple battlefields. Blotto games have frequently been used to model national security scenarios and have also been applied to electoral competition and auction theory. Experiments show that our algorithm is able to quickly compute close approximations of Nash equilibrium strategies for this game.

A receding horizon data‐driven chance‐constrained approach for energy flexibility trading in multi‐microgrid distribution network

Unexpected imbalances between supply and demand during real-time operation in the wake of high penetration of renewable energy resources necessitate system operators to improve flexibility of the system. Moreover, increasing growth in flexible distributed resources installation in distribution networks with multiple MicroGrids (MGs) such as fuel-based distributed generators, storage units, and controllable loads makes distribution system operators be able to manage energy imbalances locally through exploiting potentials of aforementioned resources in flexibility enhancement. This paper proposes a receding horizon data-driven chance-constrained energy flexibility trading framework for active distribution networks with multiple MGs. Specifically, a distance-based confidence set is constructed in a data-driven manner to apply net load uncertainty into this modelling. According to this proposed data-driven approach, the energy flexibility is modelled using flexibility envelope notion to demonstrate its dynamics in different scenarios in an imminent horizon. Furthermore, MGs' flexibility trading problem is modelled as an exact potential game that possesses a pure strategy Nash Equilibrium (NE). An iterative decentralized algorithm based on alternating direction method of multipliers is developed to achieve the NE strategy of MGs. Numerical results are also represented to show the performance efficiency of the proposed energy flexibility trading framework.

Optimized control for human-multi-robot collaborative manipulation via multi-player Q-learning

In this paper, optimized interaction control is investigated for human-multi-robot collaboration control problems, which cannot be described by the traditional impedance controller. To realize global optimized interaction performance, the multi-player non-zero sum game theory is employed to obtain the optimized interaction control of each robot agent. Regarding the game strategies, Nash equilibrium strategy is utilized in this paper. In human-multi-robot collaboration problems, the dynamics parameters of the human arm and the manipulated object are usually unknown. To obviate the dependence on these parameters, the multi-player Q-learning method is employed. Moreover, for the human-multi-robot collaboration problem, the optimized solution is difficult to resolve due to the existence of the desired reference position. A multi-player Nash Q-learning algorithm considering the desired reference position is proposed to deal with the problem. The validity of the proposed method is verified through simulation studies.

Estimating Card Fitness for Discard in Gin Rummy

Due to the computation time and resources required, there is no known optimal strategy for the game of Gin Rummy. Previous work in extensive games, such as Texas Hold ’em Poker, has found that hand fitness and information sets about the state of the game can be used to determine an improved strategy. These information sets, combined with algorithms for Counterfactual Regret Minimization, can arrive at a Nash Equilibrium strategy for smaller abstractions of extensive games. This paper builds on previous research by extending the premise of hand fitness to card fitness in the discard decision point of Gin Rummy. We argue that a card can be ranked based on whether it meets four specific characteristics at that stage in the game. These characteristics include its effect on deadwood points after one more turn, its utility to the opponent, and if it can contribute to a meld. An optimal discard choice can then be picked from the highest-ranked card by using a simplified Counterfactual regret minimization strategy that can be trained in less time due to its limited information set. While this does not look at every potential characteristic of card fitness, it outperformed other bots when evaluated in a large number of games. These bots did not consider card fitness as a whole, but rather considered characteristics separately. We argue that the characteristics defined are a part of the total information set that can determine the discard fitness of a card within a hand in the game of Gin Rummy.

Reactive Strategies: An Inch of Memory, a Mile of Equilibria

We explore how an incremental change in complexity of strategies (“an inch of memory”) in repeated interactions influences the sets of Nash equilibrium (NE) strategy and payoff profiles. For this, we introduce the two most basic setups of repeated games, where players are allowed to use only reactive strategies for which a probability of players’ actions depends only on the opponent’s preceding move. The first game is trivial and inherits equilibria of the stage game since players have only unconditional (memory-less) Reactive Strategies (RSs); in the second one, players also have conditional stochastic RSs. This extension of the strategy sets can be understood as a result of evolution or learning that increases the complexity of strategies. For the game with conditional RSs, we characterize all possible NE profiles in stochastic RSs and find all possible symmetric games admitting these equilibria. By setting the unconditional benchmark as the least symmetric equilibrium payoff profile in memory-less RSs, we demonstrate that for most classes of symmetric stage games, infinitely many equilibria in conditional stochastic RSs (“a mile of equilibria”) Pareto dominate the benchmark. Since there is no folk theorem for RSs, Pareto improvement over the benchmark is the best one can gain with an inch of memory.

Resilient strategy design for cyber-physical system under active eavesdropping attack

We consider a remote state estimation process under an active eavesdropper for cyber-physical system. A smart sensor transmits its local state estimates to a remote estimator over an unreliable network, which is eavesdropped by an adversary. The intelligent adversary can work in passive eavesdropping mode and active jamming mode. An active jamming mode enables the adversary to interfere the data transmission from sensor to estimator, and meanwhile improve the data reception of itself. To protect the transmission data from being wiretapped, the sensor with two antennas injects noise to the eavesdropping link with different power levels. Aiming at minimizing the estimation error covariance and power cost of themselves while maximizing the estimation error covariance of their opponents, a two-player nonzero-sum game is constructed for sensor and active eavesdropper. For an open-loop case, the mixed Nash equilibrium is obtained by solving an one-stage nonzero-sum game. For a long term consideration, a Markov stochastic game is introduced and a Nash Q-learning method is given to find the Nash equilibrium strategies for two players. Numerical results are provided to show the effectiveness of our theoretical conclusions.

Does mutual knowledge of preferences lead to more Nash equilibrium play? Experimental evidence

Nash equilibrium often does not seem to accurately predict behavior. In experimental game theory, it is usually assumed that the monetary payoffs in the game represent subjects’ utilities. However, subjects may actually play a very different game. In this case, mutual knowledge of preferences may not be satisfied. In our experiment, we first elicit subjects’ preferences over the monetary payoffs for all players. This allows us to identify equilibria in the games that subjects actually are playing (the preference games). We then examine whether revealing other subjects’ preferences leads to more equilibrium play and find that this information indeed has a significant effect. Furthermore, it turns out that subjects are more likely to play maxmin and maxmax strategies than Nash equilibrium strategies.

Nash equilibria in nonzero-sum differential games with impulse control

In this paper, we introduce a class of deterministic finite-horizon two-player nonzero-sum differential games where one player uses ordinary controls while the other player uses impulse controls. We use the word ‘ordinary’ to mean that Player 1 uses control strategies that are piecewise continuous functions of time. We formulate the necessary and sufficient conditions for the existence of an open-loop Nash equilibrium for this class of differential games. We specialize these results to linear-quadratic games, and show that the open-loop Nash equilibrium strategies can be computed by solving a constrained non-linear optimization problem. In particular, for the impulse player, the equilibrium timing and level of impulses can be obtained. Furthermore, for the special case of linear-state differential games, we obtain analytical characterization of equilibrium number, timing, and the level of impulse in terms of the problem data. We illustrate our results using numerical experiments.

A note on a Tarski type fixed-point theorem

In this paper we propose a basic fixed-point theorem for correspondences inspired by Tarski’s intersection point theorem. This result furnishes an efficient tool to prove the existence of pure strategy Nash equilibria for two player games with possibly discontinuous payoffs functions defined on compact real intervals.

A novel cooperative game-based method to coordinate a sustainable supply chain under psychological uncertainty in fairness concerns

This paper investigates how retailers’ fairness concerns affect cooperative relationships in a three-party sustainable supply chain (TSSC) and how to coordinate such a supply chain when the degree of fairness concern is treated as an interval. Utilizing the Nash equilibrium strategy, this study seeks equilibrium solutions and profits under five non-cooperative and cooperative models and reveals that fairness concerns indeed affect members’ decisions and their cooperation for sustainable supply chain management. Furthermore, we develop a novel coordination method by incorporating the interval-valued least square pre-nucleolus (IVLSPN) approach and the three-party Nash bargaining game model to perfectly coordinate this TSSC.

Trading and Pricing Sensor Data in Competing Edge Servers with Double Auction Markets

With the development of the IoT (Internet of Things), sensors networks can bring a large amount of valuable data. In addition to be utilized in the local IoT applications, the data can also be traded in the connected edge servers. As an efficient resource allocation mechanism, the double auction has been widely used in the stock and futures markets and can be also applied in the data resource allocation in sensor networks. Currently, there usually exist multiple edge servers running double auctions competing with each other to attract data users (buyers) and producers (sellers). Therefore, the double auction market run on each edge server needs efficient mechanism to improve the allocation efficiency. Specifically, the pricing strategy of the double auction plays an important role on affecting traders’ profit, and thus, will affect the traders’ market choices and bidding strategies, which in turn affect the competition result of double auction markets. In addition, the traders’ trading strategies will also affect the market’s pricing strategy. Therefore, we need to analyze the double auction markets’ pricing strategy and traders’ trading strategies. Specifically, we use a deep reinforcement learning algorithm combined with mean field theory to solve this problem with a huge state and action space. For trading strategies, we use the Independent Parametrized Deep Q‐Network (I‐PDQN) algorithm combined with mean field theory to compute the Nash equilibrium strategies. We then compare it with the fictitious play (FP) algorithm. The experimental results show that the computation speed of I‐PDQN algorithm is significantly faster than that of FP algorithm. For pricing strategies, the double auction markets will dynamically adjust the pricing strategy according to traders’ trading strategies. This is a sequential decision‐making process involving multiple agents. Therefore, we model it as a Markov game. We adopt Multiagent Deep Deterministic Policy Gradient (MADDPG) algorithm to analyze the Nash equilibrium pricing strategies. The experimental results show that the MADDPG algorithm solves the problem faster than the FP algorithm.

Linear-quadratic zero-sum mean-field type games: Optimality conditions and policy optimization

<p style='text-indent:20px;'>In this paper, zero-sum mean-field type games (ZSMFTG) with linear dynamics and quadratic cost are studied under infinite-horizon discounted utility function. ZSMFTG are a class of games in which two decision makers whose utilities sum to zero, compete to influence a large population of indistinguishable agents. In particular, the case in which the transition and utility functions depend on the state, the action of the controllers, and the mean of the state and the actions, is investigated. The optimality conditions of the game are analysed for both open-loop and closed-loop controls, and explicit expressions for the Nash equilibrium strategies are derived. Moreover, two policy optimization methods that rely on policy gradient are proposed for both model-based and sample-based frameworks. In the model-based case, the gradients are computed exactly using the model, whereas they are estimated using Monte-Carlo simulations in the sample-based case. Numerical experiments are conducted to show the convergence of the utility function as well as the two players' controls.</p>

Branching improved Deep Q Networks for solving pursuit-evasion strategy solution of spacecraft

<p style='text-indent:20px;'>With the continuous development of space rendezvous technology, more and more attention has been paid to the study of spacecraft orbital pursuit-evasion differential game. Therefore, we propose a pursuit-evasion game algorithm based on branching improved Deep Q Networks to obtain a space rendezvous strategy with non-cooperative target. Firstly, we transform the optimal control of space rendezvous between spacecraft and non-cooperative target into a survivable differential game problem. Next, in order to solve this game problem, we construct Nash equilibrium strategy and test its existence and uniqueness. Then, in order to avoid the dimensional disaster of Deep Q Networks in the continuous behavior space, we construct a TSK fuzzy inference model to represent the continuous space. Finally, in order to solve the complex and timeconsuming self-learning problem of discrete action sets, we improve Deep Q Networks algorithm, and propose a branching architecture with multiple groups of parallel neural Networks and shared decision modules. The simulation results show that the algorithm achieves the combination of optimal control and game theory, and further improves the learning ability of discrete behaviors. The algorithm has the comparative advantage of continuous space behavior decision, can effectively deal with the continuous space chase game problem, and provides a new idea for the solution of spacecraft orbit pursuit-evasion strategy.</p>

SG-PAC: A stochastic game approach to generate personal privacy paradox access-control policies in social networks

Online social networks are indispensable platforms where people share personal data from their daily lives. However, social sharing raises many privacy issues. One of the most difficult, the “privacy paradox,” is that social-network users want to meet their social needs by enhancing interactions with their friends and are at the same time concerned about the risk of privacy leakage. This work aims to help social-network users weigh the benefits of social sharing and the costs of privacy leakage. A proposed criterion for privacy-risk measurement can quantify the probability of privacy leakage according to multi-person competitive behaviors. By analyzing the continuous competition between a user and some unwanted knowers, a general-sum stochastic game model related to the privacy paradox is constructed. The Markov decision process (MDP) and a role-mining method are introduced to facilitate the game analysis. The existence of a Nash equilibrium strategy is proved theoretically, and effective personal access-control policies are derived from this strategy as solved by a reinforcement learning algorithm. Experiments evaluate the constructed privacy stochastic game model and the new criterion for privacy-risk measurement. This work helps users develop effective personal policies in all cases to optimize their payoffs in the privacy paradox.

Mean-field backward–forward stochastic differential equations and nonzero sum stochastic differential games

We study a general class of fully coupled backward–forward stochastic differential equations of mean-field type (MF-BFSDE). We derive existence and uniqueness results for such a system under weak monotonicity assumptions and without the non-degeneracy condition on the forward equation. This is achieved by suggesting an implicit approximation scheme that is shown to converge to the solution of the system of MF-BFSDE. We apply these results to derive an explicit form of open-loop Nash equilibrium strategies for nonzero sum mean-field linear-quadratic stochastic differential games with random coefficients. These strategies are valid for any time horizon of the game.