Mixed Strategy Nash Equilibrium Research Articles

Deep Reinforcement Learning for Ecological and Distributed Urban Traffic Signal Control with Multi-Agent Equilibrium Decision Making

Multi-Agent Reinforcement Learning (MARL) has shown strong advantages in urban multi-intersection traffic signal control, but it also suffers from the problems of non-smooth environment and inter-agent coordination. However, most of the existing research on MARL traffic signal control has focused on designing efficient communication to solve the environment non-smoothness problem, while neglecting the coordination between agents. In order to coordinate among agents, this paper combines MARL and the regional mixed-strategy Nash equilibrium to construct a Deep Convolutional Nash Policy Gradient Traffic Signal Control (DCNPG-TSC) model, which enables agents to perceive the traffic environment in a wider range and achieves effective agent communication and collaboration. Additionally, a Multi-Agent Distributional Nash Policy Gradient (MADNPG) algorithm is proposed in this model, which is the first time the mixed-strategy Nash equilibrium is used for the improvement in the Multi-Agent Deep Deterministic Policy Gradient algorithm traffic signal control strategy to provide the optimal signal phase for each intersection. In addition, the eco-mobility concept is integrated into MARL traffic signal control to reduce pollutant emissions at intersections. Finally, simulation results in synthetic and real-world traffic road networks show that DCNPG-TSC outperforms other state-of-the-art MARL traffic signal control methods in almost all performance metrics, because it can aggregate the information of neighboring agents and optimize the agent’s decisions through gaming to find an optimal joint equilibrium strategy for the traffic road network.

Off-policy actor-critic deep reinforcement learning methods for alert prioritization in intrusion detection systems

Alert prioritization plays a very important role in network security as it helps security teams manage and respond to the overwhelming volume of alerts generated by intrusion detection systems (IDSs). Although current deep reinforcement learning (DRL) based deep deterministic policy gradient (DDPG) has achieved good performance for alert prioritization, there are still limitations in dealing with DDPG's instability and limited exploration capabilities. In this paper, we propose TD3-AP and SAC-AP, two DRL empowered off-policy actor-critic alert prioritization methods based on twin delayed deep deterministic policy gradient (TD3) and soft actor-critic (SAC), respectively. We model the interaction between an adversary and a defender as a zero-sum game and use a double oracle framework to obtain mixed strategy Nash equilibrium (MSNE). Our objective is to minimize the defender's loss which represents the inability of the defender to investigate alerts generated due to the attacks. To demonstrate the benefits and scalability of the proposed approaches, we conduct extensive experiments on three datasets: MQTT-IoT-IDS2020, DARPA 2000 LLDOS 1.0 and CSE-CIC-IDS2018. The results depict that TD3-AP and SAC-AP exhibit a decrease in the defender's loss by 50% and 14.28%, respectively, in comparison to the alert prioritization method based on DDPG. The improvements are significantly higher when the proposed approaches are compared with traditional alert prioritization methods including Uniform, Snort, Fuzzy and RAP. Additionally, we also provide an analysis of the interpretability of our results through the utilization of SHapley Additive exPlanations (SHAP). We assess the sensitivity of our proposed approaches to different hyperparameters and evaluate the training time and computational resources necessary for their implementation.

Multi-battle contests over complementary battlefields

AbstractThis paper studies contests with complementary prizes where each agent simultaneously distributes a fixed budget over multiple battlefields. Each battlefield has a single prize which is divided among the competitors in proportion to an arbitrary power function of their investment levels. A unique pure strategy Nash equilibrium is shown to exist under arbitrarily sensitive battlefield success functions if objective functions exhibit constant subunitary elasticity of substitution between prize shares. In contrast, Blotto contests with linear objectives have only mixed strategy Nash equilibria if battlefield success functions are sufficiently sensitive to investment levels. Sufficient complementarity between prize shares allows pure strategy Nash equilibria to exist under arbitrarily sensitive battlefield success functions.

Changing It Up: Determining the Nash Equilibria for Major League Baseball Pitchers

In order to prevent batters from hitting a pitch, pitchers must decide on a strategic balance of many different pitch types. While each pitcher has preferred pitches (likely those with which he is most confident), he cannot over-utilize his dominant pitches, or batters will be able to gain a strategic advantage in trying to put the ball into play. We analyze Major League Baseball (MLB) pitch data from 2008 to 2018 in order to determine whether or not MLB pitchers are able to reach the theorized mixed-strategy Nash equilibrium given the pitcher’s skill in utilizing each type of pitch. Our data suggests that MLB pitchers are in fact rational, and succeed in reaching the mixed-strategy Nash Equilibrium.

A multi-step attack path prediction method for oil & gas intelligence pipeline cyber physics system based on CPNE

Owing to the integration of the 5th Generation Mobile Communication Technology (5 G) and industrial internet construction, Oil & Gas intelligent pipeline network has witnessed accelerated construction and development. The Oil & Gas intelligent pipeline network proposed by China National Petroleum Corporation exhibits the advantages such as high integration, data unification, and intelligent decision-making, which helps to greatly reduce manual management costs, optimize risk decision-making by managers, and avoid risks such as human misuse. However, the highly integrated Cyber-physical system of the Internet and the Internet of Things faces various cyber security threats, and traditional intrusion detection systems exhibit drawbacks in the face of complex cyber-attack behaviors. In response to incomplete multi-step attack path prediction in traditional prediction methods, this paper proposes a multi-step attack path prediction method for Oil & Gas intelligence pipeline cyber physics system (CPS) based on Colored Petri nets and mixed strategy Nash equilibrium (CPNE). Taking into account the importance of assets and the dependency relationships between assets within the cyber-physical system of Oil & Gas intelligent pipeline network, CPNE constructs a node dependency model for the cyber-physical system of Oil & Gas intelligent pipeline network, simulates multi-step attack and defense games, calculates the probability of node failure triggering and transfer using mixed strategy Nash equilibrium, and extracts the risk of multi-step attack paths for different attack strategies. With the filtration and separation unit of a certain natural gas station as an example, this paper establishes a CPNE model to rank the risk of the first multi-step attack single attack source path and predict the risk of subsequent multi-step attack source paths, thus predicting multi-step attack paths. Taking three rounds of attacks as an example, compared to the paths predicted by traditional probability attack graph models, the proposed CPNE model is more advantageous in multi attack source path prediction, with a 2.33-fold increase in the relative coverage rate of path predictions.

Countering ARP spoofing attacks in software-defined networks using a game-theoretic approach

The Address Resolution Protocol (ARP) spoofing is a form of attack typically used by attackers to cause a denial of service or man-in-the-middle attacks. This attack comes from ARP weaknesses and aims to compromise victims' ARP caches by sending ARP packets containing fake IP-MAC pairs. To overcome ARP spoofing attacks, several approaches leverage the advantages of software-defined networking (SDN) to detect malicious users in the network. To achieve their goal, the SDN controller consecutively checks the characteristics of each ARP packet to ensure its correctness. However, this verification method can lead to latency and congestion at the controller level or render the system unusable for large-scale networks. To address this drawback, we propose a game-theoretic approach to provide an optimal verification method that considers the intelligent attackers' decision-making process during an ARP cache poisoning attempt. This approach is a zero-sum game between the attacker who wants to poison the victims' ARP caches and the defender who must avoid this poisoning. The game model results in mixed-strategy Nash equilibria that identify optimal verification methods to prevent control plane latency and congestion during attacker detection. The results show that an intelligent attacker will refrain from poisoning ARP caches with a high-impact strategy because the defender frequently checks such a strategy. In addition, the attacker's penalty value can deter both rational and irrational attackers from poisoning ARP caches. Simulations in the Mininet simulator have shown that the proposed approach can significantly mitigate control plane latency and congestion during the attacker's characteristic checking.

Study of Education and Punishment Based on Game Theory

Punishment as a means to regulate students behavior has reached a consensus in Chinese education for thousands of years, but it does not play a sufficient role in guiding the specif-ic problems in todays social practice. Based on the idea of game theory, this paper con-structs a mixed strategy game model in which schools, teachers and students participate at the same time, and through the analysis of the mixed strategy Nash equilibrium of the game model, puts forward methods to reduce students mistakes and improve students moral level. The results show that the probability of teachers implementation of educa-tional discipline is mainly related to the degree of punishment for teachers failure to im-plement educational discipline, the income and cost of paying attention to students moral cultivation; Students wrong behaviors are influenced by the spiritual benefits, costs, pun-ishments, rewards and costs obtained by teachers implementation of educational punish-ments, punishments obtained by teachers non implementation of educational punish-ments, and the benefits and costs of schools emphasis on students moral cultivation; It will be more helpful to improve the efficiency of the standardized training of students by improving the punishment for teachers failure to implement education discipline and stu-dents wrong behavior and reducing the cost of paying attention to students moral training.

Hide-and-Seek Game with Capacitated Locations and Imperfect Detection

We consider a variant of the hide-and-seek game in which a seeker inspects multiple hiding locations to find multiple items hidden by a hider. Each hiding location has a maximum hiding capacity and a probability of detecting its hidden items when an inspection by the seeker takes place. The objective of the seeker (respectively, hider) is to minimize (respectively, maximize) the expected number of undetected items. This model is motivated by strategic inspection problems, where a security agency is tasked with coordinating multiple inspection resources to detect and seize illegal commodities hidden by a criminal organization. To solve this large-scale zero-sum game, we leverage its structure and show that its mixed-strategy Nash equilibria can be characterized using their unidimensional marginal distributions, which are pure equilibria of a lower dimensional continuous zero-sum game. This leads to a two-step approach for efficiently solving our hide-and-seek game: First, we analytically solve the continuous game and derive closed-form expressions of the equilibrium marginal distributions. Second, we design a combinatorial algorithm to coordinate the players’ resources and compute equilibrium mixed strategies that satisfy the marginal distributions. We show that this solution approach computes a Nash equilibrium of the hide-and-seek game in quadratic time with linear support. Our analysis reveals novel equilibrium behaviors driven by a complex interplay between the game parameters, captured by our closed-form solutions. Funding: This work was supported by the Georgia Tech Stewart Fellowship and the Georgia Tech New Faculty Start Up Grant [for Georgia Tech New Faculty Start Up Grant]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/deca.2023.0012 .

Revisiting the Asymmetric Matching Pennies Contradiction in China.

The asymmetric matching pennies contradiction posits that contrary to the prediction of mixed-strategy Nash equilibrium, experimental subjects' choices are, in practice, based heavily on the magnitudes of their own payoffs. Own-payoff effects are robustly confirmed in the literature. Closely following the experimental setups in the literature which support the contradiction, we conduct a series of asymmetric matching pennies games in China, hypothesizing play which is closer to equilibrium frequencies than previously found. Contrary to previous experiments which were conducted in the United States, we find that there are essentially no own-payoff effects among Row players who face large payoff asymmetry. In a Quantal Response Equilibrium framework allowing for altruism or spite, the behavior of our subjects corresponded to a positive spite parameter, whereas the results of previous studies corresponded to altruism. Our results may be consistent with recent psychology literature that finds people from collectivist cultures are substantially more adept at taking the perspective of others compared with people from individualist cultures, a feature of the reasoning needed to obtain mixed-strategy equilibrium.

A Bayesian Approach with Prior Mixed Strategy Nash Equilibrium for Vehicle Intention Prediction

The state-of-the-art technology in the field of vehicle automation will lead to a mixed traffic environment in the coming years, where connected and automated vehicles have to interact with human-driven vehicles. In this context, it is necessary to have intention prediction models with the capability of forecasting how the traffic scenario is going to evolve with respect to the physical state of vehicles, the possible maneuvers and the interactions between traffic participants within the seconds to come. This article presents a Bayesian approach for vehicle intention forecasting, utilizing a game-theoretic framework in the form of a Mixed Strategy Nash Equilibrium (MSNE) as a prior estimate to model the reciprocal influence between traffic participants. The likelihood is then computed based on the Kullback-Leibler divergence. The game is modeled as a static nonzero-sum polymatrix game with individual preferences, a well known strategic game. Finding the MSNE for these games is in the PPAD cap PLS complexity class, with polynomial-time tractability. The approach shows good results in simulations in the long term horizon (10s), with its computational complexity allowing for online applications.

A game theory approach to land acquisition conflicts in Pakistan

Rapid urbanization in Pakistan requires new public infrastructures, which necessitates land acquisitions. However, Land Acquisition Act, 1894 (LAA, 1894) is primitive, recessive, complex, and prone to land conflicts, requiring systematic research exploring the logic of the legal conduct. This paper converts LAA, 1894 into an extensive-form game to explain the logic of land acquisition conflicts in Pakistan, as incumbents interact strategically. The Nash equilibrium (NE) of the extensive-form game suggests Land Acquisition Collector has an absolute advantage over settlers, reasoned by flawed compensation assessment. Thus, a bad win-win situation induces illegitimate behavior, extending the paper to Artificial Price Hike and Corruption-based game models. The mixed-strategy NE of both models suggests that illegitimate behaviors remain unchecked because of the high litigation costs, lack of transparency, the informality of land, and insufficient cadastral information. Finally, the models are calibrated with current land acquisition conflicts and legal precedents in the study area. This paper provides generalized interactive game models applicable to study the logic of land conflicts amid legal and policy implementation and suggests policies to improve the land administration and legal system to manage conflicts.

Correlated Equilibrium and Evolutionary Stability in 3-Player Rock-Paper-Scissors

In the game of rock-paper-scissors with three players, this paper identifies conditions for a correlated equilibrium that differs from the mixed strategy Nash equilibrium and is evolutionarily stable. For this to occur, the correlation device attaches more probability to three-way ties and solo-winner outcomes than would result from the Nash equilibrium. The correlated equilibrium is evolutionarily stable because any mutant fares worse than a signal-following player when facing two players who follow their own correlated signals. The critical quality of the correlation device is to make this true both for potential mutants who would disobey their signal and instead choose the action which would beat the action signaled to the player, as well as for potential mutants who would deviate to the action that would be beaten by what the device signals to the player. These findings reveal how a strict correlated equilibrium can produce evolutionarily stable strategies for rock-paper-scissors with three players.

Optimal strategy of the simultaneous dice game Pig for multiplayers: when reinforcement learning meets game theory

In this work, we focus on using reinforcement learning and game theory to solve for the optimal strategies for the dice game Pig, in a novel simultaneous playing setting. First, we derived analytically the optimal strategy for the 2-player simultaneous game using dynamic programming, mixed-strategy Nash equilibrium. At the same time, we proposed a new Stackelberg value iteration framework to approximate the near-optimal pure strategy. Next, we developed the corresponding optimal strategy for the multiplayer independent strategy game numerically. Finally, we presented the Nash equilibrium for simultaneous Pig game with infinite number of players. To help promote the learning of and interest in reinforcement learning, game theory and statistics, we have further implemented a website where users can play both the sequential and simultaneous Pig game against the optimal strategies derived in this work.

Differential N-players game: Nash equilibria and Mather measures

Abstract We study Nash equilibria for the deterministic ergodic N-players game. We introduce pure strategies, mixed strategies and Nash equilibria associated with those. We show that a Nash equilibrium in mixed strategies exists and it is a Mather measure for the Lagrangian system defined by the cost functional. In conclusion, we show that the mean field limit of the N-players game is described by the ergodic partial differential equation’s system for a continuum of players.

Research on Network Security Situation Awareness and Dynamic Game Based on Deep Q Learning Network

<p>In today’s increasingly complex network environment, increasingly severe network attacks, and real-time dynamic changes in offensive and defensive scenarios, network security technology has also evolved from passive security to active security technology, and expanded from analyzing unilateral security elements to comprehensively analyzing overall network security. Aiming at the problem of inaccurate assessment results due to the lack of comprehensive analysis of threat information, protection information and environmental information in existing assessment methods, this paper proposes a defensive random game model. This model analyzes threat propagation and establishes a threat propagation access relationship network, and then establishes a random game model for the game process of threat action and protection strategy implementation to solve the mixed strategy Nash equilibrium. The model comprehensively analyzes the dynamic changes of the security situation elements, ignoring the short-term situation elements that will not change, such as topology structure, service information, etc., and mainly analyzes the dynamic changes of attack information, vulnerability information, and defense measures, and predicts from the host and network levels. The experimental verification shows that the prediction method in this paper can improve the prediction accuracy and is more in line with the offensive and defensive scenarios.</p> <p> </p>

On the solution of games with arbitrary payoffs: An application to an over‐the‐counter financial market

AbstractThis paper defines a variety of game theoretic solution concepts in the language of soft set theory. We begin by defining the Nash equilibrium in pure strategies. We assume that the gains of the players are totally ordered and non‐desirable alternatives are absent. Moreover, we introduce the notions of strong and semi‐strong utility. These two completely new notions, serve as a mechanism for converting non‐ordered gains into totally ordered ones. We define the Nash equilibrium in mixed strategies in a general framework by introducing the notion of an extended game and strategy space. We finally define the Nash solution to cooperative bargaining games within the framework of soft set theory, illustrate a practical application to an over‐the‐counter financial market, and provide a detailed numerical example.

Disequilibrium Play in Tennis

Are the serves of the world’s best tennis pros consistent with the theoretical prediction of Nash equilibrium in mixed strategies? We analyze their serve direction choices (to the returner’s left, right or body) with data from an online database called the Match Charting Project. Using a new methodology, we test and decisively reject a key implication of a mixed strategy Nash equilibrium, namely, that the probability of winning a service game is the same for all serve directions. We also use dynamic programming (DP) to numerically solve for the best-response serve strategies to probability models of service game outcomes estimated for individual server-returner pairs, such as Novak Djokovic serving to Rafael Nadal. We show that for most elite pro servers, the DP serve strategy significantly increases their service game win probability compared to the mixed strategies they actually use, which we estimate using flexible reduced-form logit models. Stochastic simulations verify that our results are robust to estimation error. Classification- C61, C73, L21

Nash equilibrium realization of population games based on social learning processes.

In the two-population game model, we assume the players have certain imitative learning abilities. To simulate the learning process of the game players, we propose a new swarm intelligence algorithm by combining the particle swarm optimization algorithm, where each player can be considered a particle. We conduct simulations for three typical games: the prisoner's dilemma game (with only one pure-strategy Nash equilibrium), the coin-flip game (with only one fully-mixed Nash equilibrium), and the coordination game (with two pure-strategy Nash equilibria and one fully-mixed Nash equilibrium). The results show that when the game has a pure strategy Nash equilibrium, the algorithm converges to that equilibrium. However, if the game does not have a pure strategy Nash equilibrium, it exhibits periodic convergence to the only mixed-strategy Nash equilibrium. Furthermore, the magnitude of the periodical convergence is inversely proportional to the introspection rate. After conducting experiments, our algorithm outperforms the Meta Equilibrium Q-learning algorithm in realizing mixed-strategy Nash equilibrium.

Implicit coordination in an <inline-formula><tex-math id="M1">$ n $</tex-math></inline-formula>-player target selection game

We show that a class of target selection games has a unique symmetric mixed strategy Nash equilibrium $ x^\ast $. We also present a simple recursion that computes $ x^\ast $ within a finite number of iterations. Along the way, we introduce the concept of value compactness that points directly to the properties of the unique symmetric mixed strategy equilibrium, including the fact that the unique equilibrium does not force any player to make an arbitrary decision about associating positive probability with subsets of targets having a particular utility– a property that we refer to as implicit coordination.

Secure Downlink Transmission Strategies against Active Eavesdropping in燦OMA Systems: A Zero-Sum Game Approach

Non-orthogonal multiple access technology (NOMA), as a potentially promising technology in the 5G/B5G era, suffers from ubiquitous security threats due to the broadcast nature of the wireless medium. In this paper, we focus on artificial-signal-assisted and relay-assisted secure downlink transmission schemes against external eavesdropping in the context of physical layer security, respectively. To characterize the non-cooperative confrontation around the secrecy rate between the legitimate communication party and the eavesdropper, their interactions are modeled as a two-person zero-sum game. The existence of the Nash equilibrium of the proposed game models is proved, and the pure strategy Nash equilibrium and mixed-strategy Nash equilibrium profiles in the two schemes are solved and analyzed, respectively. The numerical simulations are conducted to validate the analytical results, and show that the two schemes improve the secrecy rate and further enhance the physical layer security performance of NOMA systems.