Two-player Game Research Articles

Enhancing Two-Player Performance Through Single-Player Knowledge Transfer: An Empirical Study on Atari 2600 Games

Playing two-player games using reinforcement learning and self-play can be challenging due to the complexity of two-player environments and the possible instability in the training process. We propose that a reinforcement learning algorithm can train more efficiently and achieve improved performance in a two-player game if it leverages the knowledge from the single-player version of the same game. This study examines the proposed idea in ten different Atari 2600 environments using the Atari 2600 RAM as the input state. We discuss the advantages of using transfer learning from a single-player training process over training in a two-player setting from scratch, and demonstrate our results in a few measures such as training time and average total reward. We also discuss a method of calculating RAM complexity and its relationship to performance.

On the implementation of zero-determinant strategies in repeated games

Zero-determinant strategies are a class of strategies in repeated games which unilaterally control payoffs. Zero-determinant strategies have attracted much attention in studies of social dilemma, particularly in the context of evolution of cooperation. So far, not only general properties of zero-determinant strategies have been investigated, but zero-determinant strategies have been applied to control in the fields of information and communications technology and analysis of imitation. Here, we further deepen our understanding on general mathematical properties of zero-determinant strategies. We first prove that zero-determinant strategies, if exist, can be implemented by some one-dimensional transition probability. Next, we prove that, if a two-player game has a non-trivial potential function, a zero-determinant strategy exists in its repeated version. These results assist us to implement zero-determinant strategies in broader situations.

Nash meets Łukasiewicz: computing equilibria through logic

Abstract A Nash equilibrium is a strategy profile of a game in which none of the players involved has any gain by changing alone his/her own strategy. Given a two-player game, we show a codification of all of its Nash equilibria into Łukasiewicz infinitely-valued logic, that is, we derive a propositional theory in this logic whose models codify all the Nash equilibria. Based on such propositional theory, we derive a polynomial reduction from the problem of computing a Nash equilibrium to the problem of satisfiability of (sets of) formulas of Łukasiewicz infinitely-valued logic. These applications of logic to game theory lead to new methods for computing Nash equilibria.

An individual evolutionary learning model meets Cournot

We extend the individual evolutionary learning model by incorporating other-regarding considerations and apply the model to some Cournot games. Using a model fitted to the experimental data of a repeated three-player Cournot game (with non-linear cost and demand functions), we provide out-of-sample predictions regarding the “feedback effects” and “number effects” and test them using the data gathered via newly conducted experiments. The prediction regarding the feedback effect is partially confirmed. Namely, it is observed for the three- and four-player games, but not the two-player game. The prediction regarding the number effect is also partially confirmed in that while the model predicts the number effect to be observed with detailed feedback, and not under aggregate feedback, the effect is observed with both types of feedback.

The equilibrium strategy of insurance companies’ dividends and reinsurance games

We investigated the optimal risk management strategies of an insurance company within a two-player non-cooperative differential game framework. The key feature of this model is considering the company’s surplus as a strategic variable to make the insurance company more attractive to shareholders. In this setup, shareholders are participants who demand a share of the surplus, while managers are participants concerned with the risk management of the surplus. The objective of the shareholders is to maximize the expected discounted dividends. We address this asymmetric game under two different assumptions about the managers’ objectives: in the first scenario, the managers aim to minimize the probability of bankruptcy; in the second scenario, the managers aim to maximize the expected discounted utility derived from the surplus.

Identifying influential nodes in social networks from the perspective of attack-defense game.

Influence spread analysis, a critical component of social network studies, focuses on the patterns and effects of information dissemination among interconnected entities. The core of influence spread analysis is to identify influential nodes that involve two distinct aspects: influence maximization (IM) and influence blocking maximization (IBM). However, when IM and IBM occur simultaneously, identifying influential nodes becomes an intricate decision-making challenge. This study addresses identifying influential nodes in social networks through an attack-defense game perspective, where an attacker maximizes influence and a defender minimizes it. We first develop a two-player static zero-sum game model considering resource constraints. Based on the equilibrium strategy of this game, we redefine the concept of influential nodes from various viewpoints. Extensive experiments on synthetic and real-world networks show that, in most cases, the defender preferentially defends critical nodes, while the attacker adopts the decentralized strategy. Only when resources are unevenly matched do both players tend to adopt centralized strategies. This study expands the connotation of influential nodes and provides a novel paradigm for the social network analysis with significant potential applications.

Stationary Markov Equilibrium Strategies in Asynchronous Stochastic Games: Existence and Computation

We study Asynchronous Dynamic games and show that in games with a finite state space and finite action sets, one can obtain the pure strategy Markov perfect equilibrium by using a simple backward induction method when the time period for the game is finite. The equilibrium strategies for games with an infinite horizon are then obtained as the point-wise limit of the equilibrium strategies of a sequence of finite horizon games, where the finite horizon games are truncated versions of the original game with successively longer time periods. We also show that if the game has a fixed K-period cycle, then there is a stationary Markov equilibrium. Using these results, we derive an algorithm to compute the equilibrium strategies. We test the algorithm in three experiments. The first is a two-player asynchronous game with three states and three actions. In the second experiment, we compute the equilibrium of a cybersecurity game in which there are two players, an attacker and a defender. In the third experiment, we compute the stationary equilibrium of a duopoly game with two firms that choose an output in alternate periods.

Off-policy reinforcement-learning-based fault-tolerant [formula omitted] control for topside separation systems with time-varying uncertainties

The topside separation system plays a pivotal role in the treatment of produced water within offshore oil and gas production operations. Due to high-humidity and salt-infested marine environments, topside separation systems are susceptible to dynamic model variations and valve faults. In this work, fault-tolerant control (FTC) of topside separation systems subject to structural uncertainties and slugging disturbances is studied. The system is configured as a cascade structure, comprising a water level control subsystem and a pressure-drop-ratio (PDR) control subsystem. A fault-tolerant H∞ control framework is developed to cope with actuator faults and slugging disturbances. To enhance control performance in the presence of actuator faults and model uncertainties while reducing sensitivity to slugging disturbances, the fault-tolerant H∞ control problem for the topside separation system is established as the two-player differential game problem. In addition, a Nash equilibrium solution for the fault-tolerant H∞ control problem is achieved through the solution of the game algebraic Riccati equation (GARE). A model-free approach is presented to implement the proposed fault-tolerant H∞ control method using off-policy reinforcement learning (RL). Simulation studies demonstrate the effectiveness of the solution.

Strategic safeguarding: A game theoretic approach for analyzing attacker-defender behavior in DNN backdoors

Deep neural networks (DNNs) are fundamental to modern applications like face recognition and autonomous driving. However, their security is a significant concern due to various integrity risks, such as backdoor attacks. In these attacks, compromised training data introduce malicious behaviors into the DNN, which can be exploited during inference or deployment. This paper presents a novel game-theoretic approach to model the interactions between an attacker and a defender in the context of a DNN backdoor attack. The contribution of this approach is multifaceted. First, it models the interaction between the attacker and the defender using a game-theoretic framework. Second, it designs a utility function that captures the objectives of both parties, integrating clean data accuracy and attack success rate. Third, it reduces the game model to a two-player zero-sum game, allowing for the identification of Nash equilibrium points through linear programming and a thorough analysis of equilibrium strategies. Additionally, the framework provides varying levels of flexibility regarding the control afforded to each player, thereby representing a range of real-world scenarios. Through extensive numerical simulations, the paper demonstrates the validity of the proposed framework and identifies insightful equilibrium points that guide both players in following their optimal strategies under different assumptions. The results indicate that fully using attack or defense capabilities is not always the optimal strategy for either party. Instead, attackers must balance inducing errors and minimizing the information conveyed to the defender, while defenders should focus on minimizing attack risks while preserving benign sample performance. These findings underscore the effectiveness and versatility of the proposed approach, showcasing optimal strategies across different game scenarios and highlighting its potential to enhance DNN security against backdoor attacks.

Different strokes in randomised strategies: Revisiting Kuhn's theorem under finite-memory assumptions

Two-player (antagonistic) games on (possibly stochastic) graphs are a prevalent model in theoretical computer science, notably as a framework for reactive synthesis.Optimal strategies may require randomisation when dealing with inherently probabilistic goals, balancing multiple objectives, or in contexts of partial information. There is no unique way to define randomised strategies. For instance, one can use so-called mixed strategies or behavioural ones. In the most general setting, these two classes do not share the same expressiveness. A seminal result in game theory — Kuhn's theorem — asserts their equivalence in games of perfect recall.This result crucially relies on the possibility for strategies to use infinite memory, i.e., unlimited knowledge of all past observations. However, computer systems are finite in practice. Hence it is pertinent to restrict our attention to finite-memory strategies, defined as automata with outputs. Randomisation can be implemented in these in different ways: the initialisation, outputs or transitions can be randomised or deterministic respectively. Depending on which aspects are randomised, the expressiveness of the corresponding class of finite-memory strategies differs.In this work, we study two-player concurrent stochastic games and provide a complete taxonomy of the classes of finite-memory strategies obtained by varying which of the three aforementioned components are randomised. Our taxonomy holds in games of perfect and imperfect information with perfect recall, and in games with more than two players. We also provide an adapted taxonomy for games with imperfect recall.

Generalised Kochen–Specker theorem for finite non-deterministic outcome assignments

The Kochen–Specker (KS) theorem is a cornerstone result in quantum foundations, establishing that quantum correlations in Hilbert spaces of dimension d ≥ 3 cannot be explained by (consistent) hidden variable theories that assign a single deterministic outcome to each measurement. Specifically, there exist finite sets of vectors in these dimensions such that no non-contextual deterministic ({0, 1}) outcome assignment is possible obeying the rules of exclusivity and completeness—that the sum of assignments to every set of mutually orthogonal vectors be ≤1 and the sum of value assignments to any d mutually orthogonal vectors be equal to 1. Another central result in quantum foundations is Gleason’s theorem that justifies the Born rule as a mathematical consequence of the quantum formalism. The KS theorem can be seen as a consequence of Gleason’s theorem and the logical compactness theorem. In a similar vein, Gleason’s theorem also indicates the existence of KS-type finite vector constructions to rule out other finite-alphabet outcome assignments beyond the {0, 1} case. Here, we propose a generalisation of the KS theorem that rules out hidden variable theories with outcome assignments in the set {0, p, 1 − p, 1} for p ∈ [0, 1/d) ∪ (1/d, 1/2]. The case p = 1/2 is especially physically significant. We show that in this case the result rules out (consistent) hidden variable theories that are fundamentally binary, i.e., theories where each measurement has fundamentally at most two outcomes (in contrast to the single deterministic outcome per measurement ruled out by KS). We present a device-independent application of this generalised KS theorem by constructing a two-player non-local game for which a perfect quantum winning strategy exists (a Pseudo-telepathy game) while no perfect classical strategy exists even if the players are provided with additional no-signaling resources of PR-box type (with marginals in {0, 1/2, 1}).

A Two-Player Game for Multi-Scale Topology Optimization of Static and Dynamic Compliances of Triply Periodic Minimal Surface-Based Lattice Structures

A Two-Player Game for Multi-Scale Topology Optimization of Static and Dynamic Compliances of Triply Periodic Minimal Surface-Based Lattice Structures

Multi-adversarial autoencoders: Stable, faster and self-adaptive representation learning

The variational autoencoder (VAE) and generative adversarial networks (GAN) are two prominent approaches to achieving a probabilistic generative model by way of an autoencoder and a two-player minimax game. While VAEs often suffer from over-simplified posterior approximations, the adversarial autoencoder (AAE) has shown promise by adopting GAN to match the variational posterior to an arbitrary prior through adversarial training. Both VAEs and GANs face significant challenges such as training stability, mode collapse, and difficulty in extracting meaningful latent representations. In this paper, we propose the Multi-adversarial Autoencoder (MAAE), which extends the AAE framework by incorporating multiple discriminators and enabling soft-ensemble feedback. By adaptively regulating the collective feedback from multiple discriminators, MAAE captures a balance between fitting the data distribution and performing accurate inference and accelerates training stability while extracting meaningful and interpretable latent representations. Experimental evaluations on MNIST, CIFAR10, and CelebA datasets demonstrate significant improvements in latent representation, quality of generated samples, log-likelihood, and a pairwise comparison metric, with comparisons to recent methods.

Discovering Dadu: A Ludemic Enigma from South Asia

Abstract Dadu is a traditional board game exclusively played within the Muslim Dawoodi Bohra community in South Asia. Most households have their own hand-stitched cloth board, together with a set of wood-turned playing pieces and cowrie shells for dice. Though formally a two-player game, it is commonly played in large teams during family gatherings. Non-community members rarely take part and the game remains virtually unknown to outsiders. Descriptions of the game are absent from the scholarly literature and it does not find mention in any of the major game encyclopedias. Surviving boards and communal memory trace the game back to the early 20th century, but the hybrid nature of the game and the origin of the Dawoodi Bohra community in Yemen suggest that it may go back several centuries further. The present article uses ludemic theory to analyze the constituent elements of the game and demonstrate their affinity with elements in other games both inside and outside South Asia. The findings are contextualized within the wider history of the Dawoodi Bohra community, suggesting that Dadu may have resulted from the adaptation of a West Asian Tab game to a South Asian single track race game. A detailed set of rules gathered from interviews with members of the Dawoodi Bohra community is appended to the article.

Proof Complexity and Beyond

Proof complexity is a multi-disciplinary research area that addresses questions of the general form “how difficult is it to prove certain mathematical facts?” The current workshop focussed on recent advances in our understanding that the analysis of an appropriately tailored concept of “proof” underlies many of the arguments in algorithms, geometry or combinatorics research that make the core of modern theoretical computer science. These include the analysis of practical Boolean satisfiability (SAT) solving algorithms, the size of linear or semidefinite programming formulations of combinatorial optimization problems, the complexity of solving total NP search problems by local methods, and the complexity of describing winning strategies in two-player round-based games, to name just a few important examples.

Illuminating a Graph

We introduce a two-player game where the goal is to illuminate all edges of a graph. At each step the first player, called Illuminator, taps a vertex. The second player, called Adversary, reveals the edges incident with that vertex (consistent with the edges incident with the already tapped vertices). Illuminator tries to minimize the taps needed, and the value of the game is the number of taps needed with optimal play. We provide bounds on the value in trees and general graphs. In particular, we show that the value for the path on n vertices is 2 3 n + O ( 1 ) , and there is a constant ε > 0 such that for every caterpillar on n vertices, the value is at most ( 1 – ε ) n + 1 .

Correction to: A two-player game representation for a class of infinite horizon control problems under state constraints

Correction to: A two-player game representation for a class of infinite horizon control problems under state constraints

Trilateral game and vertical collaboration models in a two-stage green supply chain with substitutable green products

This paper explores the trilateral game and vertical collaboration model based on Stackelberg’s (manufacturer-leadership game) and Bertrand’s game-theoretical methodology for a two-stage green supply chain where the duopolistic manufacturers and the retailer are ecologically conscious. Both manufacturers produce and sell two substitutable green products through a common retailer. The selling prices and green levels (GLs) determine the demand for both green products. A game theoretical approach is implemented in the trilateral game model, and the result shows that the manufacturer with a bigger sales volume achieves superior performance in terms of earnings. In the vertical collaboration model, a manufacturer and retailer collaborate to optimize pricing issues, GLs, and profits. Two-player games among three participants are performed for this collaboration. The results of the vertical collaboration model show that the overall profit in vertical collaboration is greater than the sum of the individual profits corresponding to two participants in the trilateral game model. Whereas, the manufacturer outside the collaboration experiences a decline in profits. Further, a selection criterion of manufacturers is also developed to maximize the overall profit of the retailer. Finally, a numerical example and a sensitivity analysis are performed to demonstrate the model’s implementation and stability.

Identification and estimation of entry games under symmetry of unobservables

Abstract This paper provides a new point identification and estimation method for two-player entry games with complete information, based on symmetry of unobservables. Neither equilibrium selection nor parametric distributional assumptions are required. In addition, a weaker support condition is used relative to the existing literature. Unlike other semiparametric estimators, the estimator proposed here is $\sqrt{n}$-consistent. A test of the required symmetry condition is provided. Monte Carlo evidence shows that the estimator performs well with moderate sample sizes, and is robust to unimodal and multimodal error distributions. The estimator is applied to an entry game of discount retailers in Jia (2008). Our results suggest that researchers should apply the normality assumption with caution.

Some Algorithmic Results for Eternal Vertex Cover Problem in Graphs

The eternal vertex cover problem is a variant of the vertex cover problem. It is a two-player (attacker and defender) game in which, given a graph $G=(V,E)$, the defender needs to allocate guards at some vertices so that the allocated vertices form a vertex cover. The attacker can attack one edge at a time, and the defender needs to move the guards along the edges such that at least one guard moves through the attacked edge and the new configuration still remains a vertex cover. The attacker wins if no such move exists for the defender. The defender wins if there exists a strategy to defend the graph against infinite sequence of attacks. The minimum number of guards with which the defender can form a winning strategy is called the eternal vertex cover number of $G$, and is denoted by $evc(G)$. Given a graph $G$, the problem of finding the eternal vertex cover number is NP-hard for general graphs and remains NP-hard even for bipartite graphs. We have given a polynomial time algorithm to find the Eternal vertex cover number in chain graphs and $P_4$-sparse graphs. We have also given a linear-time algorithm to find the eternal vertex cover number for split graphs, an important subclass of chordal graphs.