Iterated Prisoner's Dilemma Research Articles

Diffusion and pattern formation in spatial games.

Diffusion plays an important role in a wide variety of phenomena, from bacterial quorum sensing to the dynamics of traffic flow. While it generally tends to level out gradients and inhomogeneities, diffusion has nonetheless been shown to promote pattern formation in certain classes of systems. Formation of stable structures often serves as a key factor in promoting the emergence and persistence of cooperative behavior in otherwise competitive environments, however, an in-depth analysis on the impact of diffusion on such systems is lacking. We therefore investigate the effects of diffusion on cooperative behavior using a cellular automaton (CA) model of the noisy spatial iterated prisoner's dilemma (IPD), physical extension, and stochasticity being unavoidable characteristics of several natural phenomena. We further derive a mean-field (MF) model that captures the three-species predation dynamics from the CA model and highlight how pattern formation arises in this new model, then characterize how including diffusion by interchange similarly enables the emergence of large scale structures in the CA model as well. We investigate how these emerging patterns favors cooperative behavior for parameter space regions where IPD error rates classically forbid such dynamics. We thus demonstrate how the coupling of diffusion with nonlinear dynamics can, counterintuitively, promote large-scale structure formation and in return establish new grounds for cooperation to take hold in stochastic spatial systems.

Relatedness in zero-determinant strategies

Relatedness is necessary and causal in the development of social life. Interlayer relatedness is a measure of how one player’s decisions affect the decisions of other players in the game. The relatedness can be positive or negative. We had to determine how effective each strategy was under specific conditions, and how the correlation between players affected their payoffs. In this paper, we analytically study the strategies that enforce linear payoff relationships in the Iterated Prisoner’s Dilemma (IPD) game considering both a relatedness factor. As a result, we first reveal that the payoffs of two players and three players can be represented by the form of determinants as shown by Press and Dyson even with the factor.

The Latest Researches of Kharkov University on the Evolution of Population Behavior Scenarios

The results of research on the evolution of population behavior scenarios at the Department of Artificial Intelligence at Kharkov University are considered. The competition between different behavioral scenarios and population agents during the strategies evolution with memory is discussed. The strategies interact with each other in an iterative prisoner's dilemma, earning bonus points according to a payoff matrix. Research focuses on three collective characteristics, such as memory, level of aggressiveness (proportion of non-cooperation), and complexity of strategy (variety of behaviors). Different evolutionary scenarios arise when using different selection rules for strategies intended for removal in subsequent stages of evolution. PACS: 02.50.Le, 05.10.−a, 87.23.Kg, 89.75.Fb.

‘I don’t want to play with you anymore’: dynamic partner judgements in moody reinforcement learners playing the prisoner’s dilemma

Abstract Emerging reinforcement learning algorithms that utilize human traits as part of their conceptual architecture have been demonstrated to encourage cooperation in social dilemmas when compared to their unaltered origins. In particular, the addition of a mood mechanism facilitates more cooperative behaviour in multi-agent iterated prisoner dilemma (IPD) games, for both static and dynamic network contexts. Mood-altered agents also exhibit humanlike behavioural trends when environmental aspects of the dilemma are altered, such as the structure of the payoff matrix used. It is possible that other environmental effects from both human and agent-based research will interact with moody structures in previously unstudied ways. As the literature on these interactions is currently small, we seek to expand on previous research by introducing two more environmental dimensions; voluntary interaction in dynamic networks, and stability of interaction through varied network restructuring. From an initial Erdos–Renyi random network, we manipulate the structure of a network IPD according to existing methodology in human-based research, to investigate possible replication of their findings. We also facilitated strategic selection of opponents through the introduction of two partner evaluation mechanisms and tested two selection thresholds for each. We found that even minimally strategic play termination in dynamic networks is enough to enhance cooperation above a static level, though the thresholds for these strategic decisions are critical to desired outcomes. More forgiving thresholds lead to better maintenance of cooperation between kinder strategies than stricter ones, despite overall cooperation levels being relatively low. Additionally, moody reinforcement learning combined with certain play termination decision strategies can mimic trends in human cooperation affected by structural changes to the IPD played on dynamic networks—as can kind and simplistic strategies such as Tit-For-Tat. Implications of this in comparison with human data is discussed, and suggestions for diversification of further testing are made.

Research on the Iterated Prisoner Dilemma Based on Combinations of Two-side Strategies

The endless prisoner's dilemma game is described as a "primer" using the perspectives of the bilateral parties. Current research focuses on which player responds most effectively, which is a kind of self-protect behavior. Content analysis and mathematical justification were undertaken for the prisoners dilemma iteration to understand better how to improve the game in a partially competitive and cooperative collaborative setting. This aims to apply the two-side strategies to the real world, especially the political events in which the whole human outcome should be maximized. Participants, who were mostly chosen from game theory specialists from an experiment, provided the decision parameters and rules, in which they have to decide among certain personal strategy choices. The comparisons findings show that two-sided cooperation produces the best results. Still, two-sided inaction results in the poorest, and single cooperation and single noncooperation result from personal sacrifice and personal selfishness, which is a psychological phenomenon.

Understanding Partnership Formation and Repeated Contributions in Federated Learning: An Analytical Investigation

Limited access to large-scale data is a key obstacle to building machine learning (ML) applications in practice, partly due to a reluctance of information exchange among data owners out of privacy and data security concerns. To address this “information silo” problem, federated learning (FL) techniques have been proposed to enable decentralized model training via an orchestrating central server and have received increasing attention in several industries (including healthcare and finance). Despite its superior privacy protection property, adoption of FL is limited by a lack of systematic understanding of its underlying economics. In this paper, we take an analytical approach to answer two questions: (1) when do data owners prefer to form a FL partnership over building ML models by themselves and (2) how can different contractual mechanisms be used to promote repeated contributions to FL (the cooperative outcome that benefits all participants). We formulate an iterated prisoner’s dilemma (IPD) model that accounts for unique FL characteristics, including the specification of the payoff matrix and the involvement of a central server to sanction noncooperation. We find that partnership formation requires participants to be not too forward-looking in temporal preferences, which is contrary to the conventional wisdom in IPD. Furthermore, to promote repeated contributions, it is insufficient to only rely on penalties imposed by the central server or by participants for noncooperation, but a combination of both is enough. Our work advances theoretical understanding of the economics of FL and provides prescriptive insights that can inform FL participant selection and contract design. This paper was accepted by D. J. Wu, information systems. Funding: This work was partially supported by Cisco Research. Supplemental Material: The online appendices are available at https://doi.org/10.1287/mnsc.2023.00611 .

Reactive means in the iterated Prisoner’s dilemma

The Iterated Prisoner’s Dilemma (IPD) is a well studied framework for understanding direct reciprocity and cooperation in pairwise encounters. However, measuring the morality of various IPD strategies is still largely lacking. Here, we partially address this issue by proposing a suit of plausible morality metrics to quantify four aspects of justice. We focus our closed-form calculation on the class of reactive strategies because of their mathematical tractability and expressive power. We define reactive means as a tool for studying how actors in the IPD and Iterated Snowdrift Game (ISG) behave under typical circumstances. We compute reactive means for four functions intended to capture human intuitions about “goodness” and “fair play”. Two of these functions are strongly anticorrelated with success in the IPD and ISG, and the other two are weakly anticorrelated with success. Our results will aid in evaluating and comparing powerful IPD strategies based on machine learning algorithms, using simple and intuitive morality metrics.

Softening and Hardening in the Iterated Prisoner’s Dilemma

This article presents two methods to evaluate and draw distinction among general cooperative and aggressive strategic behavior in the iterated prisoner’s dilemma (IPD). The first method identifies classes of strategies, clusters them into subclasses based on their cooperative or aggressive disposition, and then comprehensively assess their behavior. The second method relies on applying transformative manipulations that soften or harden strategies of a given set and compare them with the results obtained with the initial set. Although examined classes of initial strategies are very different, the results are stable and convergent. While our findings agree with the classical analyses of the IPD, the results are qualified and precise. Furthermore, the reproducibility of prior results via new experimental methods offer experimental evidence. The proposed comparative strategy analysis method is a new instrument that can have broad implications for analyzing games beyond the prisoner’s dilemma.

Application of the Iterated Prisoner's Dilemma and Evolutionary Game Theory in Siting Nuclear Power Plants in South Korea

The quality of an urban living environment largely depends on the planning and development of public facilities, which are often halted or delayed due to the NIMBY (not in my backyard) phenomenon. In such facilities, environmental costs are borne solely by the residents of proximity to the facility, while public goods/services produced by the facility are reaped equally by residents across the greater region, which in turn presents a complex dynamic of public and self-interest. This paper uses repeated games and evolutionary game theory to identify the optimal negotiation strategy for the government when siting nuclear power plant facilities in South Korea. This study simulated a tournament containing 36 selected iterated prisoner's dilemma strategies and considered factors including mean payoff values, payoff matrices, and Axelrod's Ecological Variant to deduce an optimal strategy. The results showed that AON2, a memory-2 strategy of direct reciprocity, would provide the most stable and high return negotiations.

The Approach Repetition Rate Efficiency in Memorable Iterated Prisoner Dilemma Game

The Approach Repetition Rate Efficiency in Memorable Iterated Prisoner Dilemma Game

Probabilistic memory-one strategies to dominate the iterated prisoner’s dilemma over networks

The Iterated Prisoner’s Dilemma (IPD) has been a classical game theoretical scenario used to model behaviour interactions among agents. From the famous Axelrod’s tournament, and the successful results obtained by the Tit for Tat strategy, to the introduction of the zero-determinant strategies in the last decade, the game theory community has been exploring the performance of multiple strategies for years. This article grounds on such previous work, studying probabilistic memory-one strategies (PMO) and using evolutionary game theory, to analyse the criteria to find the most successful set of strategies in networked topologies. The results are nearly deterministic in discrete PMO scenarios. However, results become much more complex when moving to continuous ones, and there is no optimal strategy for a given scenario. Finally, this article describes how, using machine learning and evolutionary techniques; a cluster of agents, playing synchronously and adaptively, is able to dominate the rest of the population.

Misperception influence on zero-determinant strategies in iterated Prisoner\u2019s Dilemma

Zero-determinant (ZD) strategies have attracted wide attention in Iterated Prisoner’s Dilemma (IPD) games, since the player equipped with ZD strategies can unilaterally enforce the two players’ expected utilities subjected to a linear relation. On the other hand, uncertainties, which may be caused by misperception, occur in IPD inevitably in practical circumstances. To better understand the situation, we consider the influence of misperception on ZD strategies in IPD, where the two players, player X and player Y, have different cognitions, but player X detects the misperception and it is believed to make ZD strategies by player Y. We provide a necessary and sufficient condition for the ZD strategies in IPD with misperception, where there is also a linear relationship between players’ utilities in player X’s cognition. Then we explore bounds of players’ expected utility deviation from a linear relationship in player X’s cognition with also improving its own utility.

Situational optimization function analysis: An ideal performance analysis inspired on Lewin's equation.

This study presents the situational optimization function analysis (SOFA) and has three aims. First, to develop a Bayesian implementation of SOFA. Second, to compare this implementation with three other maximum likelihood-based models in their accuracy to estimate true scores. The third aim is to show how joint modeling can be used for validity research. A simulation study was used to examine the second aim, while an empirical example was used to illustrate the third aim. The simulation study used three data generating processes, with varying degrees of deviation from linear models and with different sample sizes. Results of the simulation study showed that the Bayesian implementation supersedes the other models. In the empirical example, data collected from 66 participants using an iterated prisoner dilemma and a scale measuring cooperation-competition attitudes were used. Results showed that joint modeling is the best fitting model, also increasing the correlation between the true scores of both measures (deviations from the iterated prisoner dilemma and the scale). Finally, implications, limitations and future studies are discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

At Lunch with Freeman Dyson

This is the story of a minor discovery in mathematical game theory. It concerns the prisoner’s dilemma game, which, played once, involves little strategy. But consider the iterated prisoner’s dilemma (IPD) game. In the IPD, there is information in the previous plays, which each player can use to devise a superior strategy that remains self-interested.

Exploring Monte Carlo Negotiation Search with Nontrivial Agreements

The application of automated negotiations to general game playing is a research area with far-reaching implications. Non-zero sum games can be used to model a wide variety of real-world scenarios and automated negotiation provides a framework for more realistically modeling the behavior of agents in these scenarios. A particular recent development in this space is the Monte Carlo Negotiation Search (MCNS) algorithm, which can negotiate to find valuable cooperative strategies for a wide array of games (such as those of the Game Description Language). However, MCNS only proposes agreements corresponding to individual sequences of moves without any higher-level notions of conditional or stateful strategy. Our work attempts to lift this restriction. We present two contributions: extensions to the MCNS algorithm to support more complex agreements and an agreement language for GDL games suitable for use with our algorithm. We also present the results of a preliminary experiment in which we use our algorithm to search for an optimal agreement for the iterated prisoners dilemma. We demonstrate significant improvement of our algorithm over random agreement sampling, although further work is required to more consistently produce optimal agreements.

МОДЕЛИРОВАНИЕ ПРОСТРАНСТВА СТРАТЕГИЙ ЗАДАЧИ «ДИЛЕММА ЗАКЛЮЧЕННОГО»

An iterated version of the game "Prisoner's Dilemma" is used as a model of cooperation largely due to the wide range of strategies that the subjects can use. The problem of the effec-tiveness of strategies for solving the Iterated Prisoner's Dilemma (IPD) is most often considered from the point of view of information models, where strategies do not take into account the relationship that arise when real people play. Some of these strategies are obvious, others depend upon social context. In our paper, we use one of the promising directions in the development of studying IPD strategies – the use of artificial neural networks. We use neural networks as a modeling tool and as a part of game environment. The main goal of our work is to build an information model that predicts the behavior of an individual person as well as group of people in the situation of solving of social dilemma. It takes into account social relationship, including those caused by experimental influence, gender differences, and individual differences in the strategy for solving cognitive tasks. The model demonstrates the transition of individual actions into socially determined behavior. Evaluation of the effect of socialization associated with the procedure of the game provides additional information about the effectiveness and characteristics of the experimental impact.The paper defines the minimum unit of analysis of the IPD player's strategy in a group, the identity with which can be considered as a variable. It discusses the influence of the experi-mentally formed group identity on the change of preferred strategies in social dilemmas. We use the possibilities of neural networks as means of categorizing the results of the prisoner's iterative dilemma in terms of the strategy applied by the player, as well as social factors. We define the patterns of changes in the IPD player's strategy before and after socialization are determined. The paper discusses the questions of real player's inclination to use IPD solution strategies in their pure form or to use the same strategy before and after experimental inter-ventions related to social identity formation. It is shown that experimentally induced socialization can be considered as a mechanism for increasing the degree of certainty in the choice of strategies when solving IPD task. It is found out that the models based on neural networks turn out to be more efficient after experi-mentally evoked social identity in a group of 6 people; and the models based on neural net-works are least effective in the case of predicting a subject's belonging to a gender group. When solving IPD problems by real people, it turns out to be possible to talk about generalized strategies that take into account not only the evolutionary properties of «pure» strategies, but also reflect various social factors.

The Goldilocks Principle of Cooperation: Understanding Federated Learning in Healthcare via Iterated Prisoner's Dilemma

The Goldilocks Principle of Cooperation: Understanding Federated Learning in Healthcare via Iterated Prisoner's Dilemma

Parallel Theatre: An actor framework in Java for high performance computing

Theatre is an actor-based system currently implemented in Java, which enables modelling, analysis and implementation of predictable time-dependent distributed systems like cyber-physical systems. Theatre, though, can also be used for untimed complex systems. The core feature of Theatre is its control-based character. A reflective control layer, which can be tailored to the needs of specific application domains, is responsible for scheduling and dispatching messages asynchronously exchanged among actors. A Theatre system is a federation of theatres (computing nodes), upon which application actors are partitioned. Actors can move from a theatre to another. In its distributed implementation, Theatre rests on a socket network initially established among the interacting theatres. Messages and actors exploit Java serialization/deserialization when moving from a theatre to another. This paper proposes a novel extension of Theatre, Parallel Theatre, which is developed for an exploitation of the computing potential of nowadays multi-core machines with shared memory. Parallel Theatre avoids sockets and Java serialization. A key factor of Parallel Theatre consists in being a completely lock-free computing framework. Locks are avoided not only at the application level, as it was already demonstrated by classical actor systems, but also in the runtime system, thus enabling high-performance computing of scalable systems. This paper describes Parallel Theatre and the particular control forms which were developed for untimed and timed parallel systems. Then two scalable models are presented. The first one is concerned with a parallel algorithm for matrix multiplication. The second example is related to a parallel simulation of a multi-agent model for the Iterated Prisoner's Dilemma (IPD), which permits to observe the emergence of cooperation in large populations of players. The experimental results confirm the achievement of good execution performance.

Cooperative responses in rats playing a 2 × 2 game: Effects of opponent strategy, payoff, and oxytocin

The present study tested cooperation in rats playing a 2 × 2 game (2 players, 2 responses) in an operant chamber, where players choose to cooperate or defect without knowledge of their partner's choice. We evaluated cooperative responses in rats (Subjects) playing different games [iterated Prisoner's Dilemma (IPD), Stag Hunt] with a Stooge partner utilizing different response strategies [Tit-for-tat (TFT), Win-stay, Lose-shift (WSLS), Random], and we determined the effects of oxytocin (OT). IPD trial outcomes and payoffs included mutual cooperation (reward, R, 3 sugar pellets each), mutual defection (punishment, P, 1 pellet each), or unilateral defection (temptation, T, 5 pellets) and cooperation (sucker, S, 0 pellets). Stag Hunt was similar, except that T = 2 pellets. We hypothesized that Subjects would make more cooperative responses when playing Stag Hunt vs IPD, when playing IPD with a Stooge using TFT vs WSLS or Random, and when treated with OT. At baseline, Subjects’ overall likelihood of cooperation was unaffected by the game (IPD vs SH) or by the Stooges’ response strategy (TFT, WSLS, Random). Cooperative responses earned Subjects more pellets, except when playing with a Stooge using a random strategy. Trial outcomes (R, T, S or P) also varied by game and strategy, although the mutual defection (P) was the most common. Systemic pretreatment with OT increased Subjects’ cooperative responses, resulting in fewer P and more R outcomes. In particular, IPD-Random Subjects were more cooperative, even at the expense of earning fewer pellets. These results demonstrate that OT increases cooperative behavior in rats playing 2 × 2 games.

Evolutionary Optimization of Cooperative Strategies for the Iterated Prisoner's Dilemma

The iterated prisoner's dilemma (IPD) has been studied in fields as diverse as economics, computer science, psychology, politics, and environmental studies. This is due, in part, to the intriguing property that its Nash equilibrium is not globally optimal. Typically treated as a single-objective problem, a player's goal is to maximize their own score. In some work, minimizing the opponent's score is an additional objective. In this article, we explore the role of explicitly optimizing for mutual cooperation in IPD player performance. We implement a genetic algorithm in which each member of the population evolves using one of four multiobjective fitness functions: selfish, communal, cooperative, and selfless, the last three of which use a cooperative metric as an objective. As a control, we also consider two single-objective fitness functions. We explore the role of representation in evolving cooperation by implementing four representations for evolving players. Finally, we evaluate the effect of noise on the evolution of cooperative behaviors. Testing our evolved players in tournaments in which a player's own score is the sole metric, we find that players evolved with mutual cooperation as an objective are very competitive. Thus, learning to play nicely with others is a successful strategy for maximizing personal reward.