Iterated Prisoner's Dilemma Research Articles

Human cooperation in social dilemmas: comparing the Snowdrift game with the Prisoner's Dilemma

Explaining the evolution of cooperation among non-relatives is one of the major challenges for evolutionary biology. In this study, we experimentally examined human cooperation in the iterated Snowdrift game (ISD), which has received little attention so far, and compared it with human cooperation in the iterated Prisoner's Dilemma (IPD), which has become the paradigm for the evolution of cooperation. We show that iteration in the ISD leads to consistently higher levels of cooperation than in the IPD. We further demonstrate that the most successful strategies known for the IPD (generous Tit-for-Tat and Pavlov) were also successfully used in the ISD. Interestingly, we found that female players cooperated significantly more often than male players in the IPD but not in the ISD. Moreover, female players in the IPD applied Tit-for-Tat-like or Pavlovian strategies significantly more often than male players, thereby achieving significantly higher pay-offs than male players did. These data demonstrate that the willingness to cooperate does not only depend on the type of the social dilemma, but also on the class of individuals involved. Altogether, our study shows that the ISD can potentially explain high levels of cooperation among non-relatives in humans. In addition, the ISD seems to reflect the social dilemma more realistically than the IPD because individuals obtain immediate direct benefits from the cooperative acts they perform and costs of cooperation are shared between cooperators.

Topology control with IPD network creation games

Network creation games couple a two-players game with the evolution of network structure. A vertex player may increase its own payoff with a change of strategy or with a modification of its edge-defined neighbourhood. By referring to the iterated prisoners dilemma (IPD) game we show that this evolutionary dynamics converges to network-Nash equilibria, where no vertex is able to improve its payoff. The resulting network structure exhibits a strong dependence on the parameter of the payoff matrix. Degree distributions and cluster coefficients are also strongly affected by the specific interactions chosen for the neighbourhood exploration. This allows network creation games to be seen as a promising artificial-social-systems approach for a distributive topology control of complex networked systems.

Coordinating team players within a noisy Iterated Prisoner’s Dilemma tournament

In this paper, we present our investigation into the use of a team of players within a noisy Iterated Prisoner’s Dilemma (IPD) tournament. We show that the members of such a team are able to use a pre-arranged sequence of moves that they make at the start of each interaction in order to recognise one another, and that by coordinating their actions they can increase the chances that one of the team members wins the round-robin style tournament. We consider, in detail, the factors that influence the performance of this team and we show that the problem that the team members face, when they attempt to recognise one another within the noisy IPD tournament, is exactly analogous to the problem, studied in information theory, of communicating reliably over a noisy channel. Thus we demonstrate that we can use error-correcting codes to implement this recognition, and by doing so, further optimise the performance of the team.

Cooperation in networked populations of selfish adaptive agents: Sensitivity to network structure

This paper investigates the adaptation of cooperating strategies in an iterated prisoner's dilemma (IPD) game with individually learning agents, subject to the structure of the interaction network. In particular, we study how cooperation or defection comes to dominate the population on Watts–Strogatz networks, under varying average path lengths. Our results are in good agreement with previous works on discrete choice dynamics on networks, but are in stark contrast with results from the evolution of cooperation literature. We argue that the latter is because the different adaptation method used (i.e., adaptive learning instead of ‘evolutionary’ strategy switching).

Evolutionary dynamics of the continuous iterated Prisoner's dilemma

The iterated prisoner's dilemma (IPD) has been widely used in the biological and social sciences to model dyadic cooperation. While most of this work has focused on the discrete prisoner's dilemma, in which actors choose between cooperation and defection, there has been some analysis of the continuous IPD, in which actors can choose any level of cooperation from zero to one. Here, we analyse a model of the continuous IPD with a limited strategy set, and show that a generous strategy achieves the maximum possible payoff against its own type. While this strategy is stable in a neighborhood of the equilibrium point, the equilibrium point itself is always vulnerable to invasion by uncooperative strategies, and hence subject to eventual destabilization. The presence of noise or errors has no effect on this result. Instead, generosity is favored because of its role in increasing contributions to the most efficient level, rather than in counteracting the corrosiveness of noise. Computer simulation using a single-locus infinite alleles Gaussian mutation model suggest that outcomes ranging from a stable cooperative polymorphism to complete collapse of cooperation are possible depending on the magnitude of the mutational variance. Also, making the cost of helping a convex function of the amount of help provided makes it more difficult for cooperative strategies to invade a non-cooperative equilibrium, and for the cooperative equilibrium to resist destabilization by non-cooperative strategies. Finally, we demonstrate that a much greater degree of assortment is required to destabilize a non-cooperative equilibrium in the continuous IPD than in the discrete IPD. The continuous model outlined here suggests that incremental amounts of cooperation lead to rapid decay of cooperation and thus even a large degree of assortment will not be sufficient to allow cooperation to increase when cooperators are rare. The extreme degree of assortment required to destabilize the non-cooperative equilibrium, as well as the instability of the cooperative equilibrium, may help explain why cooperation in Prisoner's Dilemmas is so rare in nature.

Kinds of kindness: classifying the causes of altruism and cooperation

Kinds of kindness: classifying the causes of altruism and cooperation

Simulating Two-Level Negotiations

International negotiations can be modeled as a two-level process that involves competing interests within and between groups. This modified simulation of the classic prisoner's dilemma introduces students to the negotiation process and challenges them to consider how different de- cision structures can affect outcomes. Students fill both leadership and negotiation roles and gain an active learning experience that exposes them to important international relations concepts. All of the instruc- tions, handouts, and materials are included to provide instructors of international relations or comparative politics with a resource that can be utilized with groups of varying size, ability, and composition. International negotiation is a complex process that is often discussed at an abstract level in introductory courses to international relations. Using simulations of this process can place the relevance of key concepts discussed in international relations theory, such as absolute versus relative gains, into perspective for the participants. Role playing also increases the participant's awareness of these complexities. The prisoner's dilemma has often been used as a tool to simulate international bar- gaining as the game has integral commonalities with the international system. Both situations lack a sovereign to enforce agreements. Therefore, the dominant strategy for the individual or state is often to defect. For example, the U.S. benefits from other nations signing the Kyoto Protocol even though America does not have to reduce emissions or spend more on protecting the environment. Defecting from the Kyoto treaty while other developed nations cooperate has the greatest benefits and the least costs for the United States. In the classic, single play or one-shot, prisoner's dilemma (PD) cooperation is highly unlikely as neither participant knows whether they can trust the other party in the game. In an iterated prisoner's dilemma (IPD) cooperation becomes more likely as groups know that interaction will occur in the future. This ''shadow of the future'' encourages groups or individuals to cooperate (Axelrod 1984). This sim- ulation is a finite IPD. It has been utilized in primary schools and high schools as well as in undergraduate courses at both Florida State University and the University of Connecticut. What makes this simulation unique is the intergroup nature of the dilemma as well as the inclusion of diplomats and the incorporation of various regime types. Students gain direct experience with small group decision making in a democracy or an authoritarian regime.

Human friendship favours cooperation in the Iterated Prisoner's Dilemma

In the last decades, many studies have attempted to analyse the factors that may favour the evolution of cooperation. Among unrelated individuals, cooperation is expected to occur when partners exchange altruistic acts one another (i.e., reciprocity) or when the donor of an altruistic act may obtain secondary benefits from the act (e.g., increased reputation). The iterated prisoner's dilemma (IPD) is frequently used to analyse cooperation between two players. In this game, cooperation is constantly at risk of exploitation. Therefore, previous knowledge of the other player's attitude towards cooperation may positively affect an individual's decision to cooperate during the game. In various social species (including humans), group members may form friendly relationships that reduce uncertainty about the partner response and increase the mutual exchange of benefits. In light of this, we analysed whether humans cooperate more when playing an IPD with a friend than with a stranger. Each subject played an IPD twice, with a friend and a stranger, and had £10 per round to donate or not to the other player. Our results evidenced that humans cooperate more with, and donate more money to the other player when playing with a friend than with a stranger. Moreover, in game two subjects playing with unfamiliar players followed a raise the stakes strategy (RTS), i.e. they donated more money as the game progressed. The sense of trust in the other player's willingness to reciprocate the altruistic act is probably an important pre-requisite favouring cooperation and it may explain the differences between friends and unfamiliar players. The use of a RTS strategy in game two, however, indicates that subjects were attempting to increase the level of cooperation with unfamiliar players. These findings are consistent with the observation that friendship (and the capacity to build cooperative relationships) may be beneficial to individuals in many social animals and in humans.

Particle Swarm Optimization Approaches to Coevolve Strategies for the Iterated Prisoner's Dilemma

This paper presents and investigates the application of coevolutionary training techniques based on particle swarm optimization (PSO) to evolve playing strategies for the nonzero sum problem of the iterated prisoner's dilemma (IPD). Three different coevolutionary PSO techniques are used, differing in the way that IPD strategies are presented: A neural network (NN) approach in which the NN is used to predict the next action, a binary PSO approach in which the particle represents a complete playing strategy, and finally, a novel approach that exploits the symmetrical structure of man-made strategies. The last technique uses a PSO algorithm as a function approximator to evolve a function that characterizes the dynamics of the IPD. These different PSO approaches are compared experimentally with one another, and with popular man-made strategies. The performance of these approaches is evaluated in both clean and noisy environments. Results indicate that NNs cooperate well, but may develop weak strategies that can cause catastrophic collapses. The binary PSO technique does not have the same deficiency, instead resulting in an overall state of equilibrium in which some strategies are allowed to exploit the population, but never dominate. The symmetry approach is not as successful as the binary PSO approach in maintaining cooperation in both noisy and noiseless environments-exhibiting selfish behavior against the benchmark strategies and depriving them of receiving almost any payoff. Overall, the PSO techniques are successful at generating a variety of strategies for use in the IPD, duplicating and improving on existing evolutionary IPD population observations.

Behavioral Diversity, Choices and Noise in the Iterated Prisoner's Dilemma

Real-world dilemmas rarely involve just two choices and perfect interactions without mistakes. In the iterated prisoner's dilemma (IPD) game, intermediate choices or mistakes (noise) have been introduced to extend its realism. This paper studies the IPD game with both noise and multiple levels of cooperation (intermediate choices) in a coevolutionary environment, where players can learn and adapt their strategies through an evolutionary algorithm. The impact of noise on the evolution of cooperation is first examined. It is shown that the coevolutionary models presented in this paper are robust against low noise (when mistakes occur with low probability). That is, low levels of noise have little impact on the evolution of cooperation. On the other hand, high noise (when mistakes occur with high probability) creates misunderstandings and discourages cooperation. However, the evolution of cooperation in the IPD with more choices in a coevolutionary learning setting also depends on behavioral diversity. This paper further investigates the issue of behavioral diversity in the coevolution of strategies for the IPD with more choices and noise. The evolution of cooperation is more difficult to achieve if a coevolutionary model with low behavioral diversity is used for IPD games with higher levels of noise. The coevolutionary model with high behavioral diversity in the population is more resistant to noise. It is shown that strategy representations can have a significant impact on the evolutionary outcomes because of different behavioral diversities that they generate. The results further show the importance of behavioral diversity in coevolutionary learning.

Evolution of Iterated Prisoner's Dilemma Game Strategies in Structured Demes Under Random Pairing in Game Playing

We discuss the evolution of strategies in a spatial iterated prisoner's dilemma (IPD) game in which each player is located in a cell of a two-dimensional grid-world. Following the concept of structured demes, two neighborhood structures are used. One is for the interaction among players through the IPD game. A player in each cell plays against its neighbors defined by this neighborhood structure. The other is for mating strategies by genetic operations. A new strategy for a player is generated by genetic operations from a pair of parent strings, which are selected from its neighboring cells defined by the second neighborhood structure. After examining the effect of the two neighborhood structures on the evolution of cooperative behavior with standard pairing in game-playing, we introduce a random pairing scheme in which each player plays against a different randomly chosen neighbor at every round (i.e., every iteration) of the game. Through computer simulations, we demonstrate that small neighborhood structures facilitate the evolution of cooperative behavior under random pairing in game-playing.

Friendship, cliquishness, and the emergence of cooperation

The evolution of cooperation is a central problem in biology and the social sciences. While theoretical work using the iterated prisoner's dilemma (IPD) has shown that cooperation among non-kin can be sustained among reciprocal strategies (i.e. tit-for-tat), these results are sensitive to errors in strategy execution, cyclical invasions by free riders, and the specific ecology of strategies. Moreover, the IPD assumes that a strategy's probability of playing the PD game with other individuals is independent of the decisions made by others. Here, we remove the assumption of independent pairing by studying a more plausible cooperative dilemma in which players can preferentially interact with a limited set of known partners and also deploy longer-term accounting strategies that can counteract the effects of random errors. We show that cooperative strategies readily emerge and persist in a range of noisy environments, with successful cooperative strategies (henceforth, cliquers) maintaining medium-term memories for partners and low thresholds for acceptable cooperation (i.e. forgiveness). The success of these strategies relies on their cliquishness—a propensity to defect with strangers if they already have an adequate number of partners. Notably, this combination of medium-term accounting, forgiveness, and cliquishness fits with empirical studies of friendship and other long-term relationships among humans.

Spatial strategies in a generalized spatial prisoner’s dilemma

Many strategies, such as tit-for-tat, have been proposed in the iterated prisoner’s dilemma (IPD) in which the prisoner’s dilemma (PD) is carried out repeatedly with two players. A spatial version of the iterated prisoner’s dilemma (SPD) has been studied, where a player at each site plays the IPD game with all the players in the neighborhood. However, the strategies studied in the SPD consider the past actions of a single opponent only. We studied spatial strategies that depend on the configuration of actions taken by all neighbors (as opposed to conventional temporal strategies). Since generosity can be considered as a spatial strategy, we first investigate the generosity required when an action error is involved. We also propose several spatial strategies that outperform many others.

Economic demography in fuzzy spatial dilemmas and power laws

Adaptive agents, playing the iterated Prisoner’s Dilemma (IPD) in a two-dimensional spatial setting and governed by Pavlovian strategies (“higher success-higher chance to stay”), are used to approach the problem of cooperation between self-interested individuals from a novel angle: We investigate the effect of different possible measures of success (MS) used by players to asses their performance in the game. These MS involve quantities such as: the player’s utilities U, his cumulative score (or “capital”) W, his neighborhood “welfare”, etc. To handle an imprecise concept like “success” the agents use fuzzy logic. The degree of cooperation, the “economic demography” and the “efficiency” attained by the system depend dramatically on the MS. Specifically, patterns of “segregation” or “exploitation” are observed for some MS. On the other hand, power laws, that may be interpreted as signatures of critical self-organization (SOC), constitute a common feature for all the MS.

Co-evolutionary learning with strategic coalition for multiagents

In a dynamic system, such as social and economic systems, complex interactions emerge among its members. In that case, their behaviors become adaptive according to changing environment. In this paper, we use the iterated prisoner's dilemma (IPD) game, which is simple yet capable of dealing with complex problems, to model the dynamic system, and propose strategic coalition to obtain superior adaptive agents and simulate its emergence in a co-evolutionary learning environment. Also, we introduce the concept of confidence for agents in a coalition and show how such confidence helps improving the generalization ability of the evolved agents using strategic coalition. Experimental results show that co-evolutionary learning with coalition and confidence can produce better performing agents that generalize well against unseen agents.

Cooperation in the iterated prisoner's dilemma is learned by operant conditioning mechanisms.

The prisoner's dilemma (PD) is the leading metaphor for the evolution of cooperative behavior in populations of selfish agents. Although cooperation in the iterated prisoner's dilemma (IPD) has been studied for over twenty years, most of this research has been focused on strategies that involve nonlearned behavior. Another approach is to suppose that players' selection of the preferred reply might be enforced in the same way as feeding animals track the best way to feed in changing nonstationary environments. Learning mechanisms such as operant conditioning enable animals to acquire relevant characteristics of their environment in order to get reinforcements and to avoid punishments. In this study, the role of operant conditioning in the learning of cooperation was evaluated in the PD. We found that operant mechanisms allow the learning of IPD play against other strategies. When random moves are allowed in the game, the operant learning model showed low sensitivity. On the basis of this evidence, it is suggested that operant learning might be involved in reciprocal altruism.

Errors of Judgment and Reporting in a Law Merchant system

In a system developed by Milgrom et al. (1990; Econ. Politics 2: 1) to model the Champagne fairs of Medieval Europe, private judges both adjudicate disputes and report on whether players have unpaid debts, but lack enforcement capability. An iterated prisoner's dilemma models a trading encounter where there is a short-term incentive to cheat but a longterm incentive to cooperate. Players have the option to learn about each other's past behavior at some cost. This system is meant to help the cooperators find each other as well as encourage cooperation. Under specified conditions, the system forms a sequential equilibrium strategy and the players should cooperate and pay its costs. This paper generalizes the model so as to allow for errors. Errors place further restraints on the conditions under which the system remains in equilibrium and the players should follow it. Different conditions are found under which the system is more sensitive to false positives or false negatives; a greater sensitivity to false positives corresponds to a stronger presumption of innocence. When the players pay the costs of the information, the system to provide it can serve as a correlating mechanism to pair cooperators in the prisoner's dilemma.

Stabilization of tag-mediated interaction by sexual reproduction in an evolutionary agent system

The evolution of cooperation in a system of agents playing the iterated prisoner's dilemma (IPD) is investigated. We present results for the standard 2-person IPD as well as the more general N-person IPD (NIPD) game. In our computational model, agents have visible tags and choose whether to interact or not based upon these. We consider the evolutionary stability of the evolving populations. We extend previous work by introducing sexual reproduction (recombination) of agents and by analyzing its influence on the evolving populations. We observed the occasional formation of very stable cooperative societies, as opposed to previous results without sexual reproduction. These cooperative societies are able to resist invasions of “mimics” (defecting agents with the tag of a cooperating agent).

Learning by pigeons playing against tit-for-tat in an operant prisoner's dilemma.

Each of four pigeons was exposed to a single random-ratio schedule of reinforcement in which the probability of reinforcement for a peck on either of two keys was 1/25. Reinforcer amounts were determined by an iterated prisoner's dilemma (IPD) matrix in which the "other player" (a computer) played tit-for-tat. One key served as the cooperation (C) key; the other served as the defection (D) key. If a peck was scheduled to be reinforced and the D-key was pecked, the immediate reinforcer of that peck was always higher than it would have been had the C-key been pecked. However, if the C-key was pecked and the following peck was scheduled to be reinforced, reinforcement amount for pecks on either key were higher than they would have been if the previous peck had been on the D-key. Although immediate reinforcement was always higher for D-pecks, the overall reinforcement rate increased linearly with the proportion of C-pecks. C-pecks thus constituted a form of self-control. All the pigeons initially defected with this procedure. However, when feedback signals were introduced that indicated which key had last been pecked, cooperation (relative rate of C-pecks)--hence, self-control--increased for all the pigeons.

Transitions between cooperative and non-cooperative responding in the ‘Pigeon's Dilemma’

Two pigeons played Iterated Prisoner's Dilemma (IPD) against a simulated opponent pre-programmed to play ‘Tit-For-Tat’ (TFT) and ‘Random’ (RND) strategies. Each pigeon received differential amounts of access to food following choices of either ‘cooperate’ or ‘defect’ on a trial. After 1000 trials against TFT and 500 trials against RND, results indicated that choice allocation was optimal when the birds played against RND but was sub-optimal when the birds played against TFT. In order to determine why the pigeons responded suboptimally against TFT, a trial-by-trial analysis of the data was conducted. The analysis revealed that once a pigeon had received the ‘Sucker's’ payoff ( S), it was more likely to defect and receive the ‘Temptation’ payoff ( T) than to cooperate and receive the ‘Reward’ ( R) payoff. Local reinforcement contingencies appear to determine suboptimal responding against TFT in the iterated Pigeon's Dilemma.