Iterated Prisoner's Dilemma Game Research Articles

The Approach Repetition Rate Efficiency in Memorable Iterated Prisoner Dilemma Game

The Approach Repetition Rate Efficiency in Memorable Iterated Prisoner Dilemma Game

An Exploration of Cooperation During an Asymmetric Iterated Prisoner’s Dilemma Game

We investigated how the contingent delivery of a cultural consequence on target culturants in an asymmetric iterated prisoner’s dilemma game (IPDG) affected players’ choices to cooperate. The asymmetric IPDG creates an analogue to environmental conditions in which behaviors and cultural practices create and sustain income disparities among different members of the population more generally. The asymmetric IPDG allows researchers to explore how these inequalities affect cooperation between players experiencing unequal payoffs. Six undergraduate students divided into three dyads participated in an ABABCDCD reversal design. An asymmetric IPDG in conditions A and C ensured that one player received a greater number of points regardless of the second participant's choices, analogous to contingencies that produce income inequalities. In conditions B and D, a metacontingency was arranged such that delivery of a cultural consequence (CC; bonus points equally distributed among the dyad) was contingent on the oscillating production of target aggregate products (APs) across two consecutive cycles. When participants coordinated responding and contacted the target AP → CC relation, the point disparity was reduced. However, individual contingencies in direct competition for the favored player were arranged to reduce the probability of cooperative responding. Results showed that the CC selected certain oscillations between target APs across dyads resulting in a decrease in the point disparity between the players. This study illustrates how laboratory scientists can arrange experiments that capture some of the complexity of social behavior as it occurs with respect to cultural, political, and economic systems and the associated networks of contingencies.

VERBAL INTERACTION PROMOTES COOPERATION IN AN ITERATED PRISONER’S DILEMMA GAME: A MULTIPLE BASELINE METACONTINGENCY EXPERIMENT

Some metacontingency experiments were based on cooperation procedures such as the iterated prisoner’s dilemma game (IPDG), but dismissed earlier results on cooperation as pertaining only to operant (not cultural) selection and did not control verbal interactions among participants. The present study evaluated the effects of verbal interactions on participants’ choices in an IPDG. Three sets of four university students played in four networked computers (screened by panels) and were exposed to conditions with or without permission to use a virtual chat room in a multiple baseline design. Without verbal interaction, choices varied, but tended to be all-defect. Once verbal interaction was allowed, choices quickly shifted and stabilized in all-cooperate on almost all trials. An IPDG can be interpreted as programming a metacontingency in which the higher payoff for the group (a cultural consequence) selects participants’ choices of the cooperative alternative (a culturant). As the cooperation literature had similarly found, verbal interactions among participants even through virtual chat room promotes the selection by the higher payoff. Metacontingency and cooperation procedures such as the IPDG are indistinguishable and their results must be evaluated together.

Selective attention to historical comparison or social comparison in the evolutionary iterated prisoner’s dilemma game

This paper investigates an evolutionary iterated prisoner’s dilemma (IPD) model of multiple agents, in which agents interact in terms of the pair-wise IPD game while adapting their attitudes towards income stream risk. Specifically, agents will become more risk averse (or more risk seeking) if their game payoffs exceed (or fall below) their expectations. In particular, agents use their peers’ average payoffs as expectations (social comparison) when their payoffs are lower than their peers’ averages, but use their own historical payoffs as expectations (historical comparison) when their payoffs are higher than their peers’ averages. Such selective attention to social comparison or historical comparison manifests a desire for continuous improvement of agents. Simulations are conducted to investigate the evolution of cooperation under the selective attention mechanism. Results indicate that agents can sustain a highly cooperative equilibrium when they consider selective attention in adjusting their risk attitudes. This holds true for both the well-mixed and the network-based games, even in the presence of uncertain game payoffs. The reason is that, selective attention can significantly induce agents to adhere to conditional cooperation as well as to identify uncertainty in payoffs, which enhances the risk-averse behavior of agents in the IPD game. As a result, high levels of cooperation can be attained.

Solving the Crowdsourcing Dilemma Using the Zero-Determinant Strategies

Crowdsourcing is a promising technology to accomplish a complex task via eliciting services from a large group of contributors. Recent observations indicate that the success of crowdsourcing has been threatened by the malicious behaviors of the contributors. In this paper, we analyze the attack problem using an iterated prisoner's dilemma (IPD) game and propose a reward-penalty expected payoff algorithm based on zero-determinant (ZD) strategies to reward a worker's cooperation or penalize its defection in order to entice the final cooperation. Both theoretical analysis and simulation studies are performed, and the results indicate that the proposed algorithm has the following two attractive characteristics: 1) the requestor can incentivize the worker to become cooperative without any long-term extra cost; and 2) the proposed algorithm is fair so that the requestor cannot arbitrarily penalize an innocent worker to increase its payoff even though it can dominate the game. To the best of our knowledge, we are the first to adopt the ZD strategies to stimulate both players to cooperate in an IPD game. Moreover, our proposed algorithm is not restricted to solve only the problem of crowdsourcing dilemma - it can be employed to tackle any problem that can be formulated into an IPD game.

Asymmetrical expectations of future interaction and cooperation in the iterated prisoner's dilemma game

This paper addresses how players’ asymmetrical expectations of future interaction affect their cooperation in the two-player iterated prisoner's dilemma (IPD) game. A player’ expectation of future interaction reflects his/her willingness to continue the interaction with his/her opponent. With the application of a co-evolutionary learning model with a niching technique that is used to search optimal strategies for players, simulation experiments are conducted to present the cooperative or uncooperative outcomes between the players with distinct expectations. Results indicate that, if one player has higher expectation of future interaction than the other, the former may be exploited by the latter. Such exploitation is mainly due to the former player's higher tendency toward mutual cooperation, which triggers the latter player's unilateral defection. Considering that real-world games always involve uncertainty or external disturbance, the games with uncertain payoffs or noise are also performed. Results indicate that the exploitation also exists in the game with uncertain payoffs. However, it may be absent in the presence of high levels of noise. The reason is that the cooperative tendency of the high-expectation player declines with the increasing level of noise. As a result, the two players are engaged in only low levels of cooperation in the IPD game. Nevertheless, when asymmetrical noise is taken into account, it is the player disturbed with higher levels of noise that is exploited by the other. In other words, the player whose strategy is interfered with higher levels of noise is more afraid of endless mutual defection caused by noise, which makes he/she cooperate more than his/her opponent, even though he/she may have a lower expectation on their future interaction.

Older and younger adults’ interactions with friends and strangers in an iterated prisoner’s dilemma

ABSTRACTThis study examined age-related differences in behavioral reactions to interpersonal conflict within an iterated prisoner’s dilemma (PD). Participants completed an iterated PD game alone and with a partner, either a stranger or a friend who accompanied them to the session. The partner, however, was actually a program that occasionally behaved selfishly or always reciprocated. Afterwards, participants formed trait impressions of their partner’s morality and competence. Participants cooperated more with friends than strangers and more with reciprocating partners than selfish ones. Older adults cooperated more with selfish partners and offered more favorable impressions than did younger adults. Overall, perceived partner trait morality was positively associated with cooperative behavior. Relative to younger adults, older adults were more passive during conflict but grew less so as selfishness continued. This passivity co-occurred with more favorable partner impressions and better objective performance, suggesting a degree of calibration not shown by younger adults.

High mutual cooperation rates in rats learning reciprocal altruism: The role of payoff matrix.

Cooperation is one of the most studied paradigms for the understanding of social interactions. Reciprocal altruism -a special type of cooperation that is taught by means of the iterated prisoner dilemma game (iPD)- has been shown to emerge in different species with different success rates. When playing iPD against a reciprocal opponent, the larger theoretical long-term reward is delivered when both players cooperate mutually. In this work, we trained rats in iPD against an opponent playing a Tit for Tat strategy, using a payoff matrix with positive and negative reinforcements, that is food and timeout respectively. We showed for the first time, that experimental rats were able to learn reciprocal altruism with a high average cooperation rate, where the most probable state was mutual cooperation (85%). Although when subjects defected, the most probable behavior was to go back to mutual cooperation. When we modified the matrix by increasing temptation rewards (T) or by increasing cooperation rewards (R), the cooperation rate decreased. In conclusion, we observe that an iPD matrix with large positive reward improves less cooperation than one with small rewards, shown that satisfying the relationship among iPD reinforcement was not enough to achieve high mutual cooperation behavior. Therefore, using positive and negative reinforcements and an appropriate contrast between rewards, rats have cognitive capacity to learn reciprocal altruism. This finding allows to infer that the learning of reciprocal altruism has early appeared in evolution.

Flexible asynchronous simulation of iterated prisoner’s dilemma based on actor model

The wide range of applications of the Iterated prisoner’s dilemma (IPD) game made it a popular subject of study for the research community. As a consequence, numerous experiments have been conducted by researchers along the last decades. However, topics related with scaling simulation leveraging existing HPC infrastructure in the field of IPD did not always play a relevant role in such experimental work. The main contribution of this paper is a new simulation framework, based on asynchronous communication and its implementation oriented to distributed environments. Such framework is based on the modern Akka actor platform, that supports concurrent, distributed and resilient message-driven simulations; which are exemplified over the IPD game as a case study. We also present several interesting results regarding the introduction of asynchrony into the IPD simulation in order to obtain an efficient framework, so the whole simulation becomes scalable when using HPC facilities. The influence of asynchrony on the algorithm itself is also discussed, and the results show that it does not hamper the simulation.

The effects of heterogeneous interaction and risk attitude adaptation on the evolution of cooperation

This paper addresses the evolution of cooperation in a multi-agent system with agents interacting heterogeneously with each other based on the iterated prisoner’s dilemma (IPD) game. The heterogeneity of interaction is defined in two models. First, agents in a network are restricted to interacting with only their neighbors (local interaction). Second, agents are allowed to adopt different IPD strategies against different opponents (discriminative interaction). These two heterogeneous interaction scenarios are different to the classical evolutionary game, in which each agent interacts with every other agent in the population by adopting the same strategy against all opponents. Moreover, agents adapt their risk attitudes while engaging in interactions. Agents with payoffs above (or below) their aspirations will become more risk averse (or risk seeking) in subsequent interactions, wherein risk is defined as the standard deviation of one-move payoffs in the IPD game. In simulation experiments with agents using only own historical payoffs as aspirations (historical comparison), we find that the whole population can achieve a high level of cooperation via the risk attitude adaptation mechanism, in the cases of either local or discriminative interaction models. Meanwhile, when agents use the population’s average payoff as aspirations (social comparison) for adapting risk attitudes, the high level of cooperation can only be sustained in a portion of the population (i.e., partial cooperation). This finding also holds true in both of the heterogeneous scenarios. Considering that payoffs cannot be precisely estimated in a realistic IPD game, simulation experiments are also conducted with a Gaussian disturbance added to the game payoffs. The results reveal that partial cooperation in the population under social comparison is more robust to the variation in payoffs than the global cooperation under historical comparison.

Power Asymmetries and Punishment in a Prisoner’s Dilemma with Variable Cooperative Investment

In many two-player games, players that invest in punishment finish with lower payoffs than those who abstain from punishing. These results question the effectiveness of punishment at promoting cooperation, especially when retaliation is possible. It has been suggested that these findings may stem from the unrealistic assumption that all players are equal in terms of power. However, a previous empirical study which incorporated power asymmetries into an iterated prisoner's dilemma (IPD) game failed to show that power asymmetries stabilize cooperation when punishment is possible. Instead, players cooperated in response to their partner cooperating, and punishment did not yield any additional increase in tendency to cooperate. Nevertheless, this previous study only allowed an all-or-nothing–rather than a variable–cooperation investment. It is possible that power asymmetries increase the effectiveness of punishment from strong players only when players are able to vary their investment in cooperation. We tested this hypothesis using a modified IPD game which allowed players to vary their investment in cooperation in response to being punished. As in the previous study, punishment from strong players did not increase cooperation under any circumstances. Thus, in two-player games with symmetric strategy sets, punishment does not appear to increase cooperation.

Cooperation in the evolutionary iterated prisoner’s dilemma game with risk attitude adaptation

The Iterated Prisoner’s Dilemma (IPD) game has been commonly used to investigate the cooperation among competitors. However, most previous studies on the IPD focused solely on maximizing players’ average payoffs without considering their risk preferences. By introducing the concept of income stream risk into the IPD game, this paper presents a novel evolutionary IPD model with agents seeking to balance between average payoffs and risks with respect to their own risk attitudes. We build a new IPD model of multiple agents, in which agents interact with one another in the pair-wise IPD game while adapting their risk attitudes according to their received payoffs. Agents become more risk averse after their payoffs exceed their aspirations, or become more risk seeking after their payoffs fall short of their aspirations. The aspiration levels of agents are determined based on their historical self-payoff information or the payoff information of the agent population. Simulations are conducted to investigate the emergence of cooperation under these two comparison methods. Results indicate that agents can sustain a highly cooperative equilibrium when they consider only their own historical payoffs as aspirations (called historical comparison) in adjusting their risk attitudes. This holds true even for the IPD with a short game encounter, for which cooperation was previously demonstrated difficult. However, when agents evaluate their payoffs in comparison with the population average payoff (called social comparison), those agents with payoffs below the population average tend to be dissatisfied with the game outcomes. This dissatisfaction will induce more risk-seeking behavior of agents in the IPD game, which will constitute a strong deterrent to the emergence of mutual cooperation in the population.

Borderline personality disorder, but not euthymic bipolar disorder, is associated with a failure to sustain reciprocal cooperative behaviour: implications for spectrum models of mood disorders.

Borderline personality disorder (BPD) and bipolar disorder (BD) have overlapping clinical presentations and symptoms - sources of persistent clinical confusion. Game-theory can characterize how social function might be sub-optimal in the two disorders and move the field beyond the anecdotal description of clinical history. Here, we tested the hypothesis that BPD and BD can be distinguished on the basis of diminished reciprocal altruism in iterated Prisoner's Dilemma (PD) games. Twenty females with BPD, 20 females with euthymic BD and 20 healthy (non-clinical) females, matched for age and cognitive ability, were assessed for Axis-I and personality disorders, and completed psychometric measures of state affect, impulsivity and hostility. Participants completed two iterated PD games and a test of gaze-cueing. In the PD games, BPD participants failed to show statistically stable preferences to cooperate with social partners (playing tit-for-tat) and made significantly fewer cooperative responses compared to BD or controls (ANOVA main effect p = 0.03, post-hoc Tukey p < 0.05 for both comparisons). BPD participants were also less likely to sustain cooperation following experiences involving mutual cooperation than the other groups. Neither BPD nor BD participants demonstrated impairments in shifting visual attention on the basis of other peoples' gaze. These data indicate that BPD is (selectively) associated with difficulties in establishing, and then maintaining, reciprocal cooperation, involving altruism. These difficulties are not seen in euthymic BD. Our data support the differentiation of BPD from BD and offer fresh insights into the social difficulties experienced by individuals with diagnoses of BPD.

Risk consideration and cooperation in the iterated prisoner’s dilemma

This paper investigates the cooperative behavior in the two-player iterated prisoner's dilemma (IPD) game with the consideration of income stream risk. The standard deviation of one-move payoffs for players is defined for measuring the income stream risk, and thus the risk effect on the cooperation in the two-player IPD game is examined. A two-population coevolutionary learning model, embedded with a niching technique, is developed to search optimal strategies for two players to play the IPD game. As experimental results illustrate, risk-averse players perform better than risk-seeking players in cooperating with opponents. In particular, in the case with short game encounters, in which cooperation has been demonstrated to be difficult to achieve in previous work, a high level of cooperation can be obtained in the IPD if both players are risk-averse. The reason is that risk consideration induces players to negotiate for stable gains, which lead to steady mutual cooperation in the IPD. This cooperative pattern is found to be quite robust against low levels of noise. However, with increasingly higher levels of noise, only intermediate levels of cooperation can be achieved in games between two risk-averse players. Games with risk-seeking players get to even lower cooperation levels. By comparing the players' strategies coevolved with and without a high level of noise, the main reason for the reduction in the extent of cooperation can be explained as the lack of contrition and forgiveness of players in the high-noise interactions. Moreover, although increasing encounter length is helpful in improving cooperation in the noiseless and low-noise IPD, we find that it may enforce the absence of contrition and forgiveness, and thus make cooperation even more difficult in the high-noise games.

Influences of agents with a self-reputation awareness component in an evolutionary spatial IPD game.

Iterated prisoner’s dilemma (IPD) researchers have shown that strong positive reputations plus an efficient reputation evaluation system encourages both sides to pursue long-term collaboration and to avoid falling into mutual defection cycles. In agent-based environments with reliable reputation rating systems, agents interested in maximizing their private interests must show concern for other agents as well as their own self-reputations–an important capability that standard IPD game agents lack. Here we present a novel learning agent model possessing self-reputation awareness. Agents in our proposed model are capable of evaluating self-behaviors based on a mix of public and private interest considerations, and of testing various solutions aimed at meeting social standards. Simulation results indicate multiple outcomes from the addition of a small percentage of self-reputation awareness agents: faster cooperation, faster movement toward stability in an agent society, a higher level of public interest in the agent society, the resolution of common conflicts between public and private interests, and a lower potential for rational individual behavior to transform into irrational group behavior.

Cooperation in wireless networks: a game‐theoretic framework with reinforcement learning

A game-theoretic framework based on the iterated prisoner's dilemma (IPD) is proposed to model the repeated dynamic interactions of multiple source nodes when communicating with multiple destinations in an ad hoc wireless network. In such networks where nodes are autonomous, selfish and not familiar with other nodes' strategies, fully cooperative behaviours cannot be assumed. Therefore reinforcement learning is studied to relate the utility function of each source node to actions previously taken in order to learn a strategy that maximises their expected future reward. Particularly, a Q-learning algorithm is proposed to allow network nodes to adapt to and play the IPD game against opponents with a variety of known and unknown strategies. Simulation results illustrate that the proposed Q-learning algorithm allows network nodes to play optimally and achieve their maximum expected return values.

Evolution of repeated prisoner's dilemma play under logit dynamics

In an evolutionary set-up, we append an ecology of iterated prisoner's dilemma (IPD) game strategies, consisting of unconditional cooperators (AllC), unconditional defectors (AllD) and reactive players (TFT) with two repeated strategies that have received less attention in the evolutionary IPD game literature: the error-proof, “generous” tit-for-tat (GTFT) which, with a certain probability, re-establishes cooperation after a (possibly by mistake) defection of the opponent and the penitent, “stimulus–response” (WSLS) strategy that resets cooperation after the opponent punished for defection. An abundance of rock–paper–scissors like patterns is discovered in the 3×3 ecologies comprising Pavlovian and “generous” players. Interestingly, the evolutionary success of Pavlov seems to depend on the absence of unconditional cooperators in the ecologies investigated.

An examination of evolved behavior in two reinforcement learning systems

Using agent-based simulation experiments, we assess the relative performance of two Reinforcement Learning System (RLS) paradigms – the classical Learning Classifier System (LCS) and an enhancement, the Extended Classifier System (XCS) – in the context of playing the Iterated Prisoner's Dilemma (IPD) game. In prior research, the XCS outperforms the LCS in solving the Animats-and-Maze and Boolean Multiplexer test problems. Our work has overlaps with and is an extension of such efforts in that it allows assessment of each system's ability to (a) cope with delayed environmental feedback, (b) evolve irrational choice as the optimal behavior, and (c) cope with unpredictable input from the environment. We find that while the XCS is considerably superior to the LCS, in terms of four key performance metrics, in playing IPD games against a deterministic, reactive game-playing agent (Tit-for-Tat), the LCS does better against an unpredictable opponent (Rand) albeit with significant evolutionary effort.Further, upon examining each XCS enhancement in isolation, we see that specific LCS variants equipped with a single XCS feature, do better than the traditional LCS model and/or the XCS model in terms of particular metrics against both types of opponents but, again, usually with greater evolutionary effort. This suggests that if offline, rather than online, performance and specific performance goals are the focus, then one may construct relatively-simpler LCS variants rather than full-fledged XCS systems. Further assessments using LCS variants equipped with combinations of XCS features should help better comprehend the synergistic impacts of these features on performance in the IPD.

PSO Algorithm for IPD Game

Mechanisms promoting the evolution of cooperation in two-player, two-strategy evolutionary games have been discussed in great detail over the past decades. Understanding the effects of repeated interactions in multi-player with multi-choice is a formidable challenge. This paper presents and investigates the application of co-evolutionary training techniques based on particle swarm optimization (PSO) to evolve cooperation for the iterated prisoner’s dilemma (IPD) game with multiple choices in noisy environment. Several issues will be addressed, which include the evolution of cooperation and the evolutionary stability in the presence of multiple choices and noise. First is using PSO approach to evolve cooperation. The second is the impact of noise on the evolution of cooperation is examined.

An Iterated Four-Player Prisoner's Dilemma Game with an External Selecting Agent: A Metacontingency Experiment

In these series of experiments we used an iterated prisoners' dilemma game (IPDG) to examine the effect of metacontingencies on aggregate products of the interrelated behavior of four players. Results of the first experiment showed that cultural level consequences in the form of points delivered to all players) contingent on aggregate products XXXX or YYYY increased the frequency of those productions. In subsequent experiments we added a baseline condition where the players experienced only the individual behavioral contingencies embedded in the game. Then we imposed the metacontingency on the XXXX aggregate product or, alternatively, on YYYY. After several reversals, we discontinued the metacontingency and the players again experienced only the individual contingencies of the game. In one experiment we used a yoked control to assess the effect of market feedback independent from the metacontingency relation. Results indicate that the cultural consequence (market feedback) controlled production of aggregate products even when its magnitude was minimal, that the metacontingency relation was necessary, and that it maintained relations among the behavior of individuals which resulted in the worst individual outcomes for all players.