Spatial Prisoner's Dilemma Research Articles

Adjustment of link weights based on tolerance promotes cooperation in spatial prisoner's dilemma game

Understanding and explaining cooperative behavior in human society has become an open question. In this paper, we propose a dynamic adjustment of pair relationships in a spatial prisoner's dilemma game. Unlike previous studies that individuals dynamically adjust the intensity of interaction with their opponents at each step, this work consider tolerance, in which the intensity of interaction is adjusted when the time of successive defections by an individual exceeds a tolerance threshold T. We find that although the proposed mechanism can significantly improve cooperation compared to traditional versions, a higher tolerance for continuous defection behavior is not conducive to the evolution of cooperation. Furthermore, an environmental adaptor that dynamically adjusts the paired relationship with the opponent at all times is beneficial for the evolution of cooperation. And the higher the degree of adjustment in the paired relationship, the lower the probability of continuous exploitation by defector. We hope that our work can provide some insights into explaining the existence and maintenance of cooperation.

Reciprocal-reward–based environmental feedback promotes cooperation in spatial prisoner's dilemma game

Reciprocal rewards for cooperative behavior are seen as a way of solving social dilemmas. In fact, reward values should not be fixed, but rather linked to the level of social development. Thus, this work proposes a mechanism by which reciprocal reward values in the spatial prisoner's dilemma game depend on region environmental feedback. Therefore, the reciprocal reward values are non-linear and inversely correlated with the environmental state. Numerical simulations show that the non-linear and dynamic reciprocal reward can significantly improve cooperation compared to the traditional version, which can also reduce social costs to some extent. In addition, a higher level of sensitivity to changes in the environment is beneficial for cooperative evolution. Specifically, while setting reciprocal rewards based on greater local environmental feedback can increase the threshold of temptation b at which cooperators become extinct, its contribution to encouraging cooperation is limited. Furthermore, different interval ranges of b-values will have a corresponding range of local environment to encourage cooperation to achieve the optimum. Finally, the results show that the system has good robustness.

Emergence of an unpredictable evolution in a spatial prisoner’s dilemma via a player’s multiple perspectives

Spatial prisoner’s dilemma (SPD) has attracted researchers’ attention as a model of conflict for players. In SPD, players have two different strategies, namely, defectors and cooperators. A defector earns a high payoff from an opponent co-operator while getting nothing from an opponent defector. On the contrary, cooperators promote a win–win relationship between the two cooperators. These mechanisms influence population dynamics in SPD, and many SPD models have been developed. However, little is known about the emergence of an unstable or unpredictable evolution in population dynamics using an SPD model, which may be observed in living systems. In addressing this issue, two SPD models were proposed. In both models, players change the neighborhood definition in accordance with their strategies and sometimes select the rule for this change using probability or local information. Result showed that our models generated characteristic population patterns that may be linked to a self-organized criticality (SOC), a term referring to many systems of interconnected, nonlinear elements that evolve over time into a critical state. In fact, the second model could be spontaneously close to the critical point using local information.

Spontaneous symmetry breaking of cooperation between species.

In mutualistic associations, two species cooperate by exchanging goods or services with members of another species for their mutual benefit. At the same time, competition for reproduction primarily continues with members of their own species. In intra-species interactions, the prisoner's dilemma is the leading mathematical metaphor to study the evolution of cooperation. Here we consider inter-species interactions in the spatial prisoner's dilemma, where members of each species reside on one lattice layer. Cooperators provide benefits to neighbouring members of the other species at a cost to themselves. Hence, interactions occur across layers but competition remains within layers. We show that rich and complex dynamics unfold when varying the cost-to-benefit ratio of cooperation, r. Four distinct dynamical domains emerge that are separated by critical phase transitions, each characterized by diverging fluctuations in the frequency of cooperation: (i) for large r cooperation is too costly and defection dominates; (ii) for lower r cooperators survive at equal frequencies in both species; (iii) lowering r further results in an intriguing, spontaneous symmetry breaking of cooperation between species with increasing asymmetry for decreasing r; (iv) finally, for small r, bursts of mutual defection appear that increase in size with decreasing r and eventually drive the populations into absorbing states. Typically, one species is cooperating and the other defecting and hence establish perfect asymmetry. Intriguingly and despite the symmetrical model set-up, natural selection can nevertheless favour the spontaneous emergence of asymmetric evolutionary outcomes where, on average, one species exploits the other in a dynamical equilibrium.

Memory–based adaptive interaction willingness enhances cooperation in spatial prisoner's dilemma

As an important part of the evolution of cooperation, interaction diversity has attracted extensive attention. Traditional studies mainly focus exclusively on either time or space, and the comprehensive impact of these two dimensions on the evolution of cooperation deserves further exploration. In this paper, the interaction willingness is abstracted as the network weight, and an adaptive mechanism for interaction willingness based on the delay effect is proposed by combining the network weight and accumulated payoff. Specifically, individuals adjust their willingness to participate in the game with a delay, considering the change in accumulated payoffs within a certain period, rather than the change in immediate payoffs. The degree of delay is reflected in the length of individuals' memory of historical payoffs. Simulation results deviate from the intuition that more information facilitates cooperation, and in certain modes of interaction willingness adjustment, the correlation between cooperation level and memory–length is non–monotonic. Long–memory is conducive to promoting cooperation under the condition of a relatively large range adjustment of interaction willingness. Conversely, with a narrow adjustment range of interaction willingness, cooperation can be effectively established with short–memory. These results hold substantial implications for both theory and practice, as slight adjustment often corresponds to low loss and high operability. In addition, the underlying mechanism is furthermore explored, revealing that the ‘isolation zones’ composed of individuals with low interaction willingness play an important role in promoting cooperation.

The universal probabilistic reward based on the difference of payoff realizes the evolution of cooperation

Many previous studies regarding the theoretical models of social dilemmas have described that rewarding opponents can help players to cooperate with each other. Those studies deal with reward not for defectors, but for cooperators. However, many prior researches concerning punishment that is also considered to be necessary for the evolution of cooperation discuss punishment on cooperators as well as defectors. Considering the group level, defectors who have too much payoff due to many cooperators around them will lose their superiority over cooperators by rewarding not only cooperators but also defectors surrounding them. As a result, cooperation among players can be realized.Therefore, this study proposes the new universal probabilistic reward based on the difference of payoff, where the greater the payoff of each player is, the higher the probability of giving reward to their opponents becomes regardless of the strategy of their opponents. This paper shows that by introducing the reward of this study into the framework of the spatial prisoner's dilemma game utilized in previous studies, the high-level evolution of cooperation can be achieved especially in both the relatively sparse regular and scale-free topology of connections. In addition, the programming method utilized to construct the simulator for this study will serve as teaching materials for specialized education at Kanagawa Institute of Technology.

Restructuring of neighborhood definition based on strategies will enhance the cooperation in a spatial prisoner's dilemma

Defectors often survive in spatial game theory, prompting researchers to devise models for cooperators' survival. However, most models overlook players' perspectives based on their strategies. This study introduces an agent-based model considering players' varied neighborhood conditions. In the first model, losing a competition alters the neighborhood definition, randomly adjusting the number of neighborhoods. In the second model, the number of neighborhoods changes based on the player's strategy after losing a competition. Compared to the other proposed model and a traditional spatial prisoner's dilemma model, the second model facilitates easier maintenance of cooperator density.

The cooperative evolution in the spatial prisoner's dilemma game with the local loyalty of two-strategy

The loyalty of individual strategies is an important factor affecting cooperative behavior in human society. Previous researches on the loyalty have mostly focused on the loyalty of individual to a single strategy, that is, the loyalty of individual to cooperative strategy. But from the perspective of defective strategy, individual holding defective strategy can also be seen as so-called loyalty to defective strategy.This paper presents a spatial prisoner's dilemma game model with the local loyalty of two-strategy, in which the loyalty to cooperative strategy and loyalty to defective strategy are considered simultaneously. To distinguish the loyalties of these two types of strategies, the loyalty value corresponding to defective strategy is represented as a negative number. A fitness of an individual is defined as weighted average of individual's payoff and power of average loyalty of local nearest neighbors. In strategy learning, if the loyalty of an individual and the loyalty of the target neighbor are different signs, a discount coefficient is added to the strategy transition probability function to reduce the probability.Numerical simulations show that considering individual loyalty to the two strategies can promote cooperative behavior in the system when the strength index of local loyalty in fitness function is positive, and there are optimal values for the weighting coefficient and the strength index of local loyalty, which can enable the system to reach full cooperators state. The discount coefficient to the fitness of neighbor can impact on the frequency of cooperators surviving and the proportion of surviving cooperators during phase transition.

The influence of environment-based reputation on cooperation in spatial prisoner's dilemma game

Research has shown that reputation mechanism can be effective in facilitating the generation and maintenance of cooperation in social dilemmas. In the traditional reputation mechanism, the amount of change in reputation is fixed. However, in real life, people's impression scores of a player are often influenced by the environment in which he lives. In other words, we prefer to celebrate the cooperative heroes who are surrounded by defectors and spurn the defectors who are hidden among the many cooperators. Thus, this paper proposes an environment-based reputation model, in which the increment of a player's reputation will be influenced by the local and global environment. The results of the research indicate that the greater the intensity of the environmental influence, the more it promotes the evolution of cooperation. It is worth noting that the local and global environmental focus shows peculiarities. Focusing on the global environment can improve the fraction of cooperators at low defection temptations while focusing on the local environment will facilitate cooperators’ survival at higher defection temptations.

Information accuracy of migration and imitation influences the evolution of cooperation in spatial prisoner's dilemma

Migration is ubiquitous in both biological worlds and human societies for seeking more suitable environments or greater benefits. Intensive studies have revealed that it may provide an effective mechanism to promote cooperation in the context of spatial prisoner' dilemma. As an attempt to give a more generic analysis on how migration influences the evolution of cooperation, we in this work propose a dynamic model that takes into account the agents' different tendencies on migration and imitation. More importantly, we introduce two information accuracy parameters that respectively affect the agents' decisions on the migration destination and on their imitation in the current neighborhood. The simulation results show that the higher accuracy of the information on imitation and migration contributes to a higher cooperation level. In particular, the imitation-oriented mode is more conducive to cooperation when the imitation information accuracy is high. Likewise, the migration-oriented mode is more conducive to cooperation with higher accuracy of the migration information. When imitation and migration information accuracies are both high, cooperation is effectively maintained in two different ways: the imitation-dominant pattern and the migration-dominant pattern, while the scenario in which the imitation tendency is close to the migration tendency has an inhibitory effect on cooperation. Finally, we analyse the robustness of the results under different population densities.

Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game

Many recent studies have used reinforcement learning methods to investigate the behavior of agents in evolutionary games. Q-learning, in particular, has become a mainstream method during this development. Here we introduce Q-learning agents into the evolutionary prisoner's dilemma game on a square lattice. Specifically, we associate the state space of Q-learning agents with the strategies of their neighbors, and we introduce a neighboring reward information sharing mechanism. We thus provide Q-learning agents with the payoff information of their neighbors, in addition to their strategies, which has not been done in previous studies. Through simulations, we show that considering neighborhood payoff information can significantly promote cooperation in the population. Moreover, we show that for an appropriate strength of neighborhood payoff information sharing, a chessboard pattern emerges on the lattice. We analyze in detail the reasons for the emergence of the chessboard pattern and the increase in cooperation frequency, and we also provide a theoretical analysis based on the pair approximation method. We hope that our research will inspire effective approaches for resolving social dilemmas by means of sharing more information among reinforcement learning agents during evolutionary games.

Asymmetry of individual activity promotes cooperation in the spatial prisoner's dilemma game.

We consider an aspiration-based asymmetric individual activity co-evolutionary prisoner's dilemma game model on the square lattice. In detail, each player has an individual weight that evolves with its strategy. We introduce an asymmetric setting that only μ fractions of players in the network chosen to be active can update their individual weights according to whether their payoffs satisfy their aspirations. Therefore, our individual weights can be regarded as a type of intrinsic motivation satisfaction. Through Monte Carlo simulations on the square lattice, our co-evolution mechanism has been proven to significantly promote cooperation. Furthermore, we find that the moderate μ leads to the highest cooperation level. By dividing the players into four types according to their strategies and weights, we prove that the inversion of the dominant relationship between cooperators and defectors with low weights leads to a conversion cycle of the four types of players, which enhances spatial reciprocity to promote cooperation. Moderate μ results in an appropriate number of low-weight players in this cycle. Interestingly, our results show that higher weight heterogeneity does not lead to higher levels of cooperation, which is contrary to the intuition formed in the previous work.

Antisocial behavior-based environmental feedback in spatial prisoner's dilemma game

The environment, people, and behavior are interdependent and always interactive influence. In reality, a higher cooperation level may change the environment in a way that is beneficial for all competitors. Herein, this work proposed a coevolutionary model in the prisoner's dilemma game where the community environment determines the anti-social revenue structure. Moreover, to capture this phenomenon that behavioral changes induced by the individual's perception of the external environment, the memory effect is also taken into account in this work. It suggests that individuals' stored information about community cooperative level (referred to as indirect experience) can trigger antisocial behavior by influencing their perceptions of antisocial revenue, and that their own past strategic information (referred to as direct experience) also plays a similar role. Numerous simulations show that the proposed mechanism significantly improves cooperation, especially when sensitivity to the environment is taken into account. Furthermore, when memory is considered, a lack of experience and insufficient environmental information may result in different optimal cooperative levels corresponding to different b. In addition, the impact of the sufficient indirect experience and the immediate environmental feedback on cooperation are nearly identical. When individuals have sufficient direct and indirect experiences, their roles can be interchangeably in decision-making. Finally, we discovered that the proposed mechanism exhibits good robustness. Overall, this study demonstrates an innovative use of quantitative calculations to describe the conclusions of qualitative analysis, which have theoretical and practical implications.

Investigating the effectiveness of interaction-efficiency-driven strategy updating under progressive-interaction for the evolution of the prisoner's dilemma game

In reality, interactive friend is added may one after another, instead of interacting with all at first. This work has proposed a progressive-interaction behavior driven by consistent aspiration, in which individuals interact with replacement neighbor one after another. Furthermore, payoff and interaction-efficiency are two factors considered in a modified Fermi rule, which particularly emphasizes that individuals achieving identical aspirations with less interaction frequency per unit of time will have higher interaction-efficiency and are more likely to be imitated. Thus, this work introduces the interaction-efficiency-driven strategy updating rules under progressive-interaction behavior in a spatial prisoner's dilemma game. As shown by numerical simulations, the progressive-interaction significantly enhances cooperation more than traditional versions, which may be caused by a heterogeneous number of interacting neighbors. When the interaction-efficiency-driven strategy update is taken into account, the cooperation is elevated to a higher level. Additionally, a specific point is employed to simulate the relationship between the aspiration and the b. As indicated by results, different b values have appropriate aspiration levels to promote the system to achieve the optimal cooperative level. More specifically, an obvious phase transition phenomenon can be found for the promotion results of individual aspiration level on cooperation. Besides, the phase transition phenomenon is existent when we simulate the relationship between the aspiration and the parameter delta for a given b value. In short, this work puts forward a new perspective on understanding cooperative evolution.

Promoting effect of adaptive interaction based on random neighbors to cooperation in the spatial prisoner's dilemma game

Interaction is an important step in cooperation evolution, and non-deterministic interactions are common in complex and changeable environments. Therefore, the number of interactive neighbors reflects the individual's response to the environment. Inspired by this, this study introduces an adaptively adjusted number of interactions based on random neighbors into a spatial prisoner's dilemma game, in which individuals adaptively adjust the number of interactive neighbors based on the satisfaction of returns. It is noteworthy that the interactive neighbors are randomly selected corresponding to the practical phenomenon that our friends will change over time, thereby resulting in un-fixed friends. As shown by numerous simulations, adaptively adjusting the number of interactive neighbors based on random neighbors can significantly enhance cooperation, and it performs better in strengthening cooperation than traditional deterministic interactions. Apart from that, cooperation can be promoted by an appropriate interactive neighborhood. Overall, this study is expected to help inspire more research on addressing social dilemmas.

The promoting effect of adaptive persistence aspiration on the cooperation based on the consideration of payoff and environment in prisoner's dilemma game

This work explores whether holding the last aspiration for a period of time can promote cooperation. Specifically, an evolutionary spatial prisoner's dilemma game mode is proposed, in which the players adjust strategies and aspirations by considering the payoff and environment. Therefore, the core is to allow players to hold the current aspiration for a period of time. Through numerical calculation, this study finds that the existence of an appropriate duration of aspiration can promote cooperation when b is less than a certain value. Moreover, the cooperation is gradually enhanced with the increase of T-max (maximum aspiration duration) when b is greater than it, but the enhancing effect is limited. It is also found that an appropriate value α (sensitivity to environmental change) can promote cooperation at different b intervals. Besides, this system indicates good robustness. Overall, this study provides a new perspective on exploring cooperative evolution based on aspiration.

Adaptive exit facilitates the evolution of cooperation in the spatial prisoner's dilemma game with punishment

The enforcement of costly punishment can facilitate cooperation. However, it is unclear how punishment would influence cooperation if individuals are free to exit an interaction. In this work, individuals are allowed to exit with a certain probability, which is adaptively adjusted based on the difference between expectations and payoffs. According to simulation results, adaptive exit can significantly improve cooperation even when punishment is weak; moreover, there is an optimal level of aspiration that causes adaptive exit to best promote the evolution of cooperation. Because of reciprocity, cooperative individuals maintain the lowest level of exit probability, allowing the relationship between them to be maintained, and the interaction between cooperative individuals and defectors is affected by adaptive exit processing, which is related to aspiration levels. Results show that cooperation prevails under moderate aspiration levels, because the exit mechanism protects cooperative individuals from exploitation by defectors; however, the exit mechanism does not prevent the invasion of cooperators by defectors when the aspiration level is low. The negative impact of sanctions causes defectors to increase their exit probability, and the exit of the defectors prevents nearby punishers from exploitation as well as reduces the implementation of punishment, thereby easing the punishment burden on the punisher. Therefore, the adaptive exit mechanism helps to eliminate second-order free-riding at a lower aspiration level. The study identifies the significant effects of exit on punishment, and highlights the important role of their association in the evolution of cooperation.

Incorporating social payoff into reinforcement learning promotes cooperation.

Reinforcement learning has been demonstrated to be an effective approach to investigate the dynamic of strategy updating and the learning process of agents in game theory. Most studies have shown that Q-learning failed to resolve the problem of cooperation in well-mixed populations or homogeneous networks. To this aim, we investigate the self-regarding Q-learning's effect on cooperation in spatial prisoner's dilemma games by incorporating the social payoff. Here, we redefine the reward term of self-regarding Q-learning by involving the social payoff; that is, the reward is defined as a monotonic function of the individual payoff and the social payoff represented by its neighbors' payoff. Numerical simulations reveal that such a framework can facilitate cooperation remarkably because the social payoff ensures agents learn to cooperate toward socially optimal outcomes. Moreover, we find that self-regarding Q-learning is an innovative rule that ensures cooperators coexist with defectors even at high temptations to defection. The investigation of the emergence and stability of the sublattice-ordered structure shows that such a mechanism tends to generate a checkerboard pattern to increase agents' payoff. Finally, the effects of Q-learning parameters are also analyzed, and the robustness of this mechanism is verified on different networks.

Impact of multi-step punishment on the spatial prisoner's dilemma game

Punishment has been proved to be an effective means for sustaining and promoting the evolution of cooperation. However, most works only consider one-off punishment, that is, the effect of each punishment only acts at one time step, it is often inconsistent with real-world scenarios. Therefore, different from previous works, we propose a new punishment model, which extends the punishment from one step to multiple ones. In particular, we assume that individuals will change their reputation scores according to their own behaviors, and begin to accept the punishment with intensity w when their reputation is below the reputation threshold. Through plenty of Monte Carlo simulations, it is found that there exists an optimal punishment time window to foster the cooperative behaviors, which indicates that the multi-step punishment is a double-edged sword, and too long or too short punishment is not conductive to sustain the collective cooperation.

Does a resource-storing mechanism favor “the wealthy do not fight”?—An approach from evolutionary game theory

Motivated by an interdisciplinary question of whether a resource-storing mechanism favors “the wealthy do fight” or “the wealthy do not fight,” we establish a new model based on spatial prisoner's dilemma (SPD) game where a time-accumulating payoff is allowed, and the probability of game participation depending on wealthiness is introduced. Although the model is based on a universal framework, presuming SPDs as a template, the conclusion drawn from the model is interdisciplinary rich. Our simulation results reveal that the common sense (or say, human's philosophical wisdom) of “the wealthy do not fight” helps to emerge a fairly cooperative society depending on the dilemma strength. By contrast, remarkably, it suggests that the common sense of “the wealthy do fight” begets a skewed exploitation society where defectors who exploit poor cooperators prosper.PACS numbersTheory and modeling; computer simulation, 87.15.Aa; Dynamics of evolution, 87.23.Kg.