Spatial Prisoner's Dilemma Game Research Articles

The universal probabilistic reward based on the difference of payoff realizes the evolution of cooperation

Many previous studies regarding the theoretical models of social dilemmas have described that rewarding opponents can help players to cooperate with each other. Those studies deal with reward not for defectors, but for cooperators. However, many prior researches concerning punishment that is also considered to be necessary for the evolution of cooperation discuss punishment on cooperators as well as defectors. Considering the group level, defectors who have too much payoff due to many cooperators around them will lose their superiority over cooperators by rewarding not only cooperators but also defectors surrounding them. As a result, cooperation among players can be realized.Therefore, this study proposes the new universal probabilistic reward based on the difference of payoff, where the greater the payoff of each player is, the higher the probability of giving reward to their opponents becomes regardless of the strategy of their opponents. This paper shows that by introducing the reward of this study into the framework of the spatial prisoner's dilemma game utilized in previous studies, the high-level evolution of cooperation can be achieved especially in both the relatively sparse regular and scale-free topology of connections. In addition, the programming method utilized to construct the simulator for this study will serve as teaching materials for specialized education at Kanagawa Institute of Technology.

The cooperative evolution in the spatial prisoner's dilemma game with the local loyalty of two-strategy

The loyalty of individual strategies is an important factor affecting cooperative behavior in human society. Previous researches on the loyalty have mostly focused on the loyalty of individual to a single strategy, that is, the loyalty of individual to cooperative strategy. But from the perspective of defective strategy, individual holding defective strategy can also be seen as so-called loyalty to defective strategy.This paper presents a spatial prisoner's dilemma game model with the local loyalty of two-strategy, in which the loyalty to cooperative strategy and loyalty to defective strategy are considered simultaneously. To distinguish the loyalties of these two types of strategies, the loyalty value corresponding to defective strategy is represented as a negative number. A fitness of an individual is defined as weighted average of individual's payoff and power of average loyalty of local nearest neighbors. In strategy learning, if the loyalty of an individual and the loyalty of the target neighbor are different signs, a discount coefficient is added to the strategy transition probability function to reduce the probability.Numerical simulations show that considering individual loyalty to the two strategies can promote cooperative behavior in the system when the strength index of local loyalty in fitness function is positive, and there are optimal values for the weighting coefficient and the strength index of local loyalty, which can enable the system to reach full cooperators state. The discount coefficient to the fitness of neighbor can impact on the frequency of cooperators surviving and the proportion of surviving cooperators during phase transition.

The influence of environment-based reputation on cooperation in spatial prisoner's dilemma game

Research has shown that reputation mechanism can be effective in facilitating the generation and maintenance of cooperation in social dilemmas. In the traditional reputation mechanism, the amount of change in reputation is fixed. However, in real life, people's impression scores of a player are often influenced by the environment in which he lives. In other words, we prefer to celebrate the cooperative heroes who are surrounded by defectors and spurn the defectors who are hidden among the many cooperators. Thus, this paper proposes an environment-based reputation model, in which the increment of a player's reputation will be influenced by the local and global environment. The results of the research indicate that the greater the intensity of the environmental influence, the more it promotes the evolution of cooperation. It is worth noting that the local and global environmental focus shows peculiarities. Focusing on the global environment can improve the fraction of cooperators at low defection temptations while focusing on the local environment will facilitate cooperators’ survival at higher defection temptations.

Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game

Many recent studies have used reinforcement learning methods to investigate the behavior of agents in evolutionary games. Q-learning, in particular, has become a mainstream method during this development. Here we introduce Q-learning agents into the evolutionary prisoner's dilemma game on a square lattice. Specifically, we associate the state space of Q-learning agents with the strategies of their neighbors, and we introduce a neighboring reward information sharing mechanism. We thus provide Q-learning agents with the payoff information of their neighbors, in addition to their strategies, which has not been done in previous studies. Through simulations, we show that considering neighborhood payoff information can significantly promote cooperation in the population. Moreover, we show that for an appropriate strength of neighborhood payoff information sharing, a chessboard pattern emerges on the lattice. We analyze in detail the reasons for the emergence of the chessboard pattern and the increase in cooperation frequency, and we also provide a theoretical analysis based on the pair approximation method. We hope that our research will inspire effective approaches for resolving social dilemmas by means of sharing more information among reinforcement learning agents during evolutionary games.

Asymmetry of individual activity promotes cooperation in the spatial prisoner's dilemma game.

We consider an aspiration-based asymmetric individual activity co-evolutionary prisoner's dilemma game model on the square lattice. In detail, each player has an individual weight that evolves with its strategy. We introduce an asymmetric setting that only μ fractions of players in the network chosen to be active can update their individual weights according to whether their payoffs satisfy their aspirations. Therefore, our individual weights can be regarded as a type of intrinsic motivation satisfaction. Through Monte Carlo simulations on the square lattice, our co-evolution mechanism has been proven to significantly promote cooperation. Furthermore, we find that the moderate μ leads to the highest cooperation level. By dividing the players into four types according to their strategies and weights, we prove that the inversion of the dominant relationship between cooperators and defectors with low weights leads to a conversion cycle of the four types of players, which enhances spatial reciprocity to promote cooperation. Moderate μ results in an appropriate number of low-weight players in this cycle. Interestingly, our results show that higher weight heterogeneity does not lead to higher levels of cooperation, which is contrary to the intuition formed in the previous work.

Antisocial behavior-based environmental feedback in spatial prisoner's dilemma game

The environment, people, and behavior are interdependent and always interactive influence. In reality, a higher cooperation level may change the environment in a way that is beneficial for all competitors. Herein, this work proposed a coevolutionary model in the prisoner's dilemma game where the community environment determines the anti-social revenue structure. Moreover, to capture this phenomenon that behavioral changes induced by the individual's perception of the external environment, the memory effect is also taken into account in this work. It suggests that individuals' stored information about community cooperative level (referred to as indirect experience) can trigger antisocial behavior by influencing their perceptions of antisocial revenue, and that their own past strategic information (referred to as direct experience) also plays a similar role. Numerous simulations show that the proposed mechanism significantly improves cooperation, especially when sensitivity to the environment is taken into account. Furthermore, when memory is considered, a lack of experience and insufficient environmental information may result in different optimal cooperative levels corresponding to different b. In addition, the impact of the sufficient indirect experience and the immediate environmental feedback on cooperation are nearly identical. When individuals have sufficient direct and indirect experiences, their roles can be interchangeably in decision-making. Finally, we discovered that the proposed mechanism exhibits good robustness. Overall, this study demonstrates an innovative use of quantitative calculations to describe the conclusions of qualitative analysis, which have theoretical and practical implications.

Investigating the effectiveness of interaction-efficiency-driven strategy updating under progressive-interaction for the evolution of the prisoner's dilemma game

In reality, interactive friend is added may one after another, instead of interacting with all at first. This work has proposed a progressive-interaction behavior driven by consistent aspiration, in which individuals interact with replacement neighbor one after another. Furthermore, payoff and interaction-efficiency are two factors considered in a modified Fermi rule, which particularly emphasizes that individuals achieving identical aspirations with less interaction frequency per unit of time will have higher interaction-efficiency and are more likely to be imitated. Thus, this work introduces the interaction-efficiency-driven strategy updating rules under progressive-interaction behavior in a spatial prisoner's dilemma game. As shown by numerical simulations, the progressive-interaction significantly enhances cooperation more than traditional versions, which may be caused by a heterogeneous number of interacting neighbors. When the interaction-efficiency-driven strategy update is taken into account, the cooperation is elevated to a higher level. Additionally, a specific point is employed to simulate the relationship between the aspiration and the b. As indicated by results, different b values have appropriate aspiration levels to promote the system to achieve the optimal cooperative level. More specifically, an obvious phase transition phenomenon can be found for the promotion results of individual aspiration level on cooperation. Besides, the phase transition phenomenon is existent when we simulate the relationship between the aspiration and the parameter delta for a given b value. In short, this work puts forward a new perspective on understanding cooperative evolution.

Promoting effect of adaptive interaction based on random neighbors to cooperation in the spatial prisoner's dilemma game

Interaction is an important step in cooperation evolution, and non-deterministic interactions are common in complex and changeable environments. Therefore, the number of interactive neighbors reflects the individual's response to the environment. Inspired by this, this study introduces an adaptively adjusted number of interactions based on random neighbors into a spatial prisoner's dilemma game, in which individuals adaptively adjust the number of interactive neighbors based on the satisfaction of returns. It is noteworthy that the interactive neighbors are randomly selected corresponding to the practical phenomenon that our friends will change over time, thereby resulting in un-fixed friends. As shown by numerous simulations, adaptively adjusting the number of interactive neighbors based on random neighbors can significantly enhance cooperation, and it performs better in strengthening cooperation than traditional deterministic interactions. Apart from that, cooperation can be promoted by an appropriate interactive neighborhood. Overall, this study is expected to help inspire more research on addressing social dilemmas.

The promoting effect of adaptive persistence aspiration on the cooperation based on the consideration of payoff and environment in prisoner's dilemma game

This work explores whether holding the last aspiration for a period of time can promote cooperation. Specifically, an evolutionary spatial prisoner's dilemma game mode is proposed, in which the players adjust strategies and aspirations by considering the payoff and environment. Therefore, the core is to allow players to hold the current aspiration for a period of time. Through numerical calculation, this study finds that the existence of an appropriate duration of aspiration can promote cooperation when b is less than a certain value. Moreover, the cooperation is gradually enhanced with the increase of T-max (maximum aspiration duration) when b is greater than it, but the enhancing effect is limited. It is also found that an appropriate value α (sensitivity to environmental change) can promote cooperation at different b intervals. Besides, this system indicates good robustness. Overall, this study provides a new perspective on exploring cooperative evolution based on aspiration.

Adaptive exit facilitates the evolution of cooperation in the spatial prisoner's dilemma game with punishment

The enforcement of costly punishment can facilitate cooperation. However, it is unclear how punishment would influence cooperation if individuals are free to exit an interaction. In this work, individuals are allowed to exit with a certain probability, which is adaptively adjusted based on the difference between expectations and payoffs. According to simulation results, adaptive exit can significantly improve cooperation even when punishment is weak; moreover, there is an optimal level of aspiration that causes adaptive exit to best promote the evolution of cooperation. Because of reciprocity, cooperative individuals maintain the lowest level of exit probability, allowing the relationship between them to be maintained, and the interaction between cooperative individuals and defectors is affected by adaptive exit processing, which is related to aspiration levels. Results show that cooperation prevails under moderate aspiration levels, because the exit mechanism protects cooperative individuals from exploitation by defectors; however, the exit mechanism does not prevent the invasion of cooperators by defectors when the aspiration level is low. The negative impact of sanctions causes defectors to increase their exit probability, and the exit of the defectors prevents nearby punishers from exploitation as well as reduces the implementation of punishment, thereby easing the punishment burden on the punisher. Therefore, the adaptive exit mechanism helps to eliminate second-order free-riding at a lower aspiration level. The study identifies the significant effects of exit on punishment, and highlights the important role of their association in the evolution of cooperation.

Incorporating social payoff into reinforcement learning promotes cooperation.

Reinforcement learning has been demonstrated to be an effective approach to investigate the dynamic of strategy updating and the learning process of agents in game theory. Most studies have shown that Q-learning failed to resolve the problem of cooperation in well-mixed populations or homogeneous networks. To this aim, we investigate the self-regarding Q-learning's effect on cooperation in spatial prisoner's dilemma games by incorporating the social payoff. Here, we redefine the reward term of self-regarding Q-learning by involving the social payoff; that is, the reward is defined as a monotonic function of the individual payoff and the social payoff represented by its neighbors' payoff. Numerical simulations reveal that such a framework can facilitate cooperation remarkably because the social payoff ensures agents learn to cooperate toward socially optimal outcomes. Moreover, we find that self-regarding Q-learning is an innovative rule that ensures cooperators coexist with defectors even at high temptations to defection. The investigation of the emergence and stability of the sublattice-ordered structure shows that such a mechanism tends to generate a checkerboard pattern to increase agents' payoff. Finally, the effects of Q-learning parameters are also analyzed, and the robustness of this mechanism is verified on different networks.

Impact of multi-step punishment on the spatial prisoner's dilemma game

Punishment has been proved to be an effective means for sustaining and promoting the evolution of cooperation. However, most works only consider one-off punishment, that is, the effect of each punishment only acts at one time step, it is often inconsistent with real-world scenarios. Therefore, different from previous works, we propose a new punishment model, which extends the punishment from one step to multiple ones. In particular, we assume that individuals will change their reputation scores according to their own behaviors, and begin to accept the punishment with intensity w when their reputation is below the reputation threshold. Through plenty of Monte Carlo simulations, it is found that there exists an optimal punishment time window to foster the cooperative behaviors, which indicates that the multi-step punishment is a double-edged sword, and too long or too short punishment is not conductive to sustain the collective cooperation.

Does a resource-storing mechanism favor “the wealthy do not fight”?—An approach from evolutionary game theory

Motivated by an interdisciplinary question of whether a resource-storing mechanism favors “the wealthy do fight” or “the wealthy do not fight,” we establish a new model based on spatial prisoner's dilemma (SPD) game where a time-accumulating payoff is allowed, and the probability of game participation depending on wealthiness is introduced. Although the model is based on a universal framework, presuming SPDs as a template, the conclusion drawn from the model is interdisciplinary rich. Our simulation results reveal that the common sense (or say, human's philosophical wisdom) of “the wealthy do not fight” helps to emerge a fairly cooperative society depending on the dilemma strength. By contrast, remarkably, it suggests that the common sense of “the wealthy do fight” begets a skewed exploitation society where defectors who exploit poor cooperators prosper.PACS numbersTheory and modeling; computer simulation, 87.15.Aa; Dynamics of evolution, 87.23.Kg.

Evolution of cooperation under the aspiration-based interactive diversity in the spatial prisoner's dilemma game

In the real world, an individual may have different attitudes towards his friends. From the perspective of the evolutionary game theory, it can be reflected that individuals tend to adopt different strategies towards different opponents, which can be defined as the interactive diversity. Up to now, the evolution of cooperation with individual characteristics under the interactive diversity still needs to be further explored. Herein, under the framework of interactive diversity, we consider the influence of individual aspiration on the evolution of cooperation in the prisoner's dilemma game, in which the parameter α is introduced to represent the individual aspiration level and used to measure individual satisfaction to the neighbors in the process of strategy updating. We find that for different temptations to defect, there is an optimal interval about the aspiration level α, which can significantly promote the evolution of cooperation. Moreover, we observe that the phase transition phenomenon for the cooperation exists, and the reason for this phenomenon to occur is also analyzed from the microscopic views. Overall, the current findings can provide some interesting insights into the role of individual aspiration and the interactive diversity in solving social dilemmas.

Aspiration Adjustment Based on Environmental Factors and Aspiration Duration Promote Cooperation in Spatial Prisoner's Dilemma Game

Aspiration Adjustment Based on Environmental Factors and Aspiration Duration Promote Cooperation in Spatial Prisoner's Dilemma Game

Effects of Stubborn Players and Noise on the Evolution of Cooperation in Spatial Prisoner's Dilemma Game

Effects of Stubborn Players and Noise on the Evolution of Cooperation in Spatial Prisoner's Dilemma Game

Progressive-Interaction-Mechanism Induce by Aspiration Promote Cooperation in Spatial Prisoner's Dilemma Game

Progressive-Interaction-Mechanism Induce by Aspiration Promote Cooperation in Spatial Prisoner's Dilemma Game

Evolution of cooperation in prisoner's dilemma game under the coupling of aspiration and imitation rules

Considering the important roles played by aspiration and imitation rules, we explore how cooperation evolves in the spatial prisoner's dilemma game. In detail, the strategy updating rule couples the inner aspiration payoff and the external neighborâs payoff through the parameter α. Intriguingly, we find that there exists an optimal value of α leading to the highest density of cooperators for each aspiration level A. By analyzing the dynamical strategy changes of the system, we find that an optimal α helps individuals form cooperator clusters and curbs the further spread of high-payoff defectors, and thus improves the density of cooperators. Moreover, when , the optimal value of α, which leads to the highest density of cooperators, decreases with A, the optimal range of α gradually expands and the highest cooperation level is on the rise.

Ability-based asymmetrical fitness calculation promotes cooperation in spatial prisoner's dilemma game

We investigate the ability-based fitness calculation in an asymmetric setup to the evolution of cooperation on square lattice. In particular, if players can maintain its strategy for a round, then its abilities will increase 1, otherwise, its ability will turn to 0. We also assume that if players’ abilities exceed a certain threshold M, then its environment is integrated into its fitness calculation via a simple parameter α. Through extensive numerical simulation, we find that large threshold M decreases cooperation, and large α, players’ fitness is mainly decided by its dynamical environment, promotes cooperation. We also provide theoretical analysis and get the same results with simulation. Finally, we test its robustness on homogeneous random network and small-world network. Our work provides a deeper insight to network reciprocity.

Reciprocal rewards promote the evolution of cooperation in spatial prisoner's dilemma game

To investigate the incentive mechanisms of cooperation, we propose an other-regarding preference based reciprocal rewarding mechanism by incorporating the dynamic adjustment of link weight into the spatial prisoner's dilemma game. Under the influence of other-regarding preference, players are willing to sacrifice personal payoff to reward the cooperative neighbor who has the greatest link weight. The reciprocal reward is dynamically adjusted according to the adaptive adjustment of link weight and the intensity of other-regarding preference. Extensive simulations reveal that the proposed rewarding mechanism significantly promotes the evolution of cooperation. Both the dynamic adjustment of link weight and the other-regarding preference impact factor play an essential role in the emergence of cooperative behavior. To further validate the above outcome, the evolution of cooperators' density and the alteration of average payoff and fitness are explored in a microscopic perspective.