Evolutionary Prisoner's Dilemma Game Research Articles

Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game

Many recent studies have used reinforcement learning methods to investigate the behavior of agents in evolutionary games. Q-learning, in particular, has become a mainstream method during this development. Here we introduce Q-learning agents into the evolutionary prisoner's dilemma game on a square lattice. Specifically, we associate the state space of Q-learning agents with the strategies of their neighbors, and we introduce a neighboring reward information sharing mechanism. We thus provide Q-learning agents with the payoff information of their neighbors, in addition to their strategies, which has not been done in previous studies. Through simulations, we show that considering neighborhood payoff information can significantly promote cooperation in the population. Moreover, we show that for an appropriate strength of neighborhood payoff information sharing, a chessboard pattern emerges on the lattice. We analyze in detail the reasons for the emergence of the chessboard pattern and the increase in cooperation frequency, and we also provide a theoretical analysis based on the pair approximation method. We hope that our research will inspire effective approaches for resolving social dilemmas by means of sharing more information among reinforcement learning agents during evolutionary games.

The promoting effect of adaptive persistence aspiration on the cooperation based on the consideration of payoff and environment in prisoner's dilemma game

This work explores whether holding the last aspiration for a period of time can promote cooperation. Specifically, an evolutionary spatial prisoner's dilemma game mode is proposed, in which the players adjust strategies and aspirations by considering the payoff and environment. Therefore, the core is to allow players to hold the current aspiration for a period of time. Through numerical calculation, this study finds that the existence of an appropriate duration of aspiration can promote cooperation when b is less than a certain value. Moreover, the cooperation is gradually enhanced with the increase of T-max (maximum aspiration duration) when b is greater than it, but the enhancing effect is limited. It is also found that an appropriate value α (sensitivity to environmental change) can promote cooperation at different b intervals. Besides, this system indicates good robustness. Overall, this study provides a new perspective on exploring cooperative evolution based on aspiration.

Heterogeneity reproductive ability promotes cooperation in spatial prisoner's dilemma game

Abstract Heterogeneity which widely exists in human society and the biological world, has been proved to be one of the cases that can promote the level of cooperation. For example, the level of cooperation in a heterogeneous network is relatively high when compared with that in a homogeneous network. Recent research has revealed that the effect of heterogeneity in evolutionary game theory depends on the specific definition of heterogeneity and updating rule. In this paper we explore how heterogeneity reproductive ability affects the evolution of cooperation. In this paper we associate reproductive ability with the payoff. A player with a higher payoff and a better environment is more reproductive, which is a common phenomenon existing from human society to bacterial systems. Excitingly, we find that our new constructed rule can dramatically promote cooperation in evolutionary prisoner's dilemma game. From our numerical result, we also find that moderate heterogeneity can better promote the level of cooperation. Several figures are drawn to explain why heterogeneity reproductive ability can promotes the level of cooperation. Our work finds a new mechanism to promote cooperation in spatial prisoner's dilemma game and reveals why cooperation mushrooms with heterogeneity reproductive ability.

Roles of mutation rate and co-existence of multiple strategy updating rules in evolutionary prisoner's dilemma games

The emergence and maintenance of cooperation has attracted intensive scholarly interest and has been analysed within the framework of evolutionary game theory. The role of innovation, which introduces novel strategies into the population, is a relatively understudied aspect of evolutionary game theory. Here, we investigate the effects of two sources of innovation —mutation and heterogeneous updating rules. These mechanisms allow agents to adopt strategies that do not rely on the imitation of other individuals. The model introduces —in addition to canonical imitation-based strategy updating—aspiration-based updating, whereby agents switch their strategy by referring solely to the performance of their own strategy; mutation also introduces novel strategies into the population. Our simulation results show that the introduction of aspiration-based rules into a population of imitators leads to the deterioration of cooperation. In addition, mutation, in combination with heterogeneous updating rules, also diminishes cooperators. This phenomenon is prominent when a large proportion of the population consists of imitators rather than adopters of aspiration-based updating. Nevertheless, a high mutation rate, in combination with a low aspiration level, has positive nonlinear effects, and a heterogeneous population achieves a higher level of cooperation than the weighted average of homogeneous populations. Our results demonstrate the profound role of innovation in the evolution of cooperation.

The evolutionary prisoner's dilemma game considering behavior interaction in a two-layered network

The multilayer networks are often a significantly more apt description of real-life systems, and considering that an individual's strategic choice can be influenced by the interaction between individuals and the individual's access to information in different channels, we explore the evolutionary prisoner's dilemma (PD) game in a two-layered network. The numerical simulation results show that when both of the two layers networks are heterogeneous networks, it is more beneficial to promote cooperation. However, it takes a longer time for the whole system to reach the stable state. In addition, with the increase of the temptation gain and the proportion of players whose behaviors may be directly affected by their counterparts, the final density of cooperators in the entire system gradually decreases. For the individuals whose behaviors may be directly affected by their counterparts, however, if they preferred to follow their neighbors' strategies, it could be conducive to promote the individuals' cooperation behavior.

Evolutionary prisoner's dilemma games on the network with punishment and opportunistic partner switching

Punishment and partner switching are two well-studied mechanisms that support the evolution of cooperation. Observation of human behaviour suggests that the extent to which punishment is adopted depends on the usage of alternative mechanisms, including partner switching. In this study, we investigate the combined effect of punishment and partner switching in evolutionary prisoner's dilemma games conducted on a network. In the model, agents are located on the network and participate in the prisoner's dilemma games with punishment. In addition, they can opportunistically switch interaction partners to improve their payoff. Our Monte Carlo simulation showed that a large frequency of punishers is required to suppress defectors when the frequency of partner switching is low. In contrast, cooperation is the most abundant strategy when the frequency of partner switching is high regardless of the strength of punishment. Interestingly, cooperators become abundant not because they avoid the cost of inflicting punishment and earn a larger average payoff per game but rather because they have more numerous opportunities to be referred to as a role agent by defectors. Our results imply that the fluidity of social relationships has a profound effect on the adopted strategy in maintaining cooperation.

Impact of interaction style and degree on the evolution of cooperation on Barabási-Albert scale-free network.

In this work, we study an evolutionary prisoner’s dilemma game (PDG) on Barabási–Albert scale-free networks with limited player interactions, and explore the effect of interaction style and degree on cooperation. The results show that high-degree preference interaction, namely the most applicable interaction in the real world, is less beneficial for emergence of cooperation on scale-free networks than random interaction. Besides, cooperation on scale-free networks is enhanced with the increase of interaction degree regardless whether the interaction is high-degree preference or random. If the interaction degree is very low, the cooperation level on scale-free networks is much lower than that on regular ring networks, which is against the common belief that scale-free networks must be more beneficial for cooperation. Our analysis indicates that the interaction relations, the strategy and the game payoff of high-connectivity players play important roles in the evolution of cooperation on scale-free networks. A certain number of interactions are necessary for scale-free networks to exhibit strong capability of facilitating cooperation. Our work provides important insight for members on how to interact with others in a social organization.

Coevolution of game strategy and link weight promotes cooperation in structured population

Evolutionary prisoner's dilemma game in structured populations on a weighted square lattice, on which the edge weight represents the relationship between agents and adaptively changes in time, has been proved to be an efficient way that can promote cooperation. In fact, such an adaptive link weight introduces a new time scale τa, not necessarily equal to the time scale of game strategy τɛ. Inspired from aforementioned above, we investigate the effect of w=τɛτa on the evolution of cooperative behavior. Through numerical simulation, we find cooperation can be promoted effectively with a larger value of w, which is related to the increase of average link weight in the structured population.

Changing intensity of interaction can resolve prisoner's dilemmas

We put forward a computational model which mainly focuses on the effect of changing the intensity of interaction between individuals to study the evolutionary prisoner's dilemma game in social networks. In this model, an individual will unilaterally increase the intensity of interaction from it to some of its neighbors in case it is satisfied with the current income which it obtains from the neighbor; conversely, the individual will unilaterally reduce the intensity of interaction from it to its neighbor. We show that this simple evolutionary rule can effectively shift the survival barrier of cooperators and drastically facilitate the emergence of cooperation. Interestingly, for a fixed temptation to defect, there exists the smallest increment of intensity of interaction, resulting in a plateau of high cooperation level due to the positive feedback mechanism. Furthermore, we find good agreement between simulation results and theoretical predictions obtained from an extended pair-approximation method. Meanwhile, we illustrate the dynamical evolution of cooperators on the network, and investigate the impact of noise during the strategy updates.

Dynamics, morphogenesis and convergence of evolutionary quantum Prisoner's Dilemma games on networks.

The authors proposed a quantum Prisoner's Dilemma (PD) game as a natural extension of the classic PD game to resolve the dilemma. Here, we establish a new Nash equilibrium principle of the game, propose the notion of convergence and discover the convergence and phase-transition phenomena of the evolutionary games on networks. We investigate the many-body extension of the game or evolutionary games in networks. For homogeneous networks, we show that entanglement guarantees a quick convergence of super cooperation, that there is a phase transition from the convergence of defection to the convergence of super cooperation, and that the threshold for the phase transitions is principally determined by the Nash equilibrium principle of the game, with an accompanying perturbation by the variations of structures of networks. For heterogeneous networks, we show that the equilibrium frequencies of super-cooperators are divergent, that entanglement guarantees emergence of super-cooperation and that there is a phase transition of the emergence with the threshold determined by the Nash equilibrium principle, accompanied by a perturbation by the variations of structures of networks. Our results explore systematically, for the first time, the dynamics, morphogenesis and convergence of evolutionary games in interacting and competing systems.

Percolation in spatial evolutionary prisoner's dilemma game on two-dimensional lattices.

We study the spatial evolutionary prisoner's dilemma game with updates of imitation max on triangular, hexagonal, and square lattices. We use the weak prisoner's dilemma game with a single parameter b. Due to the competition between the temptation value b and the coordination number z of the base lattice, a greater variety of percolation properties is expected to occur on the lattice with the larger z. From the numerical analysis, we find six different regimes on the triangular lattice (z=6). Regardless of the initial densities of cooperators and defectors, cooperators always percolate in the steady state in two regimes for small b. In these two regimes, defectors do not percolate. In two regimes for the intermediate value of b, both cooperators and defectors undergo percolation transitions. The defector always percolates in two regimes for large b. On the hexagonal lattice (z=3), there exist two distinctive regimes. For small b, both the cooperators and the defectors undergo percolation transitions while only defectors always percolate for large b. On the square lattice (z=4), there exist three regimes. Combining with the finite-size scaling analyses, we show that all the observed percolation transitions belong to the universality class of the random percolation. We also show how the detailed growth mechanism of cooperator and defector clusters decides each regime.

Diverse strategy-learning styles promote cooperation in evolutionary spatial prisoner's dilemma game

Observational learning and practice learning are two important learning styles and play important roles in our information acquisition. In this paper, we study a spacial evolutionary prisoner's dilemma game, where players can choose the observational learning rule or the practice learning rule when updating their strategies. In the proposed model, we use a parameter p controlling the preference of players choosing the observational learning rule, and found that there exists an optimal value of p leading to the highest cooperation level, which indicates that the cooperation can be promoted by these two learning rules collaboratively and one single learning rule is not favor the promotion of cooperation. By analysing the dynamical behavior of the system, we find that the observational learning rule can make the players residing on cooperative clusters more easily realize the bad sequence of mutual defection. However, a too high observational learning probability suppresses the players to form compact cooperative clusters. Our results highlight the importance of a strategy-updating rule, more importantly, the observational learning rule in the evolutionary cooperation.

Payoff components and their effects in a spatial three-strategy evolutionary social dilemma.

We study a three-strategy spatial evolutionary prisoner's dilemma game with imitation and logit update rules. Players can follow the always-cooperating, always-defecting or the win-stay-lose-shift (WSLS) strategies and gain their payoff from games with their direct neighbors on a square lattice. The friendliness parameter of the WSLS strategy-characterizing its cooperation probability in the first round-tunes the cyclic component of the game determining whether the game can be characterized by a potential. We measured and calculated the phase diagrams of the system for a wide range of parameters. When the game is a potential game and the logit rule is applied, the theoretically predicted phase diagram agrees very well with the simulation results. Surprisingly, this phase diagram can be accurate even in the nonpotential case if there are only two surviving strategies in the stationary state; this result harmonizes with the fact that all 2×2 games are potential games. For the imitation dynamics, we found that the effects of spatiality combined with the presence of two cooperative strategies are so strong that they suppress even substantial changes in the payoff matrix, thus the phase diagrams are independent of the cyclic component's intensity. At the same time, this type of strategy update mechanism supports the formation of cooperative clusters that results in a cooperative society in a wider parameter range compared to the logit dynamics.

Diverse roles of the reduced learning ability of players in the evolution of cooperation

Individual heterogeneity in the reproductive rate is found to play an important role in the emergence and persistence of cooperation. Most of the existing literature focused mainly on the enhancement of cooperation by the introduction of inhomogeneous teaching capability of the individuals. It is far from clear how the heterogeneous learning ability of the individuals affects the evolution of cooperation. To fill this research gap, we make comparative studies of the evolutionary spatial prisoner's dilemma game with reduced learning or teaching ability of the players, under both synchronous and asynchronous strategy updating schemes. By carrying out extensive computer simulations, we show that cooperation can always be facilitated if the inhomogeneous teaching ability of the players is considered, irrespectively of the strategy updating manner. By contrast, cooperation is promoted (inhibited) in the case of synchronous (asynchronous) strategy updating, if heterogeneous learning ability is considered, which is attributed to the reduced ability of cooperators to expand their domains.

Impact of heterogeneous activity and community structure on the evolutionary success of cooperators in social networks.

Recent empirical studies suggest that heavy-tailed distributions of human activities are universal in real social dynamics [L. Muchnik, S. Pei, L. C. Parra, S. D. S. Reis, J. S. Andrade Jr., S. Havlin, and H. A. Makse, Sci. Rep. 3, 1783 (2013)]. On the other hand, community structure is ubiquitous in biological and social networks [M. E. J. Newman, Nat. Phys. 8, 25 (2012)]. Motivated by these facts, we here consider the evolutionary prisoner's dilemma game taking place on top of a real social network to investigate how the community structure and the heterogeneity in activity of individuals affect the evolution of cooperation. In particular, we account for a variation of the birth-death process (which can also be regarded as a proportional imitation rule from a social point of view) for the strategy updating under both weak and strong selection (meaning the payoffs harvested from games contribute either slightly or heavily to the individuals' performance). By implementing comparative studies, where the players are selected either randomly or in terms of their actual activities to play games with their immediate neighbors, we figure out that heterogeneous activity benefits the emergence of collective cooperation in a harsh environment (the action for cooperation is costly) under strong selection, whereas it impairs the formation of altruism under weak selection. Moreover, we find that the abundance of communities in the social network can evidently foster the formation of cooperation under strong selection, in contrast to the games evolving on randomized counterparts. Our results are therefore helpful for us to better understand the evolution of cooperation in real social systems.

Relational diversity promotes cooperation in prisoner's dilemma games.

Relational diversity can be characterized by heterogeneous distributions of tie strengths in social networks and this diversity is present not only among humans, but throughout the animal world. We account for this observation by analyzing two network datasets from Facebook. We measure the strength of a tie by calculating the extent of overlap of friends between the two individuals. Based on the previous findings in human experiments, we argue that it is very unlikely that players will allocate their investments equally to their neighbors. There is a tendency that players prefer to donate more to their intimate friends. We find that if players preferentially allocate their investments to their good friends, cooperation will be promoted in PDG. We proved that the facilitation of the cooperative strategy relies mostly on the cooperative allies between best friends, resulting in the formation of cooperative clusters which are able to prevail against the defectors even when there is a large cost to cooperate. Moreover, we discover that the effect of relational diversity cannot be analyzed by adopting classical complex networks models, because neither of the artificial networks is able to produce networks with diverse distributions of tie strengths. It is of vital importance to introduce real social networks to study the influence of diverse relations especially when it comes to humans. This research proposes a brand new perspective to understand the influence of social relations on the emergence of cooperation in evolutionary prisoner’s dilemma games.

Optimal Cooperation Enhancement on Spatial Networks

We investigate the evolutionary Prisoner's dilemma game on the simplest spatial networks constructed as geometrical graphs. The optimal cooperation enhancement against the topology randomness is found. It is proposed that the optimal behavior of the cooperation results from the competition between individuals with high degrees and with low degrees: the former assists the formation of cooperator clusters and the latter tends to prevent the formation of such clusters.

Entanglement guarantees emergence of cooperation in quantum prisoner's dilemma games on networks.

It was known that cooperation of evolutionary prisoner's dilemma games fails to emerge in homogenous networks such as random graphs. Here we proposed a quantum prisoner's dilemma game. The game consists of two players, in which each player has three choices of strategy: cooperator (C), defector (D) and super cooperator (denoted by Q). We found that quantum entanglement guarantees emergence of a new cooperation, the super cooperation of the quantum prisoner's dilemma games, and that entanglement is the mechanism of guaranteed emergence of cooperation of evolutionary prisoner's dilemma games on networks. We showed that for a game with temptation b, there exists a threshold for a measurement of entanglement, beyond which, (super) cooperation of evolutionary quantum prisoner's dilemma games is guaranteed to quickly emerge, giving rise to stochastic convergence of the cooperations, that if the entanglement degree γ is less than the threshold , then the equilibrium frequency of cooperations of the games is positively correlated to the entanglement degree γ, and that if γ is less than and b is beyond some boundary, then the equilibrium frequency of cooperations of the games on random graphs decreases as the average degree of the graphs increases.

Robustness of cooperation on scale-free networks in the evolutionary prisoner's dilemma game

We have studied the robustness of cooperation on scale-free (SF) networks in the prisoner's dilemma game under different attack strategies. Although previous works have demonstrated that increasing heterogeneity constitutes higher levels of cooperation, we elaborated on this subject further by introducing a parameter α to take into consideration two significant aspects during an attack. We have shown that it is possible to precisely control the cooperation level on SF networks to be robust or fragile . Moreover, we studied the evolution on SF networks against a different attack strategy, taking over the nodes instead of simply removing the nodes, to address the functional importance of a node. Notably, the network structure remains the same during the evolutionary process under this attack strategy, which allows us to investigate the correlation between the functional significance and survival of cooperation. Our results highlight the underlying mechanism of cooperation behavior on SF networks and have several important implications for public health and networks security.

Social dilemma alleviated by sharing the gains with immediate neighbors

We study the evolution of cooperation in the evolutionary spatial prisoner's dilemma game (PDG) and snowdrift game (SG), within which a fraction α of the payoffs of each player gained from direct game interactions is shared equally by the immediate neighbors. The magnitude of the parameter α therefore characterizes the degree of the relatedness among the neighboring players. By means of extensive Monte Carlo simulations as well as an extended mean-field approximation method, we trace the frequency of cooperation in the stationary state. We find that plugging into relatedness can significantly promote the evolution of cooperation in the context of both studied games. Unexpectedly, cooperation can be more readily established in the spatial PDG than that in the spatial SG, given that the degree of relatedness and the cost-to-benefit ratio of mutual cooperation are properly formulated. The relevance of our model with the stakeholder theory is also briefly discussed.