Pair Approximation Method Research Articles

Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game

Many recent studies have used reinforcement learning methods to investigate the behavior of agents in evolutionary games. Q-learning, in particular, has become a mainstream method during this development. Here we introduce Q-learning agents into the evolutionary prisoner's dilemma game on a square lattice. Specifically, we associate the state space of Q-learning agents with the strategies of their neighbors, and we introduce a neighboring reward information sharing mechanism. We thus provide Q-learning agents with the payoff information of their neighbors, in addition to their strategies, which has not been done in previous studies. Through simulations, we show that considering neighborhood payoff information can significantly promote cooperation in the population. Moreover, we show that for an appropriate strength of neighborhood payoff information sharing, a chessboard pattern emerges on the lattice. We analyze in detail the reasons for the emergence of the chessboard pattern and the increase in cooperation frequency, and we also provide a theoretical analysis based on the pair approximation method. We hope that our research will inspire effective approaches for resolving social dilemmas by means of sharing more information among reinforcement learning agents during evolutionary games.

A Pair-Approximation Method for Modelling the Dynamics of Multi-Agent Stochastic Games

Developing a dynamical model for learning in games has attracted much recent interest. In stochastic games, agents need to make decisions in multiple states, and transitions between states, in turn, influence the dynamics of strategies. While previous works typically focus either on 2-agent stochastic games or on normal form games under an infinite-agent setting, we aim at formally modelling the learning dynamics in stochastic games under the infinite-agent setting. With a novel use of pair-approximation method, we develop a formal model for myopic Q-learning in stochastic games with symmetric state transition. We verify the descriptive power of our model (a partial differential equation) across various games through comparisons with agent-based simulation results. Based on our proposed model, we can gain qualitative and quantitative insights into the influence of transition probabilities on the dynamics of strategies. In particular, we illustrate that a careful design of transition probabilities can help players overcome the social dilemmas and promote cooperation, even if agents are myopic learners.

A proportional-neighborhood-diversity evolution in snowdrift game on square lattice

The subject of how to facilitate cooperation of evolution through different mechanisms has been intensively investigated. For snowdrift game, researches have shown that a moderate dilution of the number of neighbors of individuals will benefit the evolution of cooperation, and have not shown how different dilution degrees effect evolution of cooperation. This paper proposes a proportional-neighborhood-diversity (PND) mechanism which takes into account some relatively typical situations. The core lies in: We provide a proportional vector to indicate the proportion of individuals with different number of interactive neighbors. Each player has fixed spatial neighbors and only plays with interactive neighbors selected from its spatial neighbors. This study is performed by means of the Monte Carlo method and an extended pair-approximation method. The Monte Carlo simulation results show that, compared with the traditional case, introducing different dilution degrees in evolution promotes the emergence of cooperative behavior. An interesting phenomenon is that when most individuals have a smaller number of interactive neighbors, the cooperation level is relatively higher. When applied to the evolutionary game, the PND mechanism can reduce game cost since the number of interactive neighbors is less than that of fixed neighbors in whole evolutionary process. Moreover, it has potential role in social management since the proposed mechanism is more realistic.

Blume–Emery–Griffiths model for a magnetic bilayer. Magnetocaloric properties in the presence of the 1st-order phase transitions

In the paper the Blume–Emery–Griffiths model is studied for a magnetic bilayer with coordination number z=5. The Cluster Variational Method in Pair Approximation is employed. A special emphasis is put on the magnetic and magnetocaloric properties in the cases of the 1st order phase transitions. For such cases the magnetization, quadrupolar moment and magnetic entropy, as well as the isothermal entropy change and the adiabatic temperature change during demagnetization process are calculated. It is shown that in the vicinity of the temperature corresponding to the 1st order phase transitions the adiabatic demagnetization is not possible. Instead, the demagnetization process with the smallest entropy production is proposed.

A novel bilateral protocol in the bipartite network based on the public goods game

Games with more than two participants are less efficient to establish or maintain than traditional pairwise games, because each player is obliged to gather the information of multiple partners before formulating strategies or figuring out the results. In reality, exchanges between only two individuals are predominant in economic activities, for example, the transaction between a retailer and a consumer, a loaner and a loanee, an employer and an employee, etc. Generally, a highly hierarchical and diversified structure is developed in the economic system which coordinates the mutual supply between entities without direct contacts. This work investigates the public goods game on the bipartite network, where the multilateral correlations in the groups are substituted by bilateral ones between the heterogeneous nodes from a two-layer hierarchy: the layer of normal players, and the layer of group nodes denoting the groups. Strategies are selected by each type of nodes with different frequency and method. Specifically, three strategy updating protocols are designed for the group nodes. In the most fundamental condition, the game is identical to the classic public goods game. However, when alternative protocols are applied, the cooperation tendency is influenced significantly, which is a phenomenon cannot be observed in traditional public goods games on homogeneous networks. The pair approximation method is employed to predict the cooperation tendency of the system. Furthermore, the theoretic conclusion is verified in the simulation, meanwhile the factors that affect the altruism behavior are investigated.

Compensation and Hysteresis Behaviors in a Heisenberg Ferrimagnetic Mixed Spin (½, 1) System

The hysteresis loops and compensation behaviors of a ferrimagnetic mixed system, consisting of two sublattices (A and B) with different spins (SA = 1 and SB = ½) and described by Heisenberg Hamiltonian on a cubic lattice, are investigated by using the pair approximation method. We have discussed the dependence of the compensation temperature on the D- crystal field interaction and J-exchange anisotropy. The influence of the longitudinal magnetic field and the magnetic hysteresis loops are also carried out. It is found that the model of mixed spin exhibits the possibility of two compensation points for several given sets of parameters. The model yields also second-order and first-order phase transitions, in addition to tricritical points. The reentrant phenomena can occur in some narrow ranges of D /|Jz| and Jxy /|Jz |. We have also observed the existence of triple hysteresis loop behaviors under the effects of special intrinsic parameters.

The dynamics of cooperation in asymmetric sub-populations

Sacrificing personal benefits for a common good is at odds with the fundamental principle of Darwinian evolution: if only the fittest survives, then there should be no place for cooperation. But cooperative behavior actually abounds, and constitutes one of the most persistent and fascinating puzzles of nature. One solution to this puzzle is network reciprocity, where the collective dynamics of cooperators affords them protection against invading defectors. Commonly, however, such a competition does not unfold in isolation. Populations are often divided into sub-populations, with different evolutionary rules describing the interactions between them. Here we propose and study a paradigmatic model that captures the essence of this setup. Specifically, if two players belong to the same sub-population, they play the prisoner’s dilemma game. If not, they play either the harmony game, the snowdrift game, the stag-hunt game, or the prisoner’s dilemma game. Due to such an asymmetry in the interactions across sub-populations, a fascinating evolutionary dynamics sets up that greatly expands the survivability of cooperators. For instance, when the harmony game applies, cyclic dominance spontaneously emerges, wherein cooperators in one sub-population become predators of defectors in the other sub-population. One also may observe self-organized segregation, wherein both sub-populations maintain a mixed state of cooperators and defectors. As a general rule, we show that the lower the dilemma strength between sub-populations, the more abundant the cooperative strategy in the entire population. Results are confirmed by means of Monte Carlo simulations with pair approximation method, which reveals a rich plethora of novel and generally valid paths to cooperation.

The Pair Approximation method for the ferromagnetic Heisenberg model with spin [formula omitted] and arbitrary range of interactions. Application for the magnetic semiconductor CrIAs

The Pair Approximation method has been formulated for the isotropic ferromagnetic Heisenberg model with spin S=1. The exchange interactions of arbitrary range have been taken into account. The single-ion anisotropy has been considered as well as the external magnetic field. Within the method, the Gibbs free-energy has been derived, from which all thermodynamic properties can be self-consistently obtained. In order to illustrate the developed formalism, the numerical calculations have been performed for CrIAs planar magnetic semiconductor, a hypothetical material whose existence has been recently predicted by the Density Functional Theory-based calculations. For this model material, all the relevant thermodynamic magnetic properties have been studied. The numerical results have been presented in the figures and discussed.

Magnetocaloric Effect in Anisotropic Mixed Spin –1 System: Pair Approximation Method

We use the Pair Approximation method to analyze the magnetic and magnetocaloric behaviors of diluted mixed spin SA = 1 and spin SB = 1/2 with the anisotropic Heisenberg model, on a cubic lattice with coordination number z=6. Our system is described in presence of an external magnetic field; the phase diagram and thermodynamic properties related to the concentration of magnetic atom (A or B) and the single ion anisotropy are constructed and discussed. Special attention is paid to magnetocaloric properties provided by isothermal entropy change as well as the cooling capacity. These cooling power keys are plotted and discussed as a function of interaction anisotropy and magnetic component concentration of two sublattices ions A and B. Numerical results show a double peak structure in the entropy change curve and the inverse magnetocaloric effect related to the presence of the negative single-ion anisotropy.

Parent-preferred dispersal promotes cooperation in structured populations.

Dispersal is a key process for the emergence of social and biological behaviours. Yet, little attention has been paid to dispersal's effects on the evolution of cooperative behaviour in structured populations. To address this issue, we propose two new dispersal modes, parent-preferred and offspring-preferred dispersal, incorporate them into the birth-death update rule, and consider the resultant strategy evolution in the prisoner's dilemma on random-regular, small-world, and scale-free networks, respectively. We find that parent-preferred dispersal favours the evolution of cooperation in these different types of population structures, while offspring-preferred dispersal inhibits the evolution of cooperation in homogeneous populations. On scale-free networks when the strength of parent-preferred dispersal is weak, cooperation can be enhanced at intermediate strengths of offspring-preferred dispersal, and cooperators can coexist with defectors at high strengths of offspring-preferred dispersal. Moreover, our theoretical analysis based on the pair-approximation method corroborates the evolutionary outcomes on random-regular networks. We also incorporate the two new dispersal modes into three other update rules (death-birth, imitation, and pairwise comparison updating), and find that similar results about the effects of parent-preferred and offspring-preferred dispersal can again be observed in the aforementioned different types of population structures. Our work, thus, unveils robust effects of preferential dispersal modes on the evolution of cooperation in different interactive environments.

Rumor Spreading of an SIHR Model in Heterogeneous Networks Based on Probability Generating Function

This paper focuses on the modeling of a rumor spreading in heterogeneous networks. Using the probability generating function method and pair approximation method, the current research obtains nonlinear differential equations to describe the dynamics of rumor spreading. The comparison between numerical simulations and Monte Carlo simulations confirms the accuracy of our model. Furthermore, the threshold condition is also obtained in this paper. The numerical simulation results show that the heterogeneity of the network accelerates the outbreak of rumors but reduces the maximum density of spreader and the scale of rumors. The present study also examines the effects of parameters on rumor transmission and the differences between rumor transmission recovery mechanisms and disease transmission recovery mechanisms.

Dynamics analysis of SIR epidemic model with correlation coefficients and clustering coefficient in networks

In this paper, the correlation coefficients between nodes in states are used as dynamic variables, and we construct SIR epidemic dynamic models with correlation coefficients by using the pair approximation method in static networks and dynamic networks, respectively. Considering the clustering coefficient of the network, we analytically investigate the existence and the local asymptotic stability of each equilibrium of these models and derive threshold values for the prevalence of diseases. Additionally, we obtain two equivalent epidemic thresholds in dynamic networks, which are compared with the results of the mean field equations.

Changing the Intensity of Interaction Based on Individual Behavior in the Iterated Prisoner’s Dilemma Game

We present a model of changing the intensity of interaction based on the individual behavior to study the iterated prisoner’s dilemma game in social networks. In this model, each individual has an assessed score of reputation which is obtained by considering the evaluation level of interactive partners for its present behavior. We focus on the effect of evaluation level on the changing intensity of interaction between individuals. For an individual with good behavior, the higher the evaluation level of its partners for its good behavior, the better its reputation, and the higher the probability of surrounding partners interaction with it. On the contrary, for an individual with bad behavior, the lower the evaluation level of its partners for its bad behavior, the worse its reputation, and the less the probability of surrounding neighbors interaction with it. Simulation results show that this effective mechanism can drastically facilitate the emergence and maintenance of cooperation in the population under a treacherous chip. Interestingly, for a small or moderate treacherous chip, the cooperation level monotonously ascends as the evaluation level increases; however, for a higher treacherous chip, existing an optimal evaluation level, which can result in the best promotion of cooperation. Furthermore, we find better agreement between simulation results and theoretical predictions obtained from an extended pair-approximation method, although there are some tiny deviations. We also show some typical snapshots of the system and investigate the reason for appearance and persistence of cooperation. The results further show the importance of evaluation level of individual behavior in coevolutionary relationships.

Application of pair approximation method to modeling and analysis of a marriage network

Numerous studies on marital issues have frequently resorted to statistical analysis or mathematical modeling on a well-mixed homogeneous population. However, such mathematical models have largely ignored the effects of heterogeneous contact pattern among individuals of the population. In this paper, we propose a pair model to describe the evolution of a marriage network, where an unmarried male and an unmarried female become friends with a certain probability through the introduction of their common friends. For convenience this probability is termed as the introduction probability. A threshold is obtained to determine whether there is a married couple or not. We verify the model by good agreement between the numerical solution of the model, computational simulations and the real marriage data in Beijing. The results show that the positive equilibrium of the model is steady and increases with the threshold. Moreover, the introduction probability contributes to increase both the threshold and the married population size. Furthermore, it is found that increasing the number of same sex friends, and balancing the sex ratio can also enlarge the married population size. All these results can provide some insights into the evolution and the structure of the marriage network.

Changing intensity of interaction can resolve prisoner's dilemmas

We put forward a computational model which mainly focuses on the effect of changing the intensity of interaction between individuals to study the evolutionary prisoner's dilemma game in social networks. In this model, an individual will unilaterally increase the intensity of interaction from it to some of its neighbors in case it is satisfied with the current income which it obtains from the neighbor; conversely, the individual will unilaterally reduce the intensity of interaction from it to its neighbor. We show that this simple evolutionary rule can effectively shift the survival barrier of cooperators and drastically facilitate the emergence of cooperation. Interestingly, for a fixed temptation to defect, there exists the smallest increment of intensity of interaction, resulting in a plateau of high cooperation level due to the positive feedback mechanism. Furthermore, we find good agreement between simulation results and theoretical predictions obtained from an extended pair-approximation method. Meanwhile, we illustrate the dynamical evolution of cooperators on the network, and investigate the impact of noise during the strategy updates.

Pair approximation method for spin-1 Heisenberg system

Spin-1 Heisenberg system on simple cubic lattice is considered in the pair approximation method assuming that the second-nearest-neighbor exchange interaction parameter has a negative value. The system is described in presence of an external magnetic field. The effects of the negative single-ion anisotropy and the negative second-nearest-neighbor exchange interaction on magnetization, internal energy, heat capacity, entropy and free energy are investigated. There are diverse anomalies at low temperature. In the magnetization and other thermodynamic quantities, the first-order phase transitions from ferromagnetic state to antiferromagnetic state and from ferromagnetic state to paramagnetic state have been observed.

Normal and inverse magnetocaloric effect in magnetic multilayers with antiferromagnetic interlayer coupling

The thermodynamics of a spin-1/2 magnetic multilayer system with antiferromagnetic interplanar couplings is studied using the pair approximation method. Special attention is paid to magnetocaloric properties, quantified by isothermal entropy change. The multilayer consists of two kinds of magnetic planes, one of which is diluted. The intraplanar couplings in both planes have arbitrary anisotropy ranging between Ising and isotropic Heisenberg interactions. The phase diagram related to the occurrence of magnetic compensation phenomenon is constructed and discussed. Then the isothermal entropy change is discussed as a function of interaction parameters, magnetic component concentration and external magnetic field amplitude. The ranges of normal and inverse magnetocaloric effect are found and related to the presence or absence of compensation.

Selfish punishment with avoiding mechanism can alleviate both first-order and second-order social dilemma

Punishment, especially selfish punishment, has recently been identified as a potent promoter in sustaining or even enhancing the cooperation among unrelated individuals. However, without other key mechanisms, the first-order social dilemma and second-order social dilemma are still two enduring conundrums in biology and the social sciences even with the presence of punishment. In the present study, we investigate a spatial evolutionary four-strategy prisoner׳s dilemma game model with avoiding mechanism, where the four strategies are cooperation, defection, altruistic and selfish punishment. By introducing the low level of random mutation of strategies, we demonstrate that the presence of selfish punishment with avoiding mechanism can alleviate the two kinds of social dilemmas for various parametrizations. In addition, we propose an extended pair approximation method, whose solutions can essentially estimate the dynamical behaviors and final evolutionary frequencies of the four strategies. At last, considering the analogy between our model and the classical Lotka-Volterra system, we introduce interaction webs based on the spatial replicator dynamics and the transformed payoff matrix to qualitatively characterize the emergent co-exist strategy phases, and its validity are supported by extensive simulations.

Thermodynamic description of the Ising antiferromagnet on a triangular lattice with selective dilution by a modified pair-approximation method

The pair-approximation method is modified in order to describe systems with geometrical frustration. The Ising antiferromagnet on a triangular lattice with selective dilution (Kaya-Berker model) is considered and a self-consistent thermodynamic description of this model is obtained. For this purpose, the Gibbs free energy as a function of temperature, concentration of magnetic atoms on the selected sublattice, and external magnetic field is derived. In particular, the phase diagram is constructed and a comparison of different methods is presented. The thermodynamic quantities are discussed in the context of their physical validity, and the improvement in the description introduced by the modified method is emphasized.

Critical Temperature of Site-Diluted Spin-1/2 Systems with Long-Range Ferromagnetic Interactions

In the paper the Pair Approximation (PA) method for studies of the site-diluted spin-1/2 systems of arbitrary dimensionality with the long-range ferromagnetic interactions is adopted. The method allows to take into account arbitrary anisotropy of the interactions in the spin space, so it is not limited to purely Ising couplings. Within this approach, the Gibbs free energy is obtained, which allows to derive all the further interesting thermodynamic properties. In particular, we obtain an equation for the critical temperature of the second-order phase transitions for the model in question. In the study we focus our attention on the systems with ferromagnetic interactions decaying with the distance according to the power law $J(r)\propto r^{-n}$. We discuss the dependence of the critical temperature on the concentration of magnetic component and the index $n$ for selected one-, two- and three-dimensional lattices.We confirm the absence of the critical concentration for a diluted magnet with infinite interaction range. In the regime of the low concentrations of magnetic component, we find a non-linear increase of the critical temperature with the concentration in the form of $T_{c}\propto p^{n/d}$, depending on the system dimensionality $d$ and the index $n$, whereas $n > d$.