Payoff Information Research Articles

Hierarchical Noncooperative Dynamical Systems Under Intragroup and Intergroup Incentives

A framework for hierarchical noncooperative systems with dynamic agents is proposed. In the characterized framework, agents in each group are incentivized by a corresponding group manager who represents the benefits of group utility via an intra-group incentive mechanism. The coefficients in intra-group incentive functions are characterized as the group manager's strategy in this paper. The update rules that can be adopted by the group managers are proposed based on the local state and the payoff information with continual and intermittent observation on the state of the agents from the other groups. Sufficient conditions under which the trajectory of agents' state converges towards the group Nash equilibrium are derived for the proposed update rules. Furthermore, to improve the social welfare of the entire system, we propose an inter-group incentive scheme in the group managers level for a system governor to bring agents' state to a target equilibrium. To deal with the uncertain information on agents' personal payoff functions for the system governor, we present sufficient conditions to guarantee the convergence of agents' state to the target equilibrium. Three numerical examples are provided to illustrate the efficacy of our approach.

Knowing me, knowing you: an experiment on mutual payoff information in the stag hunt and Prisoner’s Dilemma

Knowing me, knowing you: an experiment on mutual payoff information in the stag hunt and Prisoner’s Dilemma

Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game

Many recent studies have used reinforcement learning methods to investigate the behavior of agents in evolutionary games. Q-learning, in particular, has become a mainstream method during this development. Here we introduce Q-learning agents into the evolutionary prisoner's dilemma game on a square lattice. Specifically, we associate the state space of Q-learning agents with the strategies of their neighbors, and we introduce a neighboring reward information sharing mechanism. We thus provide Q-learning agents with the payoff information of their neighbors, in addition to their strategies, which has not been done in previous studies. Through simulations, we show that considering neighborhood payoff information can significantly promote cooperation in the population. Moreover, we show that for an appropriate strength of neighborhood payoff information sharing, a chessboard pattern emerges on the lattice. We analyze in detail the reasons for the emergence of the chessboard pattern and the increase in cooperation frequency, and we also provide a theoretical analysis based on the pair approximation method. We hope that our research will inspire effective approaches for resolving social dilemmas by means of sharing more information among reinforcement learning agents during evolutionary games.

Strategic use of payoff information in k-hop evolutionary Best-shot networked public goods game

Globalization has led to increasingly interconnected interactions among individuals. Their payoffs are affected by the investment decision of themselves and their neighbors, which will cause conflicting interests between individual and social investment. Such problems can be modeled as a networked public goods game (NPGG). In this paper, we study the Best-shot NPGG model by introducing three mechanisms: k-hop, payoff information use strategy, and access cost. We use evolutionary game theory and present the k-hop evolutionary Best-shot networked public goods game (k-EBNPG) to explore the impact of these three mechanisms on social welfare. The results show that social welfare will increase with a diminishing margin as k increases while introducing the payoff information use strategy can significantly improve social welfare when k>1. Finally, we study the impact of access cost on social welfare and surprisingly find that social welfare will achieve the highest when the access cost is half the investment cost.

A simple mechanism for collective decision-making in the absence of payoff information

Animals are often faced with time-critical decisions without prior information about their actions' outcomes. In such scenarios, individuals budget their investment into the task to cut their losses in case of an adverse outcome. In animal groups, this may be challenging because group members can only access local information, and consensus can only be achieved through distributed interactions among individuals. Here, we combined experimental analyses with theoretical modeling to investigate how groups modulate their investment into tasks in uncertain conditions. Workers of the arboreal weaver ant Oecophylla smaragdina form three-dimensional chains using their own bodies to bridge vertical gaps between existing trails and new areas to explore. The cost of a chain increases with its length because ants participating in the structure are prevented from performing other tasks. The payoffs of chain formation, however, remain unknown to the ants until the chain is complete and they can explore the new area. We demonstrate that weaver ants cap their investment into chains, and do not form complete chains when the gap is taller than 90 mm. We show that individual ants budget the time they spend in chains depending on their distance to the ground, and propose a distance-based model of chain formation that explains the emergence of this tradeoff without the need to invoke complex cognition. Our study provides insights into the proximate mechanisms that lead individuals to engage (or not) in collective actions and furthers our knowledge of how decentralized groups make adaptive decisions in uncertain conditions.

Infants do not use payoff information to infer individual goals in joint-action events

For observers the occurrence of a joint action (JA) is inherently ambiguous with respect to the goals of the individuals involved. People may work together, for instance, because they are motivated to obtain material rewards or to help others. We hypothesized that to solve this interpretive ambiguity observers leverage information about the JA’s payoff structure. Specifically, when a JA yields material rewards for a participating agent (as well as their partner), their behavior can be straightforwardly explained as instrumental to the obtainment of these rewards. Conversely, when a JA does not yield material rewards for the agent (but does so for her partner), the unrewarded agent’s contribution needs to be accounted for by positing other types of goals, such as assisting her partner in obtaining her rewards. We examined this hypothesis across three looking-time experiments with 12-month-olds: specifically, we tested whether the absence of material rewards for an agent participating in a JA would prompt infants to interpret her participation as prosocially motivated. Consistent with this hypothesis, Experiment 1 showed that, after having been familiarized to two dyadic JA events resulting in one or both agents being rewarded, infants selectively expected the unrewarded agent to act altruistically towards her former JA partner by giving her a resource. Experiments 2–3 examined whether this expectation was driven by the prosocial interpretation of the unrewarded agent’s behavior or by changes in the number or distribution of resources between familiarization and test. Contradicting the hypothesis that infants interpreted the agent’s behavior as prosocially motivated, in Experiment 2 we observed similar looking times when the unrewarded agent performed a prosocial action (giving) or an antisocial action (taking) towards her partner at test. Further, in a close replication of the original experiment in which the change in the number of familiarized objects occurred in the test event featuring the rewarded agent rather than the unrewarded one (Experiment 3), infants produced a looking-time pattern opposite to the one first obtained. Taken together, these findings suggest that infants encoded the payoff structure of JA events (i.e., the number/distribution of resources that the interaction brought about) but did not leverage this information to infer the individual goals of participating agents. The present evidence calls for a critical re-evaluation of our original hypothesis and for further research into the mechanisms by which infants disambiguate the motives of agents involved in joint actions.

Emergence of oscillatory cooperation in a population with incomplete information

Oscillatory behaviors are ubiquitous in nature and the human society. However, most previous works fail to reproduce them in the two-strategy game-theoretical models. Here we show that oscillatory behaviors naturally emerge if incomplete information is incorporated into the cooperation evolution of a non-Markov model. Specifically, we consider a population playing prisoner’s dilemma game, where each individual can only probabilistically get access to their neighbors’ payoff information and store them within their memory with a given length. They make their decisions based upon these memories. Interestingly, we find that the level of cooperation generally cannot stabilize but render quasi-periodic oscillation, and this observation is strengthened for a longer memory and a smaller information acquisition probability. The mechanism uncovered shows that there are misperceived payoffs of the player’s neighborhood, facilitating the growth of cooperators and defectors at different stages that leads to oscillatory behaviors as a result. Our findings are robust to the underlying structure of the population. Given the omnipresence of incomplete information, our findings may provide a plausible explanation for the phenomenon of oscillatory behaviors in the real world.

Learning in rent-seeking contests with payoff risk and foregone payoff information

We test whether deviations from Nash equilibrium in rent-seeking contests can be explained by the slow convergence of payoff-based learning. We identify and eliminate two noise sources that slow down learning: first, opponents are changing their actions across rounds; second, payoffs are probabilistic, which reduces the correlation between expected and realized payoffs. We find that average choices are not significantly different from the risk-neutral Nash equilibrium predictions only when both noise sources are eliminated by supplying foregone payoff information and removing payoff risk. Payoff-based learning can explain these results better than alternative theories. We propose a hybrid learning model that combines reinforcement and belief learning with risk, social and other preferences, and show that it fits data well, mostly because of reinforcement learning.

Combining Conformist and Payoff Bias in Cultural Evolution : An Integrated Model for Human Decision-Making.

Most research on transmission biases in cultural evolution has treated different biases as distinct strategies. Here I present a model that combines both frequency dependent bias (including conformist bias) and payoff bias in a single decision-making calculus and show that such an integrated learning strategy may be superior to relying on either bias alone. Natural selection may operate on humans' relative dependence on frequency and payoff information, but both are likely to contribute to the spread of variants with high payoffs. Importantly, the magnitude of conformist bias affects the evolutionary dynamics, and I show that an intermediate level of conformity may be most adaptive and may spontaneously evolve as it resists the invasion of low-payoff variants yet enables the fixation of high-payoff variants in the population.

Reinforcement learning with foregone payoff information in normal form games

This paper studies the reinforcement learning of Erev and Roth with foregone payoff information in normal form games: players observe not only the realised payoffs but also foregone payoffs, the ones which they could have obtained if they had chosen the other actions. We provide conditions under which the reinforcement learning process almost surely converges to a regular quantal response equilibrium (Goeree et al. 2005). We also show that the reinforcement learning model and an adaptive learning model which nests experience-weighted attraction learning, payoff assessment learning and stochastic fictitious play learning models share the same asymptotic behaviour under the linear choice rule of the reinforcement learning model. In addition, we provide conditions under which the reinforcement learning process under the logit choice rule almost surely converges to a Nash equilibrium.

An evolutionary game with conformists and profiteers regarding the memory mechanism

Network evolutionary game theory provides a new perspective on how cooperative behaviors emerge in the real world and has been widely investigated. However, most studies assume that players are profiteers and ignore the existence of conformists and that players have memories, which are crucial when people make decisions. In this study, we study a memory-based snowdrift game occurring on networks and propose two strategy-updating rules based on profiteers and conformists while considering the historical strategy, memory strength, payoff information and memory length to discuss the emergence and maintenance of cooperation behaviors. In contrast to previous studies, we introduce the player’s degree of cooperation to continuousize player payoffs and we consider it when defining the player’s strategy-updating rules. In simulations, we show the evolution of the frequency of cooperation as time progresses and investigate the effects of the payoff parameter, memory strength, memory length and conformist ratio on the frequency of cooperation, and further validate the robustness of our model using different network sizes. Our results show that the memory strength, memory length, and conformist ratio can facilitate the cooperation level of the network over a large parameter area, and that the size of the network has almost no effect on the model, which shows the robustness of our model. Our work may elucidate the study of evolutionary games with conformists and memory effects.

Self-interested learning is more important than fair-minded conditional cooperation in public-goods games.

Why does human cooperation often unravel in economic experiments despite a promising start? Previous studies have interpreted the decline as the reaction of disappointed altruists retaliating in response to non-altruists (Conditional Cooperators hypothesis). This interpretation has been considered evidence of a uniquely human form of cooperation, motivated by an altruistic concern for equality ('fairness') and requiring special evolutionary explanations. However, experiments have typically shown individuals not only information about the decisions of their groupmates (social information) but also information about their own payoffs. Showing both confounds explanations based on conditional cooperation with explanations based on confused individuals learning how to better play the game (Confused Learners hypothesis). Here we experimentally decouple these two forms of information, and thus these two hypotheses, in a repeated public-goods game. Analysing 616 Swiss university participants, we find that payoff information leads to a greater decline, supporting the Confused Learners hypothesis. In contrast, social information has a small or negligible effect, contradicting the Conditional Cooperators hypothesis. We also find widespread evidence of both confusion and selfish motives, suggesting that human cooperation is maybe not so unique after all.

A Subtle Pronoun Shift Reduces Loss Aversion

In this paper, we propose and examine the de-biasing effect of self-distancing on loss aversion. In a laboratory setting, subjects made a set of binary decisions on whether to participate in a mixed gamble. Stemmed from the psychological literature on self-distancing, we manipulated the degree of self-distancing through the use of personal pronouns. We administered four treatment conditions —“I,” “You,” “No Pronoun,” and “Participant”—and subtly embedded this linguistic manipulation in the description of the payoff outcomes. Our results show that people exhibited a lower degree of loss aversion when the payoff information was conveyed using a pronoun that induced a higher level of self-distancing. Females were so much more responsive than were males to these subtle linguistic cues that the gender difference in loss aversion was closed in the two self-distanced conditions. The results have important theoretical and policy implications.

Knowing Me, Knowing You: An Experiment on Mutual Payoff Information and Strategic Uncertainty

We use a laboratory experiment to study how mutual payoff information affects play in strategic settings. Subjects play the Prisoner's Dilemma or Stag Hunt game against randomly re-matched opponents under two information treatments. In our partial-information treatment subjects are shown only their own payoffs, while in our full-information treatment they are shown both their own and their opponent's payoffs. In both treatments, they receive feedback on their opponent's action after each round. We find that mutual payoff information has a substantial effect on the equilibrium reached in the Stag Hunt and convergence in both games. Using a belief-learning model and simulations of play, we provide evidence that this effect is driven not only by initial play but also by the way subjects learn. We propose that strategic uncertainty is a probable channel driving these effects.

An escape strategy in orbital pursuit-evasion games with incomplete information

The orbital pursuit-evasion game is typically formulated as a complete-information game, which assumes the payoff functions of the two players are common knowledge. However, realistic pursuit-evasion games typically have incomplete information, in which the lack of payoff information limits the player’s ability to play optimally. To address this problem, this paper proposes a currently optimal escape strategy based on estimation for the evader. In this strategy, the currently optimal evasive controls are first derived based on the evader’s guess of the pursuer’s payoff weightings. Then an online parameter estimation method based on a modified strong tracking unscented Kalman filter is employed to modify the guess and update the strategy during the game. As the estimation becomes accurate, the currently optimal strategy gets closer to the actually optimal strategy. Simulation results show the proposed strategy can achieve optimal evasive controls progressively and the evader’s payoff of the strategy is lower than that of the zero-sum escape strategy. Meanwhile, the proposed strategy is also effective in the case where the pursuer changes its payoff function halfway during the game.

Evolution of conditional cooperation in public good games.

Cooperation declines in repeated public good games because individuals behave as conditional cooperators. This is because individuals imitate the social behaviour of successful individuals when their payoff information is available. However, in human societies, individuals cooperate in many situations involving social dilemmas. We hypothesize that humans are sensitive to both success (payoffs) and how that success was obtained, by cheating (not socially sanctioned) or good behaviour (socially sanctioned and adds to prestige or reputation), when information is available about payoffs and prestige. We propose and model a repeated public good game with heterogeneous conditional cooperators where an agent's donation in a public goods game depends on comparing the number of donations in the population in the previous round and with the agent's arbitrary chosen conditional cooperative criterion. Such individuals imitate the social behaviour of role models based on their payoffs and prestige. The dependence is modelled by two population-level parameters: affinity towards payoff and affinity towards prestige. These affinities influence the degree to which agents value the payoff and prestige of role models. Agents update their conditional strategies by considering both parameters. The simulations in this study show that high levels of cooperation are established in a population consisting of heterogeneous conditional cooperators for a certain range of affinity parameters in repeated public good games. The results show that social value (prestige) is important in establishing cooperation.

Payoff information and learning in signaling games

We add the assumption that players know their opponents' payoff functions and rationality to a model of non-equilibrium learning in signaling games. Agents are born into player roles and play against random opponents every period. Inexperienced agents are uncertain about the prevailing distribution of opponents' play, but believe that opponents never choose conditionally dominated strategies. Agents engage in active learning and update beliefs based on personal observations. Payoff information can refine or expand learning predictions, since patient young senders' experimentation incentives depend on which receiver responses they deem plausible. We show that with payoff knowledge, the limiting set of long-run learning outcomes is bounded above by rationality-compatible equilibria (RCE), and bounded below by uniform RCE. RCE refine the Intuitive Criterion (Cho and Kreps, 1987) and include all divine equilibria (Banks and Sobel, 1987). Uniform RCE sometimes but not always exists, and implies universally divine equilibrium.

Information fusion based on reputation and payoff promotes cooperation in spatial public goods game

Reputation information plays an important role in human behavioral interactions. There have been many studies that have considered reputation in spatial public goods game models, but they essentially assumed that reputation will only change the payoff structure of the game (individuals with a good reputation will get more benefits in the future). In fact, individuals with good reputation will have greater influences which can also affect the decision of neighbors around them in the strategy updating process. Grounded on this observation, we proposed an improved strategy learning rule considering both reputation and payoff information. We employ evidence theory to fuse these two aspects of information, based on which individual strategies can be updated. In addition, we construct a weight coefficient to quantify the importance and reliability of reputation. Through numerical simulations, it is unraveled that the reputation effect can greatly attract nearest neighbors to form greater clusters, thus promoting the emergence of cooperation. With the reputation weight increasing gradually, the critical enhancement factor for cooperation to arise is by degrees reduced to the lower boundary, demonstrating that an increasing tendency of strategy adoption relying on reputation is more likely to allow cooperation to thrive. In the region of low reputation weights, reputation only plays a subtle role in inducing cooperation. Within the region of high reputation weights, cooperation is dramatically boosted by reputation and cooperators can swiftly occupy the whole population. Our work may be helpful to further understand the effect of reputation on the emergence of cooperation.

Awareness of wealth inequalities breeds animosity

Abstract Cooperation is a ubiquitous trait among individuals of a species which interbreeds in nature and forms part of groups [1] . Various theories and models have been developed to explain the origins, maintenance, and continuation of cooperation within a generic population. Among these theories is the prisoner's dilemma (PD), which is a well-known framework for investigations on agents’ behavior and strategies [2] . In a traditional PD game, pairwise players are assigned the same initial endowment, and they have no access to (nor knowledge of) the opponents’ wealth throughout the course of the game. In real life, however, circumstances operate which (in most of the cases) unequally, asymmetrically or hierarchically distribute the income and resources, and the group members may be aware (completely or partially) of their opponents’ power and capacity before making their own decision to cooperate or defect. We experimentally reveal the impact of inequality and visibility by means of comparing the results of four sessions where players (1) may have equal or unequal initial endowments and (2) may be visible or invisible to opponents as far as their wealth information is concerned. Our observations disclose that wealth information makes no significant difference to the promotion of cooperation when the initial wealth distribution is fair. In the case of initially unequal endowments, hiding the payoff information enhances the frequency of cooperation while visibility of the opponent's wealth harms group synergy.

Just ask? Preference revelation and lying in a public goods experiment

This paper studies the extent to which simply asking group members to report their demand-type promotes cooperation in a public goods experiment with private, unobservable incentives. The design uses a simple public goods game, in which group members differ in both their dominant strategy contributions and socially optimal contributions. Treatments are conducted in which individuals either do not report their private payoff information, report only to group members, report to group members who can punish them, or report to a binding mechanism that charges them the socially optimal contribution for someone of their reported type. In all cases, messages are non-verifiable and participants are told that they are free to lie. When participants are able to report their type, either to their group members or to a binding mechanism, they contribute more. When reporting to a mechanism, there is a positive effect initially that fades with repetition, while reporting to group members is particularly effective at sustaining cooperation over time. Further, the misreporting of type is less frequent, considered more dishonest, and punished more harshly than free-riding. Consistent with work showing that weak punishment can backfire, the punishment mechanism is underutilized in this environment and its presence negates the positive effect of sharing information.