Prisoner's Dilemma Payoff Research Articles

Maximizing cooperation in the prisoner's dilemma evolutionary game via optimal control.

The prisoner's dilemma (PD) game offers a simple paradigm of competition between two players who can either cooperate or defect. Since defection is a strict Nash equilibrium, it is an asymptotically stable state of the replicator dynamical system that uses the PD payoff matrix to define the fitness landscape of two interacting evolving populations. The dilemma arises from the fact that the average payoff of this asymptotically stable state is suboptimal. Coaxing the players to cooperate would result in a higher payoff for both. Here we develop an optimal control theory for the prisoner's dilemma evolutionary game in order to maximize cooperation (minimize the defector population) over a given cycle time T, subject to constraints. Our two time-dependent controllers are applied to the off-diagonal elements of the payoff matrix in a bang-bang sequence that dynamically changes the game being played by dynamically adjusting the payoffs, with optimal timing that depends on the initial population distributions. Over multiple cycles nT (n>1), the method is adaptive as it uses the defector population at the end of the nth cycle to calculate the optimal schedule over the n+1st cycle. The control method, based on Pontryagin's maximum principle, can be viewed as determining the optimal way to dynamically alter incentives and penalties in order to maximize the probability of cooperation in settings that track dynamic changes in the frequency of strategists, with potential applications in evolutionary biology, economics, theoretical ecology, social sciences, reinforcement learning, and other fields where the replicator system is used.

Nonlinear adaptive control of competitive release and chemotherapeutic resistance.

We use a three-component replicator system with healthy cells, sensitive cells, and resistant cells, with a prisoner's dilemma payoff matrix from evolutionary game theory, to model and control the nonlinear dynamical system governing the ecological mechanism of competitive release by which tumors develop chemotherapeutic resistance. The control method we describe is based on nonlinear trajectory design and energy transfer methods first introduced in the orbital mechanics literature for Hamiltonian systems. For continuous therapy, the basin boundaries of attraction associated with the chemo-sensitive population and the chemo-resistant population for increasing values of chemo-concentrations have an intertwined spiral structure with extreme sensitivity to changes in chemo-concentration level as well as sensitivity with respect to resistant mutations. For time-dependent therapies, we introduce an orbit transfer method to construct continuous families of periodic (closed) orbits by switching the chemo-dose at carefully chosen times and appropriate levels to design schedules that are superior to both maximum tolerated dose (MTD) schedules and low-dose metronomic (LDM) schedules, both of which ultimately lead to fixation of sensitive cells or resistant cells. By keeping the three subpopulations of cells in competition with each other indefinitely, we avoid fixation of the cancer cell population and regrowth of a resistant tumor. The method can be viewed as a way to dynamically shape the average population fitness landscape of a tumor to steer the chemotherapeutic response curve. We show that the method is remarkably insensitive to initial conditions and small changes in chemo-dosages, an important criterion for turning the method into an actionable strategy.

AN EVOLUTIONARY MODEL OF TUMOR CELL KINETICS AND THE EMERGENCE OF MOLECULAR HETEROGENEITY DRIVING GOMPERTZIAN GROWTH.

We describe a cell-molecular based evolutionary mathematical model of tumor development driven by a stochastic Moran birth-death process. The cells in the tumor carry molecular information in the form of a numerical genome which we represent as a four-digit binary string used to differentiate cells into 16 molecular types. The binary string is able to undergo stochastic point mutations that are passed to a daughter cell after each birth event. The value of the binary string determines the cell fitness, with lower fit cells (e.g. 0000) defined as healthy phenotypes, and higher fit cells (e.g. 1111) defined as malignant phenotypes. At each step of the birth-death process, the two phenotypic sub-populations compete in a prisoner's dilemma evolutionary game with the healthy cells playing the role of cooperators, and the cancer cells playing the role of defectors. Fitness, birth-death rates of the cell populations, and overall tumor fitness are defined via the prisoner's dilemma payoff matrix. Mutation parameters include passenger mutations (mutations conferring no fitness advantage) and driver mutations (mutations which increase cell fitness). The model is used to explore key emergent features associated with tumor development, including tumor growth rates as it relates to intratumor molecular heterogeneity. The tumor growth equation states that the growth rate is proportional to the logarithm of cellular diversity/heterogeneity. The Shannon entropy from information theory is used as a quantitative measure of heterogeneity and tumor complexity based on the distribution of the 4-digit binary sequences produced by the cell population. To track the development of heterogeneity from an initial population of healthy cells (0000), we use dynamic phylogenetic trees which show clonal and sub-clonal expansions of cancer cell sub-populations from an initial malignant cell. We show tumor growth rates are not constant throughout tumor development, and are generally much higher in the subclinical range than in later stages of development, which leads to a Gompertzian growth curve. We explain the early exponential growth of the tumor and the later saturation associated with the Gompertzian curve which results from our evolutionary simulations using simple statistical mechanics principles related to the degree of functional coupling of the cell states. We then compare dosing strategies at early stage development, mid-stage (clinical stage), and late stage development of the tumor. If used early during tumor development in the subclinical stage, well before the cancer cell population is selected for growth, therapy is most effective at disrupting key emergent features of tumor development.

Speed of retaliation and international cooperation

Many international interactions are structured like a prisoner’s dilemma because there are incentives to cooperate but also incentives to defect. In an infinitely repeated prisoner dilemma interaction, valuing the future highly enough by discounting the future less can ensure cooperation. Another factor that affects international cooperation in an infinitely repeated prisoner’s dilemma interaction is how quickly actors can respond to defection with retaliation, in other words, the speed of retaliation. Speeding up retaliation to ensure cooperation is an attractive strategy because it is flexible. It can be adapted to the particular characteristics of a dyad of cooperation and can vary with different actors and even different forms of cooperation between the same two actors. An analysis of an infinitely repeated prisoner’s dilemma interaction shows that if speeding up retaliation is costless, payoffs are constant in time, and the temptation payoff is reduced in proportion to the speed-up of retaliation, then no matter how unfavorable the starting discount factor and unfavorable the payoffs, speeding up retaliation can make cooperation sustainable. This, in combination with its inherent flexibility, makes speeding up retaliation a potentially powerful tool to make cooperation sustainable. Because speeding up retaliation likely has some implementation costs and because prisoner dilemma payoffs may not be constant/even in time, there are limits on when speeding up retaliation is effective. However, the analysis shows that if implementation costs are not too high, payoffs are not too uneven, or, if uneven in a particularly unfavorable way, punishment, when inflicted, is sufficiently punishing, then speeding up retaliation can still be effective in ensuring sustained cooperation.

Phase Diagram of Prisoner's Dilemma: Empirical Evidence from the Human Subjects Experiments

This paper presents the empirical phase diagram showing the cooperation phase, the mixed phase and the defection phase of prisoner’s dilemma experiment. The empirical data includes 21 experiments using a total of 350 human subjects. Experimental parameters are the possibility of continuation within [0, 1], the sizes of population within [12, 24], and a prisoner's dilemma payoff matrix (S, R, T, P) = (-2, 2, 4, 0). Twenty-one different combinations of these parameters were used in the experiments. The results provide the first empirical instance for Folk Theorem.

Sticks and carrots: Two incentive mechanisms supporting intra-group cooperation

In this note, we introduce two distinct incentive mechanisms that support dynamic intra-group cooperation in the context of prisoner's dilemma payoffs: rewards for cooperating, and punishments for defection, where the rewarding or punishing party may be outside the relationship.

TRANSFORMING THE DILEMMA

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S. In a well-mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.

Group vs Individual Performance in Mixed-Motive Situations: Exploring an Inconsistency

Previous research has shown that groups, as compared to individuals, perform better on integrative bargaining tasks (Thompson, Peterson, & Brodt, 1996), but worse on dilemma tasks (i.e., the “discontinuity effect”; Schopler & Insko, 1992). After reaching agreement on either a cooperative or a competitive integrative bargaining task in one of three formats (three-person group vs three-person group, three-person group vs a single individual, and individual vs individual), participants were given the opportunity to either keep the agreement or defect within a prisoner's dilemma payoff structure. Groups earned more points than individuals in the bargaining task, but continued to show the discontinuity effect even in the cooperative condition. Results are interpreted in terms of shared motives for group defection, which differed depending on whether the opponent was an individual or a group.

Only the Illusion of Possible Collusion? Cheap Talk and Similar Goals: Some Experimental Evidence

Firms routinely engage in public communications that are available to various constituencies, including competitors. In a laboratory experiment with prisoner's dilemma payoffs, the authors investigate the effect of one form of these communications—cheap talk signals: statements that are costless, nonbinding, and nonverifiable and do not directly affect the payoffs for either party. The authors find that only competitors that perceive that they share goals for a joint, coordinated outcome correctly update their beliefs about their competitor's next move on the basis of cheap talk signals. The authors contend that the conditions for cheap talk to work may be so rare that cheap talk is more likely to fall on deaf ears than to result in collusion. The authors suggest implications for managers and public policymakers as well as areas for further research.

On the Dynamic Persistence of Cooperation: How Lower Individual Fitness Induces Higher Survivability

We study a model in which cooperation and defection coexist in a dynamical steady state. In our model, subpopulations of cooperators and defectors inhabit sites on a lattice. The interactions among the individuals at a site, in the form of a prisoner's dilemma (PD) game, determine their fitnesses. The chosen PD payoff allows cooperators, but not defectors, to maintain a homogeneous population. Individuals mutate between types and migrate to neighboring sites with low probabilities. We consider both density-dependent and density-independent versions of the model. The persistence of cooperation in this model can be explained in terms of the life cycle of a population at a site. This life cycle starts when one cooperator establishes a population. Then defectors invade and eventually take over, resulting finally in the death of the population. During this life cycle, single cooperators migrate to empty neighboring sites to found new cooperator populations. The system can reach a steady state where cooperation prevails if the global “birth” rate of populations is equal to their global “death” rate. The dynamic persistence of cooperation ranges over a large section of the model's parameter space. We compare these dynamics to those from other models for the persistence of altruism and to predator–prey models.

Accounting for the ‘tragedy’ in the Prisoner's Dilemma

The Prisoner's Dilemma (PD) exhibits a ‘tragedy’ in this sense: if the players are fully informed and rational, they are condemned to a jointly dispreferred outcome. In this essay I address the following question: What feature of the PD's payoff structure is necessary and sufficient to produce the tragedy? In answering it I use the notion of a “trembling-hand equilibrium”. In the final section I discuss an implication of my argument, an implication which bears on the persistence of the problem posed by the PD.

Prisoner's dilemma payoff structure in interfirm strategic alliances: An empirical test

Interfirm strategic alliances (ISAs) are proliferating rapidly. The mixed motive—competitive and cooperative—nature of ISAs is hypothetically captured by the prisoner's dilemma (PD) model. Until now, empirical research in realistic business settings into the payoff structure obtained in ISAs has been lacking. After examining the incentives inherent in the PD payoff structure, this study presents the results of a US nationwide mail survey of 342 senior executives recently involved in ISAs. The results corroborate the existence of a payoff structure which satisfies the PD payoff structure. Some implications for promoting cooperative behavior as well as future research directions are discussed.