Asymmetric Reward Research Articles

Evolution of cooperation with asymmetric rewards

Rewards, as a form of positive reinforcement, effectively encourage cooperation. In this paper, we study a multi-population prisoner's dilemma game with asymmetric rewards, where agents in the same population play prisoner's dilemma game, and agents from the giver population can reward agents from the recipient population only if they make the same choice. In well-mixed populations, asymmetric rewards can facilitate cooperation. Similarly, asymmetric rewards on the regular square lattice can effectively prevent complete defection. In both well-mixed and structured populations, seemingly disadvantageous cooperative givers play an important role in the maintenance and spread of cooperation. Especially on lattice, cooperative givers with asymmetric rewards can be active in the system through diverse cases of cyclic dominance. Our findings provide deeper insights into the impact of asymmetry on cooperative behavior and the development of altruistic behavior in real-world scenarios.

Localized and global representation of prior value, sensory evidence, and choice in male mouse cerebral cortex.

Adaptive behavior requires integrating prior knowledge of action outcomes and sensory evidence for making decisions while maintaining prior knowledge for future actions. As outcome- and sensory-based decisions are often tested separately, it is unclear how these processes are integrated in the brain. In a tone frequency discrimination task with two sound durations and asymmetric reward blocks, we found that neurons in the medial prefrontal cortex of male mice represented the additive combination of prior reward expectations and choices. The sensory inputs and choices were selectively decoded from the auditory cortex irrespective of reward priors and the secondary motor cortex, respectively, suggesting localized computations of task variables are required within single trials. In contrast, all the recorded regions represented prior values that needed to be maintained across trials. We propose localized and global computations of task variables in different time scales in the cerebral cortex.

Exploiting Symmetry in Dynamics for Model-Based Reinforcement Learning With Asymmetric Rewards

Exploiting Symmetry in Dynamics for Model-Based Reinforcement Learning With Asymmetric Rewards

Fear of missing out and market stability: A networked minority game approach

This study examines the influence of the fear of missing out (FOMO), which spreads through social networks, on financial market stability. We extend the networked evolutionary minority game model, a widely used methodology in econophysics, by incorporating the asymmetric reward structure and investors’ FOMO. The agent-based simulation results reveal that the asymmetric reward structure significantly improves the stability of the system. In contrast, the FOMO significantly reduces market stability. FOMO-induced irrational behavior disrupts the market’s normal functioning, leading to decreased stability. However, as agents become more interconnected and actively adapt their strategies, the adverse effects of FOMO on the system diminish.

Neural Representations of Post-Decision Accuracy and Reward Expectation in the Caudate Nucleus and Frontal Eye Field.

Performance monitoring that supports ongoing behavioral adjustments is often examined in the context of either choice confidence for perceptual decisions (i.e., "did I get it right?") or reward expectation for reward-based decisions (i.e., "what reward will I receive?"). However, our understanding of how the brain encodes these distinct evaluative signals remains limited because they are easily conflated, particularly in commonly used two-alternative tasks with symmetric rewards for correct choices. Previously we used a motion-discrimination task with asymmetric rewards to identify neural substrates of forming reward-biased perceptual decisions in the caudate nucleus (part of the striatum in the basal ganglia) and the frontal eye field (FEF, in prefrontal cortex). Here we leveraged this task design to partially decouple estimates of accuracy and reward expectation and examine their impacts on subsequent decisions and their representations in those two brain areas. We identified distinguishable representations of these two evaluative signals in individual caudate and FEF neurons, with regional differences in their distribution patterns and time courses. We observed that well-trained monkeys (both sexes) used both evaluative signals, infrequently but consistently, to adjust their subsequent decisions. We found further that these behavioral adjustments had reliable relationships with the neural representations of both evaluative signals in caudate, but not FEF. These results suggest that the cortico-striatal decision network may use diverse evaluative signals to monitor and adjust decision-making behaviors, adding to our understanding of the different roles that the FEF and caudate nucleus play in a diversity of decision-related computations.

Reward System in Late-Life Depression: a Cross-Sectional Case-Control Study.

Anhedonia, commonly defined as a reduced ability to feel pleasure, is a core clinical symptom of late-life depression (LLD). Deficits in reward processing are hypothesised to be associated with anhedonia. We examined differences in reward sensitivity between patients with LLD and healthy controls and explored the associations between LLD-related symptomatology, global cognition, and the reward system. The reward responsiveness of 63 patients with LLD and 58 healthy controls aged ≥60 years was assessed using the probabilistic reward learning task with an asymmetric reward schedule. Compared with healthy controls, patients with LLD displayed lower response bias and reward learning. Global cognition of all participants was positively correlated with response bias. In patients with LLD, anhedonia severity explained impaired reward learning. A deficit in reward processing is implicated in patients with LLD. Our findings suggest that executive dysfunction and anhedonia contribute to lower sensitivity to reward learning in patients with LLD.

Understanding the mechanics and consumer risks associated with play-to-earn (P2E) gaming.

Play-to-earn (P2E) gaming is a newly emerging form of gaming increasingly based on blockchain technology. In this paper, we examine the mechanics and business model of these games and their potential benefits and risks for players. The paper draws upon and critically synthesises the developing published literature on predatory monetization in gaming as well as objective market data drawn from credible online sources. P2E gaming blurs the boundaries between gaming and trading and may not yield many of the benefits promoted to consumers or otherwise conveyed through marketing and social media messaging. Particular risks include the deflationary nature of reward currencies and the asymmetric reward structures that heavily favour early investors and exploit late adopters. This paper highlights the need for greater consumer awareness of the mechanics and risks of these new gaming models. It will be important for business models to be more transparent and designed so as to encourage more equitable game outcomes, sustainable returns, a balance between intrinsic and extrinsic rewards, and protection for potentially vulnerable players.

The Effects of Social Distance and Asymmetric Reward and Punishment on Individual Cooperative Behavior in Dilemma Situations.

The behavior decisions in social dilemmas are highlighted in sociological, economic, and social psychological studies. Across two studies, the iterated prisoner’s dilemma is used as a basic paradigm to explore the effects of social distance and asymmetric reward and punishment on an individual’s cooperative behavior. Experiment 1 (N = 80) used a 2 (social distance: intimacy vs. strangeness) × 2 (symmetry of rewards: symmetric rewards vs. asymmetric rewards) within-subject design and demonstrated that when there were only two options, namely, cooperation and defection, cooperative behavior was influenced by social distance and symmetry of rewards, respectively, and the interaction was not significant. Experiment 2 (N = 80) used a 2 (social distance: intimacy vs. strangeness) × 2 (symmetry of punishment: symmetric punishment vs. asymmetric punishment) within-subject design and showed that the cooperative behavior of participants decreased when the punishment option was added, and the two levels of symmetry and asymmetry were set. Specifically, compared with the symmetric punishment group, the asymmetric punishment group was more likely to choose a defection strategy and less likely to use a punishment strategy. Moreover, there was a marginal interaction effect between social distance and symmetry of punishment, and symmetry of punishment was a significant mediator in the relationship between social distance and individual cooperation. Specifically, asymmetric punishment reduced only the cooperation rate (CR) between participants and their friends. In conclusion, in dilemma situations, asymmetric reward did not influence individual cooperative behavior at different social distances, while asymmetric punishment did, because the sense of loss was more likely to awaken an individual’s social comparison motives.

Dopamine Axons in Dorsal Striatum Encode Contralateral Visual Stimuli and Choices.

The striatum plays critical roles in visually-guided decision-making and receives dense axonal projections from midbrain dopamine neurons. However, the roles of striatal dopamine in visual decision-making are poorly understood. We trained male and female mice to perform a visual decision task with asymmetric reward payoff, and we recorded the activity of dopamine axons innervating striatum. Dopamine axons in the dorsomedial striatum (DMS) responded to contralateral visual stimuli and contralateral rewarded actions. Neural responses to contralateral stimuli could not be explained by orienting behavior such as eye movements. Moreover, these contralateral stimulus responses persisted in sessions where the animals were instructed to not move to obtain reward, further indicating that these signals are stimulus-related. Lastly, we show that DMS dopamine signals were qualitatively different from dopamine signals in the ventral striatum (VS), which responded to both ipsilateral and contralateral stimuli, conforming to canonical prediction error signaling under sensory uncertainty. Thus, during visual decisions, DMS dopamine encodes visual stimuli and rewarded actions in a lateralized fashion, and could facilitate associations between specific visual stimuli and actions.SIGNIFICANCE STATEMENT While the striatum is central to goal-directed behavior, the precise roles of its rich dopaminergic innervation in perceptual decision-making are poorly understood. We found that in a visual decision task, dopamine axons in the dorsomedial striatum (DMS) signaled stimuli presented contralaterally to the recorded hemisphere, as well as the onset of rewarded actions. Stimulus-evoked signals persisted in a no-movement task variant. We distinguish the patterns of these signals from those in the ventral striatum (VS). Our results contribute to the characterization of region-specific dopaminergic signaling in the striatum and highlight a role in stimulus-action association learning.

The interface of unidirectional rewards: Enhanced cooperation within interdependent networks

As a way to induce cooperation more effectively, positive incentives have emerged in social species forefront to negative incentives. To assess their effects in cooperation networks, we simulated a social scenario in which reward behaviors are dispensed unilaterally. To do so, we let evolve asymmetric rewards within two specular, interdependent networks, whose individuals in an upper layer have the right to reward corresponding players displaying, likewise, cooperative or defective behavior in the lower layer. With this setup, rewarded players in the lower layer can obtain a payoff equivalent to the amount of the cost of granting reward from the upper layer. Peculiarly, we find that cooperators survive for larger reward values regardless of how high the temptation to defect is. Notably, cooperators in the upper network thrive even if the temptation to defect is pretty high. By further analyzing the nature of social interactions, we find that defection is the winning strategy when rewards originated pro-socially, and that cooperation instead wins by invading the whole system when rewards originated anti-socially. These results are consistent when crosschecked to a theoretical analysis here purposely drawn up. Our work highlights the need of including complex network structures when analyzing asymmetric incentives in the evolution of cooperation, both in human and non-human species.

Frontal eye field and caudate neurons make different contributions to reward-biased perceptual decisions.

Many decisions require trade-offs between sensory evidence and internal preferences. Potential neural substrates include the frontal eye field (FEF) and caudate nucleus, but their distinct roles are not understood. Previously we showed that monkeys' decisions on a direction-discrimination task with asymmetric rewards reflected a biased accumulate-to-bound decision process (Fan et al., 2018) that was affected by caudate microstimulation (Doi et al., 2020). Here we compared single-neuron activity in FEF and caudate to each other and to accumulate-to-bound model predictions derived from behavior. Task-dependent neural modulations were similar in both regions. However, choice-selective neurons in FEF, but not caudate, encoded behaviorally derived biases in the accumulation process. Baseline activity in both regions was sensitive to reward context, but this sensitivity was not reliably associated with behavioral biases. These results imply distinct contributions of FEF and caudate neurons to reward-biased decision-making and put experimental constraints on the neural implementation of accumulation-to-bound-like computations.

Confluence of Timing and Reward Biases in Perceptual Decision-Making Dynamics.

Although the decisions of our daily lives often occur in the context of temporal and reward structures, the impact of such regularities on decision-making strategy is poorly understood. Here, to explore how temporal and reward context modulate strategy, we trained 2 male rhesus monkeys to perform a novel perceptual decision-making task with asymmetric rewards and time-varying evidence reliability. To model the choice and response time patterns, we developed a computational framework for fitting generalized drift-diffusion models, which flexibly accommodate diverse evidence accumulation strategies. We found that a dynamic urgency signal and leaky integration, in combination with two independent forms of reward biases, best capture behavior. We also tested how temporal structure influences urgency by systematically manipulating the temporal structure of sensory evidence, and found that the time course of urgency was affected by temporal context. Overall, our approach identified key components of cognitive mechanisms for incorporating temporal and reward structure into decisions.SIGNIFICANCE STATEMENT In everyday life, decisions are influenced by many factors, including reward structures and stimulus timing. While reward and timing have been characterized in isolation, ecologically valid decision-making involves a multiplicity of factors acting simultaneously. This raises questions about whether the same decision-making strategy is used when these two factors are concurrently manipulated. To address these questions, we trained rhesus monkeys to perform a novel decision-making task with both reward asymmetry and temporal uncertainty. In order to understand their strategy and hint at its neural mechanisms, we used the new generalized drift diffusion modeling framework to model both reward and timing mechanisms. We found two of each reward and timing mechanisms are necessary to explain our data.

Risk and Trouble: Adam Smith on Profit and the Protagonists of Capitalism

AbstractTo better account for deepening global inequalities, political theory could greatly enhance empirical and normative work by answering a fundamental question: What is profit? When engaging political‐economic questions, however, theorists often begin thinking from the concept of private property. This, I argue, has obscured the central role of profit as an organizing category for capitalist societies since the eighteenth century. Grounded in the dynamic and uncertain processes of production and accumulation, profit displaces the proprietary citizen and subject of natural rights and gives rise to new social protagonists, who lay claim to increasingly asymmetric rewards. In particular, the article calls attention to a dangerous contemporary sensibility, which sees profit as inherently unlimited. As a viable, robust alternative, it presents Adam Smith's idea of profit as a regular, uniform rate that acts as a productive constraint on business activity, shapes character, and safeguards against risk by widely distributing its costs.

Responses of Neurons in Lateral Intraparietal Area Depend on Stimulus-Associated Reward During Binocular Flash Suppression.

Discovering neural correlates of subjective perception and dissociating them from sensory input has fascinated neuroscientists for a long time. Bistable and multistable perception phenomena have exhibited great experimental potential to address this question. Here, we performed electrophysiological recordings from single neurons in lateral intraparietal area (LIP) of rhesus macaques during stimulus and perceptual transitions induced by binocular flash suppression (BFS). LIP neurons demonstrated transient bursts of activity after stimulus presentation and stimulus or perceptual switches but only a minority of cells demonstrated stimulus and perceptual selectivity. To enhance LIP neural selectivity, we performed a second experiment in which the competing stimuli were associated with asymmetric rewards. We found that transient and sustained activities substantially increased while the proportion of stimulus selective neurons remained approximately the same, albeit with increased selectivity magnitude. In addition, we observed mild increases in the proportion of perceptually selective neurons which also showed increase magnitude of selectivity. Importantly, the increased selectivity of cells after the reward manipulation was not directly reflecting the reward size per se but an enhancement in stimulus differentiation. Based on our results, we conjecture that LIP contributes to perceptual transitions and serves a modulatory role in perceptual selection taking into account the stimulus behavioral value.

Behavioral adjustment to asymmetric reward availability among children with and without ADHD: effects of past and current reinforcement contingencies

Altered reinforcement sensitivity is hypothesized to underlie symptoms of attention deficit hyperactivity disorder (ADHD). Here we evaluate the behavioral sensitivity of Brazilian children with and without ADHD to a change in reward availability. Forty typically developing children and 32 diagnosed with DSM-IV ADHD completed a signal-detection task in which correct discriminations between two stimuli were associated with different frequencies of reinforcement. The response alternative associated with the higher rate of reinforcement switched, without warning, after 30 rewards were delivered. The task continued until another 30 rewards were delivered. Both groups of children developed a response bias toward the initially more frequently reinforced alternative. This effect was larger in the control group. The response allocation of the two groups changed following the shift in reward availability. Over time the ADHD group developed a significant response bias toward the now more frequently reinforced alternative. In contrast, the bias of the control group stayed near zero after an initial decline following the contingency change. The overall shift in bias was similar for the two groups. The behavior of both groups of children was sensitive to the asymmetric reward distribution and to the change in reward availability. Subtle group differences in response patterns emerged, possibly reflecting differences in the time frame of reward effects and sensitivity to reward exposure.

Rewarding objects appear larger but not brighter.

Whether reward can accentuate the perception of visual objects, that is, makes them appear larger than they really are, is a long-standing and controversial question. Here, we revisit this issue with a novel two-alternative forced-choice paradigm combining asymmetric reward schedule and task reversal. In a first experiment, participants (n = 27) choose the larger of two unequally rewarded objects in some sessions and the smaller one in other sessions. Response biases toward the most rewarding object differ significantly between the reversed tasks, revealing an influence of reward on perceived sizes. In a second experiment, participants (n = 27) indicate either the brighter or darker object. In contrast with the first experiment, response biases are similar between those reversed tasks, indicating that the perceived luminance is immune to reward manipulation. Together, these results reveal that if two objects are associated with different amounts of reward, participants will perceive the more rewarded object to be slightly larger, but not brighter, than the less rewarded one.

A neuronal reward inequity signal in primate striatum.

Primates are social animals, and their survival depends on social interactions with others. Especially important for social interactions and welfare is the observation of rewards obtained by other individuals and the comparison with own reward. The fundamental social decision variable for the comparison process is reward inequity, defined by an asymmetric reward distribution among individuals. An important brain structure for coding reward inequity may be the striatum, a component of the basal ganglia involved in goal-directed behavior. Two rhesus monkeys were seated opposite each other and contacted a touch-sensitive table placed between them to obtain specific magnitudes of reward that were equally or unequally distributed among them. Response times in one of the animals demonstrated differential behavioral sensitivity to reward inequity. A group of neurons in the striatum showed distinct signals reflecting disadvantageous and advantageous reward inequity. These neuronal signals occurred irrespective of, or in conjunction with, own reward coding. These data demonstrate that striatal neurons of macaque monkeys sense the differences between other's and own reward. The neuronal activities are likely to contribute crucial reward information to neuronal mechanisms involved in social interactions.

From Honesty–Humility to fair behavior – Benevolence or a (blind) fairness norm?

Recent research has identified trait Honesty–Humility as a vital determinant of a variety of cooperative, fair, or pro-social behaviors. However, it is less clear what drives the link between Honesty–Humility and the tendency to exhibit fair behavior which we first demonstrate in a meta-analysis of prior work using the dictator game. To close this gap, we test two potential explanations, namely benevolence versus an internalized (blind) fairness norm. In a redistribution paradigm, we implemented an unfair situation in which participants and their hypothetical counterparts received asymmetric rewards for the same performance in a quiz. The asymmetry was either to the participants’ advantage or disadvantage, manipulated experimentally. Finally, participants were empowered to redistribute all rewards at will. In line with the benevolence account, individuals high in Honesty–Humility established a fair distribution if they themselves were advantaged (thus making sure they do not exploit others) whereas they refrained from rectifying the asymmetry (by taking points from the other) if they were disadvantaged. Vice versa, the pattern cannot be accommodated by the idea of a (blind) fairness norm which would have implied that individuals high in Honesty–Humility establish an equal distribution irrespective of who is advantaged.

Neuronal activity in the nucleus accumbens and hippocampus in rats during formation of seeking behavior in a radial maze.

Seeking behavior of rats in a radial maze with asymmetric reward was studied by means of synchronous recording of cell activity in the hippocampus and ventral striatum. The synchrony of cell activity in the hippocampus and nucleus accumbens was modulated by spatial position and reward; the important role in this synchronization can be played by theta rhythm. This is in line with the anatomical and physiological data on the convergence of hippocampal spatially organized positional and reward value information inputs from the amygdala and ventral segmental area to n. accumbens.

Social Perception of Organizational Practices: Asymmetric Rewards and Penalties for CSR

This study examines how social perception of corporate social responsibility (CSR) and irresponsibility (CSI) affects organizational outcomes. Drawing from the social psychology literature on stere...