Reward Coding Research Articles

Subjective neuronal coding of reward: temporal value discounting and risk

A key question in the neurobiology of reward relates to the nature of coding. Rewards are objects that are advantageous or necessary for the survival of individuals in a variety of environmental situations. Thus reward appears to depend on the individual and its environment. The question arises whether neuronal systems in humans and monkeys code reward in subjective terms, objective terms or both. The present review addresses this issue by dealing with two important reward processes, namely the individual discounting of reward value across temporal delays, and the processing of information about risky rewards that depends on individual risk attitudes. The subjective value of rewards decreases with the temporal distance to the reward. In experiments using neurophysiology and brain imaging, dopamine neurons and striatal systems discount reward value across temporal delays of a few seconds, despite unchanged objective reward value, suggesting subjective value coding. The subjective values of risky outcomes depend on the risk attitude of individual decision makers; these values decrease for risk-avoiders and increase for risk-seekers. The signal for risk and the signal for the value of risky reward covary with individual risk attitudes in regions of the human prefrontal cortex, suggesting subjective rather than objective coding of risk and risky value. These data demonstrate that important parameters of reward are coded in a subjective manner in key reward structures of the brain. However, these data do not rule out that other neurons or brain structures may code reward according to its objective value and risk.

Coding of Task Reward Value in the Dorsal Raphe Nucleus

The dorsal raphe nucleus and its serotonin-releasing neurons are thought to regulate motivation and reward-seeking. These neurons are known to be active during motivated behavior, but the underlying principles that govern their activity are unknown. Here we show that a group of dorsal raphe neurons encode behavioral tasks in a systematic manner, tracking progress toward upcoming rewards. We analyzed dorsal raphe neuron activity recorded while animals performed two reward-oriented saccade tasks. There was a strong correlation between the tonic activity level of a neuron during behavioral tasks and its encoding of reward-related cues and outcomes. Neurons that were tonically excited during the task predominantly carried positive reward signals. Neurons that were tonically inhibited during the task predominantly carried negative reward signals. Neurons that did not change their tonic activity levels during the task had weak reward signals with no tendency for a positive or negative direction. This form of correlated task and reward coding accounted for the majority of systematic variation in dorsal raphe response patterns in our tasks. A smaller component of neural activity reflected detection of reward delivery. Our data suggest that the dorsal raphe nucleus encodes participation in a behavioral task in terms of its future motivational outcomes.

The neural code of reward anticipation in human orbitofrontal cortex

An optimal choice among alternative behavioral options requires precise anticipatory representations of their possible outcomes. A fundamental question is how such anticipated outcomes are represented in the brain. Reward coding at the level of single cells in the orbitofrontal cortex (OFC) follows a more heterogeneous coding scheme than suggested by studies using functional MRI (fMRI) in humans. Using a combination of multivariate pattern classification and fMRI we show that the reward value of sensory cues can be decoded from distributed fMRI patterns in the OFC. This distributed representation is compatible with previous reports from animal electrophysiology that show that reward is encoded by different neural populations with opposing coding schemes. Importantly, the fMRI patterns representing specific values during anticipation are similar to those that emerge during the receipt of reward. Furthermore, we show that the degree of this coding similarity is related to subjects' ability to use value information to guide behavior. These findings narrow the gap between reward coding in humans and animals and corroborate the notion that value representations in OFC are independent of whether reward is anticipated or actually received.

Genetically Determined Differences in Brain Response to a Primary Food Reward: Figure 1.

Combining genetic and neuroimaging techniques may elucidate the biological underpinnings of individual differences in neurophysiology and potential vulnerabilities to disease. The TaqIA A1 variant is associated with diminished dopamine D(2) receptor density, higher body mass, and food reinforcement. It also moderates the relationship between brain response to food and future weight gain. This suggests that the polymorphism is associated with a fundamental difference in the neurophysiology of food that may predispose toward overeating. An alternative possibility is that factors, such as impulsivity, eating style, reward drive, and perception, which may covary with the polymorphism, influence reward coding and eating behavior. To distinguish between these alternatives, we used functional magnetic resonance imaging to measure neural response to the ingestion of palatable and caloric milkshakes in healthy subjects with (A1+; n = 13) and without (A1-; n = 13) the TaqIA A1 allele. The groups were selected from a larger group to be matched for linked individual factors such as age, gender, education, body mass index, impulsivity, eating style, and perceptual responses to the milkshake. We demonstrate an interaction between genotype (A1+ vs A1-) and stimulus (milkshake vs a tasteless/odorless baseline) in the midbrain, thalamus, and orbital frontal cortex; whereas A1- shows increased responses to milkshake, A1+ shows decreased responses to milkshake relative to baseline. This interaction occurs despite similar ratings of milkshake pleasantness, intensity, and familiarity. We therefore conclude that there is a specific association between the TaqIA A1 polymorphism and brain response during ingestion of a palatable food.

Adaptation of Reward Sensitivity in Orbitofrontal Neurons

Animals depend on a large variety of rewards but their brains have a limited dynamic coding range. When rewards are uncertain, neuronal coding needs to cover a wide range of possible rewards. However, when reward is likely to occur within a specific range, focusing the sensitivity on the predicted range would optimize the discrimination of small reward differences. One way to overcome the trade-off between wide coverage and optimal discrimination is to adapt reward sensitivity dynamically to the available rewards. We investigated how changes in reward distribution influenced the coding of reward in the orbitofrontal cortex. Animals performed an oculomotor task in which a fixation cue predicted the SD of the probability distribution of juice volumes, while the expected mean volume was kept constant. A subsequent cue specified the exact juice volume obtained for a correct saccade response. Population responses of orbitofrontal neurons that reflected the predicted juice volume showed adaptation to the reward distribution. Statistical tests on individual responses revealed that a quarter of value-coding neurons shifted the reward sensitivity slope significantly between two reward distributions, whereas the remaining neurons showed insignificant change or lack of adaptation. Adaptations became more prominent when reward distributions changed less frequently, indicating time constraints for assessing reward distributions and adjusting neuronal sensitivity. The observed neuronal adaptation would optimize discrimination and contribute to the efficient coding of a large variety of potential rewards by neurons with limited dynamic range.

Segregated and Integrated Coding of Reward and Punishment in the Cingulate Cortex

Reward and punishment have opposite affective value but are both processed by the cingulate cortex. However, it is unclear whether the positive and negative affective values of monetary reward and punishment are processed by separate or common subregions of the cingulate cortex. We performed a functional magnetic resonance imaging study using a free-choice task and compared cingulate activations for different levels of monetary gain and loss. Gain-specific activation (increasing activation for increasing gain, but no activation change in relation to loss) occurred mainly in the anterior part of the anterior cingulate and in the posterior cingulate cortex. Conversely, loss-specific activation (increasing activation for increasing loss, but no activation change in relation to gain) occurred between these areas, in the middle and posterior part of the anterior cingulate. Integrated coding of gain and loss (increasing activation throughout the full range, from biggest loss to biggest gain) occurred in the dorsal part of the anterior cingulate, at the border with the medial prefrontal cortex. Finally, unspecific activation increases to both gains and losses (increasing activation to increasing gains and increasing losses, possibly reflecting attention) occurred in dorsal and middle regions of the cingulate cortex. Together, these results suggest separate and common coding of monetary reward and punishment in distinct subregions of the cingulate cortex. Further meta-analysis suggested that the presently found reward- and punishment-specific areas overlapped with those processing positive and negative emotions, respectively.

Reward coding by the primate dorsal raphé neurons is context dependent

This review focuses on nicotinic–serotonergic interactions in the central nervous system (CNS). Nicotine increases 5-hydroxytryptamine (5-HT) release in the cortex, striatum, hippocampus, dorsal raphé nucleus (DRN), hypothalamus, and spinal cord. As yet, there is little firm evidence for nicotinic receptors on serotonergic terminals and thus nicotine's effects on 5-HT may not necessarily be directly mediated, but there is strong evidence that the 5-HT tone plays a permissive role in nicotine's effects. The effects in the cortex, hippocampus, and DRN involve stimulation of 5-HT1A receptors, and in the striatum, 5-HT3 receptors. The 5-HT1A receptors in the DRN play a role in mediating the anxiolytic effects of nicotine and the 5-HT1A receptors in the dorsal hippocampus and lateral septum mediate its anxiogenic effects. The increased startle and anxiety during nicotine withdrawal is mediated by 5-HT1A and 5-HT3 receptors. The locomotor stimulant effect of acute nicotine is mediated by 5-HT1A receptors and 5-HT2 receptors may play a role in the expression of a sensitised response after chronic nicotine treatment. Unfortunately, the role of 5-HT1A receptors in mediating nicotine seeking has not yet been investigated and would seem an important area for future research. There is also evidence for nicotinic–serotonergic interactions in the acquisition of the water maze, passive avoidance, and impulsivity in the five-choice serial reaction task.

The Rodent Orbitofrontal Cortex Gets Time and Direction

The orbitofrontal cortex (OFC) helps direct decision making through its flexible coding of reward and economic value. In this issue of Neuron, papers by Roesch et al. and Feierstein et al. demonstrate the importance of temporal and spatial features to processing in the rodent OFC.

Ventral pallidal neurons code incentive motivation: amplification by mesolimbic sensitization and amphetamine

Neurons in ventral pallidum fire to reward and its predictive cues. We tested mesolimbic activation effects on neural reward coding. Rats learned that a Pavlovian conditioned stimulus (CS+1 tone) predicted a second conditioned stimulus (CS+2 feeder click) followed by an unconditioned stimulus (UCS sucrose reward). Some rats were sensitized to amphetamine after training. Electrophysiological activity of ventral pallidal neurons to stimuli was later recorded under the influence of vehicle or acute amphetamine injection. Both sensitization and acute amphetamine increased ventral pallidum firing at CS+2 (population code and rate code). There were no changes at CS+1 and minimal changes to UCS. With a new 'Profile Analysis', we show that mesolimbic activation by sensitization/amphetamine incrementally shifted neuronal firing profiles away from prediction signal coding (maximal at CS+1) and toward incentive coding (maximal at CS+2), without changing hedonic impact coding (maximal at UCS). This pattern suggests mesolimbic activation specifically amplifies a motivational transform of CS+ predictive information into incentive salience coded by ventral pallidal neurons. Our results support incentive-sensitization predictions and suggest why cues temporally proximal to drug presentation may precipitate cue-triggered relapse in human addicts.