Orbitofrontal Neurons Research Articles

Risk Prediction Error Coding in Orbitofrontal Neurons

Risk is a ubiquitous feature of life. It plays an important role in economic decisions by affecting subjective reward value. Informed decisions require accurate risk information for each choice option. However, risk is often not constant but changes dynamically in the environment. Therefore, risk information should be updated to the current risk level. Potential mechanisms involve error-driven updating, whereby differences between current and predicted risk levels (risk prediction errors) are used to obtain currently accurate risk predictions. As a major reward structure, the orbitofrontal cortex is involved in coding key reward parameters such as reward value and risk. In this study, monkeys viewed different visual stimuli indicating specific levels of risk that deviated from the overall risk predicted by a common earlier stimulus. A group of orbitofrontal neurons displayed a risk signal that tracked the discrepancy between current and predicted risk. Such neuronal signals may be involved in the updating of risk information.

Neural Estimates of Imagined Outcomes in the Orbitofrontal Cortex Drive Behavior and Learning

Imagination, defined as the ability to interpret reality in ways that diverge from past experience, is fundamental to adaptive behavior. This can be seen at a simple level in our capacity to predict novel outcomes in new situations. The ability to anticipate outcomes never before received can also influence learning if those imagined outcomes are not received. The orbitofrontal cortex is a key candidate for where the process of imagining likely outcomes occurs; however, its precise role in generating these estimates and applying them to learning remain open questions. Here we address these questions by showing that single-unit activity in the orbitofrontal cortex reflects novel outcome estimates. The strength of these neural correlates predicted both behavior and learning, learning that was abolished by temporally specific inhibition of orbitofrontal neurons. These results are consistent with the proposal that the orbitofrontal cortex is critical for integrating information to imagine future outcomes.

Risk-Responsive Orbitofrontal Neurons Track Acquired Salience

Decision making is impacted by uncertainty and risk (i.e., variance). Activity in the orbitofrontal cortex, an area implicated in decision making, covaries with these quantities. However, this activity could reflect the heightened salience of situations in which multiple outcomes-reward and reward omission-are expected. To resolve these accounts, rats were trained to respond to cues predicting 100%, 67%, 33%, or 0% reward. Consistent with prior reports, some orbitofrontal neurons fired differently in anticipation of uncertain (33% and 67%) versus certain (100% and 0%) reward. However, over 90% of these neurons also fired differently prior to 100% versus 0% reward (or baseline) or prior to 33% versus 67% reward. These responses are inconsistent with risk but fit well with the representation of acquired salience linked to the sum of cue-outcome and cue-no-outcome associative strengths. These results expand our understanding of how the orbitofrontal cortex might regulate learning and behavior.

NMDA Receptors Control Cue-Outcome Selectivity and Plasticity of Orbitofrontal Firing Patterns during Associative Stimulus-Reward Learning

Neural activity in orbitofrontal cortex has been linked to flexible representations of stimulus-outcome associations. Such value representations are known to emerge with learning, but the neural mechanisms supporting this phenomenon are not well understood. Here, we provide evidence for a causal role for NMDA receptors (NMDARs) in mediating spike pattern discriminability, neural plasticity, and rhythmic synchronization in relation to evaluative stimulus processing and decision making. Using tetrodes, single-unit spike trains and local field potentials were recorded during local, unilateral perfusion of an NMDAR blocker in rat OFC. In the absence of behavioral effects, NMDAR blockade severely hampered outcome-selective spike pattern formation to olfactory cues, relative to control perfusions. Moreover, NMDAR blockade shifted local rhythmicsynchronization to higher frequencies and degradedits linkage to stimulus-outcome selective coding.These results demonstrate the importance of NMDARs for cue-outcome associative coding in OFC during learning and illustrate how NMDAR blockade disrupts network dynamics.

Willingness to Wait and Altered Encoding of Time-Discounted Reward in the Orbitofrontal Cortex with Normal Aging

Normal aging has been associated with cognitive changes, including shifts in responding for time-discounted rewards. The orbitofrontal cortex, an area previously associated with aging-related cognitive changes, is critical for normal discounting. Previously we have shown in a choice task that rats prefer immediate over delayed reward and that neural representations of delayed reward in orbitofrontal cortex were attenuated, whereas immediate reward elicited strong responses. Changes in choice performance were correlated with changes in firing rate in orbitofrontal neurons, suggesting that these reward representations were critical to the rats' ability to wait for reward. Here we asked whether age-dependent changes in discounting behavior were related to changes in the representation of delayed reward in the orbitofrontal cortex. Young (3-6 months) and aged (22-26 months) rats were trained on the same discounting paradigm used previously. We found that aged rats showed less sensitivity to increasing delay preceding reward delivery, shifting behavior away from the delayed reward more slowly than younger rats. This sensitivity was specific to delay, since choice performance did not differ between the two groups when delay was held constant and reward size varied. Aged rats exhibited a corresponding increase in the prevalence of neurons that fired more strongly for delayed reward. Again this change was specific to delay; there was no change in encoding of different-sized rewards. These results suggest that natural aging results in altered representations of reward in orbitofrontal cortex. These changes may relate to the increased ability to delay gratification and reduced impulsivity associated with aging.

Does the orbitofrontal cortex signal value?

The orbitofrontal cortex (OFC) has long been implicated in associative learning. Early work by Mishkin and Rolls showed that the OFC was critical for rapid changes in learned behavior, a role that was reflected in the encoding of associative information by orbitofrontal neurons. Over the years, new data-particularly neurophysiological data-have increasingly emphasized the OFC in signaling actual value. These signals have been reported to vary according to internal preferences and judgments and to even be completely independent of the sensory qualities of predictive cues, the actual rewards, and the responses required to obtain them. At the same time, increasingly sophisticated behavioral studies have shown that the OFC is often unnecessary for simple value-based behavior and instead seems critical when information about specific outcomes must be used to guide behavior and learning. Here, we review these data and suggest a theory that potentially reconciles these two ideas, value versus specific outcomes, and bodies of work on the OFC.

Representations of appetitive and aversive information in the primate orbitofrontal cortex

Individuals weigh information about both rewarding and aversive stimuli to make adaptive decisions. Most studies of the orbitofrontal cortex (OFC), an area where appetitive and aversive neural subsystems might interact, have focused only on reward. Using a classical conditioning task where novel stimuli are paired with a reward or an aversive air puff, we discovered that two groups of orbitofrontal neurons respond preferentially to conditioned stimuli associated with rewarding and aversive outcomes; however, information about appetitive and aversive stimuli converges on individual neurons from both populations. Therefore, neurons in the OFC might participate in appetitive and aversive networks that track the motivational significance of stimuli even when they vary in valence and sensory modality. Further, we show that these networks, which also extend to the amygdala, exhibit different rates of change during reversal learning. Thus, although both networks represent appetitive and aversive associations, their distinct temporal dynamics might indicate different roles in learning processes.

Neuronal signals for reward risk in frontal cortex

Rewards can be viewed as probability distributions of reward values. Besides expected (mean) value, a key parameter of such distributions is variance (or standard deviation), which constitutes a measure of risk. Single neurons in orbitofrontal cortex signal risk mostly separately from value. Comparable risk signals in human frontal cortex reflect risk attitudes of individual participants. Subjective outcome value constitutes the primary economic decision variable. The terms risk avoidance and risk taking suggest that risk affects subjective outcome value, a basic tenet of economic decision theories. Correspondingly, risk reduces neuronal value signals in frontal cortex of human risk avoiders and enhances value signals in risk takers. Behavioral contrast effects and reference-dependent valuation demonstrate flexible reward valuation. As a potential correlate, value signals in orbitofrontal neurons adjust reward discrimination to variance (risk). These neurophysiological mechanisms of reward risk on economic decisions inform and validate theories of economic decision making under uncertainty.

Coding of Reward Risk by Orbitofrontal Neurons Is Mostly Distinct from Coding of Reward Value

Risky decision-making is altered in humans and animals with damage to the orbitofrontal cortex. However, the cellular function of the intact orbitofrontal cortex in processing information relevant for risky decisions is unknown. We recorded responses of single orbitofrontal neurons while monkeys viewed visual cues representing the key decision parameters, reward risk and value. Risk was defined as the mathematical variance of binary symmetric probability distributions of reward magnitudes; value was defined as non-risky reward magnitude. Monkeys displayed graded behavioral preferences for risky outcomes, as they did for value. A population of orbitofrontal neurons showed a distinctive risk signal: their cues and reward responses covaried monotonically with the variance of the different reward distributions without monotonically coding reward value. Furthermore, a small but statistically significant fraction of risk responses also coded reward value. These risk signals may provide physiological correlates for the role of the orbitofrontal cortex in risk processing.

Ventromedial and Orbital Prefrontal Neurons Differentially Encode Internally and Externally Driven Motivational Values in Monkeys

The value of events that predict future rewards, thereby driving behavior, is sensitive to information arising from external (environmental) and internal factors. The ventral prefrontal cortex, an anatomically heterogeneous area, has information related to this value. We designed experiments to compare the contribution of two distinct subregions, orbital and ventromedial, of the ventral prefrontal cortex to the encoding of internal and external factors controlling the perceived motivational value. We recorded the activity of single neurons in both regions in monkeys while manipulating internal and external factors that should affect the perceived value of task events. Neurons in both regions encoded the value of task events, with orbitofrontal neurons being more sensitive to external factors such as visual cues and ventromedial neurons being more sensitive to internal factors such as satiety. Thus, the orbitofrontal cortex emphasizes signals for evaluating environment-centered, externally driven motivational processes, whereas ventromedial prefrontal cortex emphasizes signals more suited for subject-centered, internally driven motivational processes.

Theta-Band Phase Locking of Orbitofrontal Neurons during Reward Expectancy

The expectancy of a rewarding outcome following actions and cues is coded by a network of brain structures including the orbitofrontal cortex. Thus far, predicted reward was considered to be coded by time-averaged spike rates of neurons. However, besides firing rate, the precise timing of action potentials in relation to ongoing oscillations in local field potentials is thought to be of importance for effective communication between brain areas. We performed multineuron and field potential recordings in orbitofrontal cortex of rats performing olfactory discrimination learning to study the temporal structure of coding predictive of outcome. After associative learning, field potentials were marked by theta oscillations, both in advance and during delivery of reward. Orbitofrontal neurons, especially those coding information about upcoming reward with their firing rate, phase locked to these oscillations in anticipation of reward. When established associations were reversed, phase locking collapsed in the anticipatory task phase, but returned when reward became predictable again after relearning. Behaviorally, the outcome anticipation phase was marked by licking responses, but the frequency of lick responses was dissociated from the strength of theta-band phase locking. The strength of theta-band phase locking by orbitofrontal neurons robustly follows the dynamics of associative learning as measured by behavior and correlates with the rat's current outcome expectancy. Theta-band phase locking may facilitate communication of outcome-related information between reward-related brain areas and offers a novel mechanism for coding value signals during reinforcement learning.

Adaptation of Reward Sensitivity in Orbitofrontal Neurons

Animals depend on a large variety of rewards but their brains have a limited dynamic coding range. When rewards are uncertain, neuronal coding needs to cover a wide range of possible rewards. However, when reward is likely to occur within a specific range, focusing the sensitivity on the predicted range would optimize the discrimination of small reward differences. One way to overcome the trade-off between wide coverage and optimal discrimination is to adapt reward sensitivity dynamically to the available rewards. We investigated how changes in reward distribution influenced the coding of reward in the orbitofrontal cortex. Animals performed an oculomotor task in which a fixation cue predicted the SD of the probability distribution of juice volumes, while the expected mean volume was kept constant. A subsequent cue specified the exact juice volume obtained for a correct saccade response. Population responses of orbitofrontal neurons that reflected the predicted juice volume showed adaptation to the reward distribution. Statistical tests on individual responses revealed that a quarter of value-coding neurons shifted the reward sensitivity slope significantly between two reward distributions, whereas the remaining neurons showed insignificant change or lack of adaptation. Adaptations became more prominent when reward distributions changed less frequently, indicating time constraints for assessing reward distributions and adjusting neuronal sensitivity. The observed neuronal adaptation would optimize discrimination and contribute to the efficient coding of a large variety of potential rewards by neurons with limited dynamic range.

Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat.

The orbitofrontal cortex (OFC) has been implicated in decision-making under uncertainty, but it is unknown how information about the probability or uncertainty of future reward is coded by single orbitofrontal neurons and ensembles. We recorded neuronal ensembles in rat OFC during an olfactory discrimination task in which different odor stimuli predicted different reward probabilities. Single-unit firing patterns correlated to the expected reward probability primarily within an immobile waiting period before reward delivery but also when the rat executed movements toward the reward site. During these pre-reward periods, a subset of OFC neurons was sensitive to differences in probability but only very rarely discriminated on the basis of reward uncertainty. In the reward period, neurons responded during presentation or omission of reward or during both types of outcome. At the population level, neurons were characterized by a wide divergence in firing-rate variability attributable to expected probability. A population analysis using template matching as reconstruction method indicated that OFC generates a distributed representation of reward probability with a weak dependence on neuronal group size. The analysis furthermore confirmed that predictive information coded by OFC populations was quantitatively related to reward probability, but not to uncertainty.

Learning Substrates in the Primate Prefrontal Cortex and Striatum: Sustained Activity Related to Successful Actions

Learning from experience requires knowing whether a past action resulted in a desired outcome. The prefrontal cortex and basal ganglia are thought to play key roles in such learning of arbitrary stimulus-response associations. Previous studies have found neural activity in these areas, similar to dopaminergic neurons' signals, that transiently reflect whether a response is correct or incorrect. However, it is unclear how this transient activity, which fades in under a second, influences actions that occur much later. Here, we report that single neurons in both areas show sustained, persistent outcome-related responses. Moreover, single behavioral outcomes influence future neural activity and behavior: behavioral responses are more often correct and single neurons more accurately discriminate between the possible responses when the previous response was correct. These long-lasting signals about trial outcome provide a way to link one action to the next and may allow reward signals to be combined over time to implement successful learning.

Encoding of Gustatory Working Memory by Orbitofrontal Neurons

The content model regarding the functional organization of working memory in prefrontal cortex (PFC) states that different PFC areas encode different types of information in working memory depending on their afferent connections with other brain areas. Previous studies that tested this model focused on visual, auditory and somatosensory information. However, posterior areas processing this information project to widespread and overlapping regions of lateral PFC, making it difficult to establish the veracity of the model. In contrast, gustatory information enters PFC via orbitofrontal cortex (OFC), and so the content model would argue that OFC should be responsible for maintaining gustatory information in working memory. To test this, we recorded the activity of single neurons throughout PFC and gustatory cortex (GUS) from two subjects while they performed a gustatory delayed-match-to-sample task with intervening gustatory distraction. Neurons that encoded the identity of the gustatory stimulus across the delay, consistent with a role in gustatory working memory, were most prevalent in OFC and GUS compared with dorsolateral PFC and ventrolateral PFC. Gustatory information in OFC was more resilient to intervening distraction, paralleling previous findings regarding visual working memory processes in PFC and posterior sensory cortex. Our findings provide support for the content model of working memory organization. Maintaining gustatory information may be one aspect of a wider function for OFC in reward working memory that could contribute to its role in decision-making.

Beacon-like/ubiquitin-5-like immunoreactivity is highly expressed in human hypothalamus and increased in haloperidol-treated schizophrenics and a rat model of schizophrenia

The beacon gene is involved in the regulation of energy metabolism, food intake, and obesity. We localized its gene product, beacon-/ubiquitin 5-like immunoreactivity in brains of normal-weight, non-psychotic individuals, adipose (BMI over 32), non-psychotic individuals, and haloperidol-treated schizophrenics. The protein was found to be highly expressed in many neurons of the paraventricular and supraoptic hypothalamic nuclei. Besides, it was detected in neurons of other hypothalamic areas (suprachiasmatic, arcuate, and ventromedial nuclei) as well as outside the hypothalamus (Nuc. basalis Meynert, thalamus, hippocampus, and some neocortical areas). A morphometric analysis of beacon-immunoreactive hypothalamic and neocortical neurons revealed that compared to normal-weight controls in haloperidol-treated schizophrenics, there was a significant increase of protein-expressing supraoptic, paraventricular, and orbitofrontal neurons. However, a significant increase in beacon-expressing supraoptic neurons was also seen in adipose, non-psychotic individuals in comparison with normal-weight controls. Haloperidol at different doses has no effect on beacon expression in SHSY5Y neuroblastoma cells, which makes the assumption unlikely that haloperidol per se is responsible for the increased neuronal expression of the peptide in schizophrenics. In rats with a neonatal lesion of the ventral hippocampus (a widely used animal model of schizophrenia), we found an increased neuronal expression of beacon in the paraventricular and supraoptic nuclei. We suppose that elevated hypothalamic expression of beacon-like protein in non-obese schizophrenics is not primarily related to metabolic alterations, but to a certain role in schizophrenia, which is possibly unrelated to aspects of weight gain and obesity. The latter assumption finds some support by data obtained in rats with ventral hippocampus lesion.

Dynamic Changes in Representations of Preceding and Upcoming Reward in Monkey Orbitofrontal Cortex

We investigated how orbitofrontal cortex (OFC) contributes to adaptability in the face of changing reward contingencies by examining how reward representations in monkey orbitofrontal neurons change during a visually cued, multi-trial reward schedule task. A large proportion of orbitofrontal neurons were sensitive to events in this task (69/80 neurons in the valid and 48/58 neurons in the random cue context). Neuronal activity depended upon preceding reward, upcoming reward, reward delivery, and schedule state. Preceding reward-dependent activity occurred in both the valid and random cue contexts, whereas upcoming reward-dependent activity was observed only in the valid context. A greater proportion of neurons encoded preceding reward in the random than the valid cue context. The proportion of neurons with preceding reward-dependent activity declined as each trial progressed, whereas the proportion encoding upcoming reward increased. Reward information was represented by ensembles of neurons, the composition of which changed with task context and time. Overall, neuronal activity in OFC adapted to reflect the importance of different types of reward information in different contexts and time periods. This contextual and temporal adaptability is one hallmark of neurons participating in executive functions.

Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making.

Recent evidence has linked exposure to addictive drugs to an inability to employ information about adverse consequences, or outcomes, to control behavior. For instance, addicts and drug-experienced animals fail to adapt their behavior to avoid adverse outcomes in gambling and reversal tasks or after changes in the value of expected rewards. These deficits are similar to those caused by damage to the orbitofrontal cortex, suggesting that addictive drugs may cause long-lasting changes in the representation of outcome associations in a circuit that includes the orbitofrontal cortex. Here we test this hypothesis by recording from orbitofrontal neurons in a discrimination task in rats previously exposed to cocaine (30 mg/kg i.p. for 14 days). We found that orbitofrontal neurons recorded in cocaine-experienced rats failed to signal the adverse outcome at the time a decision was made in the task. The loss of this signal was associated with abnormal changes in response latencies on aversive trials. Furthermore, upon reversal of the cue-outcome associations, orbitofrontal neurons in cocaine-treated rats with enduring reversal impairments failed to reverse their cue-selectivity, while orbitofrontal neurons in cocaine-treated rats with normal performance showed an increase in the plasticity of cue-selective firing after reversal. These results provide direct neurophysiological evidence that exposure to cocaine can cause behaviorally relevant changes in the processing of associative information in a circuit that includes the orbitofrontal cortex.

Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation

We monitored single-neuron activity in the orbitofrontal cortex of rats performing a time-discounting task in which the spatial location of the reward predicted whether the delay preceding reward delivery would be short or long. We found that rewards delivered after a short delay elicited a stronger neuronal response than those delivered after a long delay in most neurons. Activity in these neurons was not influenced by reward size when delays were held constant. This was also true for a minority of neurons that exhibited sustained increases in firing in anticipation of delayed reward. Thus, encoding of time-discounted rewards in orbitofrontal cortex is independent of the encoding of absolute reward value. These results are contrary to the proposal that orbitofrontal neurons signal the value of delayed rewards in a common currency and instead suggest alternative proposals for the role this region plays in guiding responses for delayed versus immediate rewards.

Reward-period Activity in Primate Dorsolateral Prefrontal and Orbitofrontal Neurons Is Affected by Reward Schedules

Abstract Reward-period activity observed in the dorsolateral prefrontal cortex (DLPFC) and the orbitofrontal cortex (OFC) is thought to represent the detection of reward delivery. To investigate whether this activity plays the same role in these areas, we examined this activity under different reward schedules and whether the reward schedule has similar effects on this activity in each of these areas. A monkey performed an oculomotor delayed-response (ODR) task under two reward schedules. In the ODR-1 schedule, the monkey received a large amount of reward only after four successful trials, whereas in the ODR-2 schedule, it received a small amount of reward after every successful trial. Although reward-period activity was observed in both areas, more neurons exhibited this activity in the OFC. Reward-period activity was modulated by the proximity to reward delivery in both areas and this feature was observed more frequently in the OFC. The onset time of this activity also gradually advanced depending on the proximity to reward delivery. Moreover, many OFC neurons with this activity responded to free reward delivery. These results indicate that reward-period activity in the OFC represents the detection of reward delivery and that the gradual change in the magnitude and the onset time of this activity represents the expectation of reward delivery. Similar features of reward-period activity were observed in DLPFC neurons, although a significant number of DLPFC neurons did not respond to free reward delivery and no advance was observed in the onset time of this activity. These results suggest that reward-period activity in the DLPFC participates in whether or not correct performance was achieved. Thus, although similar reward-period activity was observed in both areas, the activity in the OFC represents the detection of reward delivery and is affected by the monkey's motivational state, whereas that in the DLPFC seems to participate in monitoring whether or not the necessary performance is achieved.