Changes In Contingencies Research Articles

Chronic Cocaine Self-Administration in Rhesus Monkeys: Impact on Associative Learning, Cognitive Control, and Working Memory

Cocaine users display a wide range of cognitive impairments. Because treatment outcome is dependent on baseline cognitive ability, it is clinically important to understand the underlying neurobiology of these deficits. Therefore, it is crucial to determine whether cocaine exposure by itself is an etiological factor and, if so, to determine the overall nature of cognitive deficits associated with cocaine use. This will help to guide therapeutic approaches that address cognitive components of cocaine use to improve treatment outcome. We used rhesus monkeys in a longitudinal study in which 14 animals were characterized before assignment to matched control (n = 6) and cocaine self-administration (n = 8) groups. Self-administration took place on 4 consecutive days/week over 9 months, with a maximum (and typical) daily cumulative intake of 3.0 mg/kg. Weekly cognitive assessments (total of 36) were conducted after a 72 h drug-free period. We used a stimulus discrimination task with reversal to evaluate associative learning and the cognitive control/flexibility needed to adapt to changes in reward contingencies. After extended self-administration, initial accuracy on the stimulus discrimination indicated intact associative learning. However, animals were impaired at maintaining high levels of accuracy needed to reach criterion and initiate the reversal. Increasing the reward contrast between stimuli permitted evaluation of reversal performance and revealed striking deficits in the cocaine group. Impairments in visual working memory were also observed using a delayed match-to-sample task. These results suggest a combination of generalized, possibly attentional, impairments, along with a more specific cognitive control impairment implicating orbitofrontal cortex dysfunction.

Blockade of NMDA GluN2B receptors selectively impairs behavioral flexibility but not initial discrimination learning

Behavioral flexibility is the ability to adjust behavior when environmental contingencies change and is compromised in disease states such as schizophrenia, attention deficit hyperactivity disorder, and following damage to the prefrontal cortex. The present study investigated the contribution of N-methyl-D-aspartate GluN2B receptor subunits in the initial learning and in shifting between or within discrimination strategies (i.e., strategy set-shifting and reversal learning), using tasks conducted in operant chambers. Strategy set-shifting required rats initially to learn a visual-cue discrimination (day 1) and on day 2, shift to using an egocentric spatial response strategy to obtain reward. For reversal learning, rats were trained on a response discrimination on day 1 and then required to select the opposite lever on day 2. Blockade of GluN2B receptors with systemic administration of Ro25-6981 on day 1 did not impair initial acquisition of either a response or visual-cue discrimination nor did these treatments affect performance of strategy or reversal shifts on day 2. However, administration of Ro25-6981 prior to a set-shift or reversal on day 2 significantly impaired performance on both tasks, inducing a selective increase in perseverative errors, indicative of a disruption the ability to suppress a previously acquired prepotent response. These data suggest that systemic blockade of GluN2B receptors Ro25-6981 does not appear to interfere with the initial acquisition of simple visual or response discriminations. However, these receptors do appear to play a central and selective role in the modification of previously acquired strategies or stimulus-reward associations, facilitating behavioral inhibition so that novel patterns of behavior may emerge.

P02-358 - Prediction error signal correlates with fluid intelligenceand dopamine synthesis across the lifespan

IntroductionFluid intelligence expresses the capacity for interpretation of novel stimuli and flexible behavioral adaptation to such cues. Phasic dopamine firing closely matches a temporal difference prediction error (PE) signal important for learning and rapid behavioral adaptation. Both fluid intelligence and dopaminergic neurotransmission decline with age. So far, no study investigated the relationship between fluid IQ, PE signal and direct measures of dopaminergic neurotransmission. Here we used a multimodal imaging approach that combines positron emission tomography and functional magnetic resonance imaging.MethodsA group of healthy controls was investigated with both 6-[18F]FluoroDOPA PET and functional MRI with a probabilistic reversal task. The task required a constant behavioral adaptation to changes in reward contingencies, while choosing between two abstract stimuli. A reinforcement learning algorithm was used to compute a trial-by-trial prediction error, which was the used as a regressor in the fMRI data analysis with SPM8.ResultsThe prediction error signal was associated with functional activation in the basal ganglia including the ventral striatum and putamen. Fluid intelligence was associated with the PE signal in the ventral striatum, which correlated with age-related changes in dopamine synthesis capacity in the prefrontal cortex.ConclusionThese findings provide insight into the role of age-related changes in dopaminergic neurotransmission on behavioral adaptation. The multimodal imaging approach allows the characterization of interactions between dopamine metabolism and learning-related neuronal activation and may thus be a useful tool to clarify mechanisms underlying learning and plasticity in old age, which are crucial to our understanding of successful aging.

Human reversal learning under conditions of certain versus uncertain outcomes

Reversal learning tasks assess behavioral flexibility by requiring subjects to switch from one learned response choice to a different response choice when task contingencies change. This requires both the processing of negative feedback once a learned response is no longer reinforced, and the capacity for flexible response selection. In 2-choice reversal learning tasks, subjects switch between only two responses. Multiple choice reversal learning is qualitatively different in that at reversal, it requires subjects to respond to non-reinforcement of a learned response by selecting a new response from among several alternatives that have uncertain consequences. While activity in brain regions responsible for processing unexpected negative feedback is known to increase in relation to the hedonic value of the reward itself, it is not known whether the uncertainty of reinforcement for future response choices also modulates these responses. In an fMRI study, 15 participants performed 2- and 4-choice reversal learning tasks. Upon reversal in both tasks, activation was observed in brain regions associated with processing changing reinforcement contingencies (midbrain, ventral striatum, insula), as well as in neocortical regions that support cognitive control and behavioral planning (prefrontal, premotor, posterior parietal, and anterior cingulate cortices). Activation in both systems was greater in the 4- than in the 2-choice task. Therefore, reinforcement uncertainty for future responses enhanced activity in brain systems that process performance feedback, as well as in areas supporting behavioral planning of future response choices. A mutually facilitative integration of responses in motivational and cognitive brain systems might enhance behavioral flexibility and decision making in conditions for which outcomes for future response choices are uncertain.

Differential contributions of the primate ventrolateral prefrontal and orbitofrontal cortex to serial reversal learning.

The discrimination reversal paradigm is commonly used to measure a subject's ability to adapt their behavior according to changes in stimulus-reward contingencies. Human functional neuroimaging studies have demonstrated activations in the lateral orbitofrontal cortex (OFC) and the inferior frontal gyrus (IFG) in subjects performing this task. Excitotoxic lesions of analogous regions in marmosets have revealed, however, that although the OFC is indeed critical for reversal learning, ventrolateral prefrontal cortex (VLPFC) (analogous to IFG) is not, contributing instead to higher order processing, such as that required in attentional set-shifting and strategy transfer. One major difference between the marmoset and human studies has been the level of training subjects received in reversal learning, being far greater in the latter. Since exposure to repeated contingency changes, as occurs in serial reversal learning, is likely to trigger the development of higher order, rule-based strategies, we hypothesized a critical role of the marmoset VLPFC in performance of a serial reversal learning paradigm. After extensive training in reversal learning, marmosets received an excitotoxic lesion of the VLPFC, OFC, or a sham control procedure. In agreement with our prediction, postsurgery, VLPFC lesioned animals were impaired in performing a series of discrimination reversals, but only when novel visual stimuli were introduced. In contrast, OFC lesioned animals were impaired regardless of whether the visual stimuli were the same or different from those used during presurgery training. Together, these data demonstrate the heterogeneous but interrelated involvement of primate OFC and VLPFC in the performance of serial reversal learning.

Parsing decision making processes in prefrontal cortex: Response inhibition, overcoming learned avoidance, and reversal learning

Reversal learning refers to the ability to inhibit or switch responding to an object when the object-reward contingency changes. Deficits in this process are related to social abnormalities, impulsiveness, and a number of psychiatric disorders. A range of neural regions play a role in this process, including dorsolateral prefrontal cortex (dlPFC), dorsomedial prefrontal cortex (dmPFC), and inferior frontal gyrus (IFG). However, determining the specific functional contribution of each region has proved difficult, in part because reversal learning involves multiple cognitive subprocesses such as error detection, inhibiting responding to formerly rewarded stimuli, and overcoming avoidance of previously punished stimuli. We used fMRI and an experimental task adapted from a recent neurochemical study in marmosets to parse neural responding to subprocesses of reversal learning during choice and feedback trial components. Error-feedback processing was associated with increased activity in dmPFC, dlPFC, and IFG whether participants were overcoming avoidance, inhibiting responding, or performing classic response reversal. Reduced activity in medial prefrontal cortex (mPFC) was associated with error-feedback processing for response inhibition but not overcoming avoidance. Conversely, there was significantly greater activity in anterior dmPFC during error-feedback processing in overcoming avoidance compared to response inhibition. A conjunction analysis confirmed that a striking overlap in activity was observed across the three conditions in IFG, dlPFC, and dmPFC. The results are consistent with conceptualizations of IFG function that emphasize modulating stimulus–response maps rather than purely response inhibition. The approach has implications for models of prefrontal function and neurocognitive perspectives on a range of behavioural abnormalities associated with impairments in decision making.

Fortune and reversals of fortune in visual search: Reward contingencies for pop-out targets affect search efficiency and target repetition effects

Rewards have long been known to modulate overt behavior. But their possible impact on attentional and perceptual processes is less well documented. Here, we study whether the (changeable) reward level associated with two different pop-out targets might affect visual search and trial-to-trial target repetition effects (see Maljkovic & Nakayama, 1994). Observers searched for a target diamond shape with a singleton color among distractor diamond shapes of another color (e.g., green among red or vice versa) and then judged whether the target had a notch at its top or bottom. Correct judgments led to reward, with symbolic feedback indicating this immediately; actual rewards accumulated for receipt at study end. One particular target color led to a higher (10:1) reward for 75% of its correct judgments, whereas the other singleton target color (counterbalanced over participants) yielded the higher reward on only 25% of the trials. We measured search performance in terms of inverse efficiency (response time/proportion correct). The reward schedules not only led to better performance overall for the more rewarding target color, but also increased trial-to-trial priming for successively repeated targets in that color. The actual level of reward received on the preceding trial affected this, as did (orthogonally) the likely level. When reward schedules were reversed within blocks, without explicit instruction, corresponding reversal of the impact on search performance emerged within around 6 trials, asymptoting at around 15 trials, apparently without the observers' explicit knowledge of the contingency. These results establish that pop-out search and target repetition effects can be influenced by target reward levels, with search performance and repetition effects dynamically tracking changes in reward contingency.

Prefrontal Cortex Fails to Learn from Reward Prediction Errors in Alcohol Dependence

Patients suffering from addiction persist in consuming substances of abuse, despite negative consequences or absence of positive consequences. One potential explanation is that these patients are impaired at flexibly adapting their behavior to changes in reward contingencies. A key aspect of adaptive decision-making involves updating the value of behavioral options. This is thought to be mediated via a teaching signal expressed as a reward prediction error (PE) in the striatum. However, to exert control over adaptive behavior, value signals need to be broadcast to higher executive regions, such as prefrontal cortex. Here we used functional MRI and a reinforcement learning task to investigate the neural mechanisms underlying maladaptive behavior in human male alcohol-dependent patients. We show that in alcohol-dependent patients the expression of striatal PEs is intact. However, abnormal functional connectivity between striatum and dorsolateral prefrontal cortex (dlPFC) predicted impairments in learning and the magnitude of alcohol craving. These results are in line with reports of dlPFC structural abnormalities in substance dependence and highlight the importance of frontostriatal connectivity in addiction, and its pivotal role in adaptive updating of action values and behavioral regulation. Furthermore, they extend the scope of neurobiological deficits underlying addiction beyond the focus on the striatum.

Differential effects of dopaminergic manipulations on risky choice

Evaluation of risks and rewards associated with different options is facilitated by components of the mesocorticolimbic dopamine (DA) system. Augmenting or reducing DA activity increases or decreases preference for larger, uncertain rewards when reward probabilities decrease within a session. However, manipulations of DA activity may differentially alter risky choice when shifts in the relative value of probabilistic rewards are greater or lesser than those experienced previously. We investigated the effects of amphetamine and the DA antagonist flupenthixol on risk discounting, whereby we altered the manner in which reward probabilities changed. Rats chose between a "Small/Certain" (one pellet) and a "Large/Risky" lever that delivered four pellets in a probabilistic manner that changed during a session. Separate groups of rats were trained with a descending (100%, 50%, 25%, 12.5%), ascending (12.5-100%) or mixed (100%, 12.5%, 25%, 50%) order of probabilities associated with the large/risky option. Flupenthixol consistently decreased preference for the large/risky option. In contrast, amphetamine increased preference for the large/risky lever when the probabilities decreased over a session, but reduced preference in the ascending condition. Reductions in normal DA tone consistently biases choice away larger, probabilistic rewards. In contrast, increases in DA release may disrupt adjustments in behavior in response to changes in the relative value of certain versus uncertain rewards. These findings further clarify the role of DA in mediating risk/reward judgments and how perturbations in DA signaling may interfere with the ability to adjust decision making in response to changes in reward contingencies.

Disconnection of the Entorhinal Cortex and Dorsomedial Striatum Impairs the Sensitivity to Instrumental Contingency Degradation

The capacity to detect changes in the causal efficacy of actions is mediated by a number of brain areas, including the entorhinal cortex (EC) and the posterior part of the dorsomedial striatum (pDMS). In this study we examined whether interactions between the EC and pDMS are required to detect changes in the instrumental contingency. Rats that received EC-pDMS disconnection lesions, that is, unilateral cell body lesions of the EC and contralateral dopamine depletions of the pDMS, were trained to press two levers, with one delivering food pellets and the other a sucrose solution. Thereafter, we tested whether rats were sensitive (1) to a selective devaluation of the value of one of two outcomes using a specific satiety procedure, and (2) to a selective degradation of one of two contingencies controlling instrumental choice behavior. Our results reveal that rats with EC-pDMS disconnection lesions were sensitive to outcome devaluation. However, unlike rats with sham lesions or unilateral EC and pDMS lesions, rats with EC-pDMS disconnection lesions showed a reduced sensitivity to contingency degradation. These findings suggest that EC and pDMS may be part of a neural system that supports the detection of changes in the causal relationship between an action and its consequences.

Political Rhetoric in a World Risk Society

We live in a world of risks that lurk everywhere, in the food and water we consume, in the viruses and bacteria we encounter, and in the global political scene that seems more and more volatile. This article pursues two lines of inquiry: first, I use the concept of risk, and specifically the work of Ulrich Beck, to show how the relationship between science, politics, and rhetoric is being transformed from earlier, politically progressive, twentieth-century conceptions of the role of science in public culture. Second, I try to explain how the concept of risk has altered political culture and requires a different form of prudence for political rhetoric. These two lines of inquiry work to demonstrate how uncertainty and contingency are now the products of techno-scientific rationality. This way of thinking about contingency changes how we understand the practices of political rhetoric and the constitution of public culture.

Dissociable responses to punishment in distinct striatal regions during reversal learning

Adaptive behavior depends on the ability to flexibly alter our choices in response to changes in reward and punishment contingencies. One brain region frequently implicated in such behavior is the striatum. However, this region is functionally diverse and there are a number of apparent inconsistencies across previous studies. For instance, how can significant BOLD responses in the ventral striatum during punishment-based reversal learning be reconciled with the frequently demonstrated role of the ventral striatum in reward processing? Here we attempt to address this question by separately examining BOLD responses during reversal learning driven by reward and during reversal learning driven by punishment. We demonstrate simultaneous valence-specific and valence-nonspecific signals in the striatum, with the posterior dorsal striatum responding only to unexpected reward, and the anterior ventral striatum responding to both unexpected punishment as well as unexpected reward. These data help to reconcile conflicting findings from previous studies by showing that distinct regions of the striatum exhibit dissociable responses to punishment during reversal learning.

A role for medial prefrontal dopaminergic innervation in instrumental conditioning.

To investigate the involvement of dopaminergic projections to the prelimbic and infralimbic cortex in the control of goal-directed responses, a first experiment examined the effect of pretraining 6-OHDA lesions of these cortices. We used outcome devaluation and contingency degradation procedures to separately assess the representation of the outcome as a goal or the encoding of the contingency between the action and its outcome. All groups acquired the instrumental response at a normal rate, indicating that dopaminergic activity in the medial prefrontal cortex is not necessary for the acquisition of instrumental learning. Sham-operated animals showed sensitivity to both outcome devaluation and contingency degradation. Animals with dopaminergic lesions of the prelimbic cortex, but not the infralimbic cortex, failed to adapt their instrumental response to changes in contingency, whereas their response remained sensitive to outcome devaluation. In a second experiment, aimed at determining whether dopamine was specifically needed during contingency changes, we performed microinfusions of the dopamine D(1)/D(2) receptor antagonist flupenthixol in the prelimbic cortex only before contingency degradation sessions. Animals with infusions of flupenthixol failed to adapt their response to changes in contingency, thus replicating the deficit of animals with dopaminergic lesions in Experiment 1. These results demonstrate that dissociable neurobiological mechanisms support action-outcome relationships and goal representation, dopamine signaling in the prelimbic cortex being necessary for the former but not the latter.

Effects on executive function following damage to the prefrontal cortex in the rhesus monkey (Macaca mulatta).

Executive function is a term used to describe the cognitive processes subserved by the prefrontal cortex (PFC). An extensive body of work has characterized the effects of damage to the PFC in nonhuman primates, but it has focused primarily on the capacity of recognition and working memory. One limitation in studies of the functional parcellation of the PFC has been the absence of tests that assess executive function or its functional components. The current study used an adaptation of the Wisconsin Card Sorting Test, a classic test of frontal lobe and executive function in humans, to assess the effects of bilateral lesions in the dorsolateral PFC on executive function in the rhesus monkey (Macaca mulatta). The authors used the category set-shifting task, which requires the monkey to establish a pattern of responding to a specific category (color or shape) based on reward contingency, maintain that pattern of responding, and then shift to responding to a different category when the reward contingency changes. Rhesus monkeys with lesions of the dorsolateral PFC were impaired in abstraction, establishing a response pattern to a specific category and maintaining and shifting that response pattern on the category set-shifting task.

Neurological correlates of reward responding in adolescents with and without externalizing behavior disorders.

Opposing theories of striatal hyper- and hypodopaminergic functioning have been suggested in the pathophysiology of externalizing behavior disorders. To test these competing theories, the authors used functional MRI to evaluate neural activity during a simple reward task in 12- to 16-year-old boys with attention-deficit/hyperactivity disorder and/or conduct disorder (n = 19) and in controls with no psychiatric condition (n = 11). The task proceeded in blocks during which participants received either (a) monetary incentives for correct responses or (b) no rewards for correct responses. Controls exhibited striatal activation only during reward, shifting to anterior cingulate activation during nonreward. In contrast, externalizing adolescents exhibited striatal activation during both reward and nonreward. Externalizing psychopathology appears to be characterized by deficits in processing the omission of predicted reward, which may render behaviors that are acquired through environmental contingencies difficult to extinguish when those contingencies change.

Neural substrates of contingency learning and executive control: dissociating physical, valuative, and behavioral changes

Contingency learning is fundamental to cognition. Knowledge about environmental contingencies allows behavioral flexibility, as executive control processes accommodate the demands of novel or changing environments. Studies of experiential learning have focused on the relationship between actions and the values of associated outcomes. However, outcome values have often been confounded with the physical changes in the outcomes themselves. Here, we dissociated contingency learning into valuative and non-valuative forms, using a novel version of the two-alternative choice task, while measuring the neural effects of contingency changes using functional magnetic resonance imaging (fMRI). Changes in value-relevant contingencies evoked activation in the lateral prefrontal cortex (LPFC), posterior parietal cortex (PPC), and dorsomedial prefrontal cortex (DMPFC) consistent with prior results (e.g., reversal-learning paradigms). Changes in physical contingencies unrelated to value or to action produced similar activations within the LPFC, indicating that LPFC may engage in generalized contingency learning that is not specific to valuation. In contrast, contingency changes that required behavioral shifts evoked activation localized to the DMPFC, supplementary motor, and precentral cortices, suggesting that these regions play more specific roles within the executive control of behavior.

Auditory Stimulation Dishabituates Olfactory Responses via Noradrenergic Cortical Modulation

Dishabituation is a return of a habituated response if context or contingency changes. In the mammalian olfactory system, metabotropic glutamate receptor mediated synaptic depression of cortical afferents underlies short-term habituation to odors. It was hypothesized that a known antagonistic interaction between these receptors and norepinephrine ß-receptors provides a mechanism for dishabituation. The results demonstrate that a 108 dB siren induces a two-fold increase in norepinephrine content in the piriform cortex. The same auditory stimulus induces dishabituation of odor-evoked heart rate orienting bradycardia responses in awake rats. Finally, blockade of piriform cortical norepinephrine ß-receptors with bilateral intracortical infusions of propranolol (100 μM) disrupts auditory-induced dishabituation of odor-evoked bradycardia responses. These results provide a cortical mechanism for a return of habituated sensory responses following a cross-modal alerting stimulus.

Lesions of the Medial Striatum in Monkeys Produce Perseverative Impairments during Reversal Learning Similar to Those Produced by Lesions of the Orbitofrontal Cortex

The ability to switch responding between two visual stimuli based on their changing relationship with reward is dependent on the orbitofrontal cortex (OFC). OFC lesions in humans, monkeys, and rats disrupt performance on a common test of this ability, the visual serial discrimination reversal task. This finding is of particular significance to our understanding of psychiatric disorders such as obsessive-compulsive disorder (OCD) and schizophrenia, in which behavioral inflexibility is a prominent symptom. Although OFC dysfunction can occur in these disorders, there is considerable evidence for more widespread dysfunction within frontostriatal and frontoamygdalar circuitry. Because the contribution of these subcortical structures to behavioral flexibility is poorly understood, the present study compared the effects of excitotoxic lesions of the medial striatum (MS), amygdala, and OFC in the marmoset monkey on performance of the serial reversal task. All monkeys were able to learn a novel stimulus-reward association but, compared with both control and amygdala-lesioned monkeys, those with MS or OFC lesions showed a perseverative impairment in their ability to reverse this association. However, whereas both MS and OFC groups showed insensitivity to negative feedback, only OFC-lesioned monkeys showed insensitivity to positive feedback. These findings suggest that, for different reasons, both the MS and OFC support behavioral flexibility after changes in reward contingencies, and are consistent with the hypothesis that striatal and OFC dysfunction can contribute to pathological perseveration.

Context-Dependent Changes in Functional Circuitry in Visual Area MT

Animals can flexibly change their behavior in response to a particular sensory stimulus; the mapping between sensory and motor representations in the brain must therefore be flexible as well. Changes in the correlated firing of pairs of neurons may provide a metric of changes in functional circuitry during behavior. We studied dynamic changes in functional circuitry by analyzing the noise correlations of simultaneously recorded MT neurons in two behavioral contexts: one that promotes cooperative interactions between the two neurons and another that promotes competitive interactions. We found that identical visual stimuli give rise to differences in noise correlation in the two contexts, suggesting that MT neurons receive inputs of central origin whose strength changes with the task structure. The data are consistent with a mixed feature-based attentional strategy model in which the animal sometimes alternates attention between opposite directions of motion and sometimes attends to the two directions simultaneously.

Amygdala and Orbitofrontal Cortex Lesions Differentially Influence Choices during Object Reversal Learning

In nonhuman primates, interaction between the orbitofrontal cortex (OFC) and the amygdala (AMG) has been seen as critical for learning and subsequently changing associations between stimuli and reinforcement. However, it is still unclear what the precise role of the OFC is in altering these stimulus-reward associations, and recent research has questioned whether the AMG makes an essential contribution at all. To gain a better understanding of the role of these two structures in flexibly associating stimuli with reinforcement, we reanalyzed a set of previously published data from groups of monkeys with either OFC or AMG lesions that had been tested on an object reversal learning task. Based on trial-by-trial analyses of rewarded and unrewarded choices, we report two new findings. First, monkeys with OFC lesions were, compared with both control and AMG groups, unable to use correctly performed trials to optimally guide subsequent choices. Second, monkeys with AMG lesions showed the opposite pattern of behavior. This group benefited more than controls from correctly performed trials that followed an error. Finally, as has been reported by others, after a change in reward contingencies, monkeys with OFC lesions also showed a slightly greater tendency to choose the previously rewarded object. These findings demonstrate that the OFC and AMG make different contributions to object reversal learning not highlighted previously.