Stimulus-reward Learning Research Articles

Parkinson's disease duration determines effect of dopaminergic therapy on ventral striatum function

We investigated the hypothesis that variation in endogenous dopamine (DA) across brain regions explains dissimilar effects of dopaminergic therapy on aspects of cognition in early Parkinson's disease (PD). Extensive degeneration of DA-producing cells in the substantia nigra cause dorsal striatum (DS) DA deficiency and movement abnormalities. Particularly in early PD, this contrasts with relative sparing of the dopaminergic cells of the ventral tegmental area (VTA). The hypothesis predicts that DS-mediated cognitive functions are deficient at baseline and improved by DA replacement, whereas functions depending upon VTA-innervated brain regions are normal off medication and worsen with treatment. The latter pattern presumably owes to overdose of relatively DA-replete VTA-supplied brain regions with medication levels titrated to DS-mediated motor symptoms. As PD progresses, however, VTA degeneration increases. Impairment in cognitive operations performed by VTA-innervated brain regions, such as the ventral striatum (VS), is expected. We compared the performance of early and late PD patients, on and off dopaminergic medication, relative to age-matched controls, on reward learning, previously shown to implicate the VS. As expected, in early PD, stimulus-reward learning was normal off medication, but worsened with DA replacement. At more advanced disease stages, PD patients learned stimulus-reward contingencies more poorly than controls and early PD patients off medication. Furthermore, dopaminergic medication did not worsen reward learning in late PD patients, in line with the dopamine overdose hypothesis. Unlike its effect on DS-mediated functions, however, DA-replacement therapy did not improve reward learning in late PD patients.

NMDA Receptors Control Cue-Outcome Selectivity and Plasticity of Orbitofrontal Firing Patterns during Associative Stimulus-Reward Learning

Neural activity in orbitofrontal cortex has been linked to flexible representations of stimulus-outcome associations. Such value representations are known to emerge with learning, but the neural mechanisms supporting this phenomenon are not well understood. Here, we provide evidence for a causal role for NMDA receptors (NMDARs) in mediating spike pattern discriminability, neural plasticity, and rhythmic synchronization in relation to evaluative stimulus processing and decision making. Using tetrodes, single-unit spike trains and local field potentials were recorded during local, unilateral perfusion of an NMDAR blocker in rat OFC. In the absence of behavioral effects, NMDAR blockade severely hampered outcome-selective spike pattern formation to olfactory cues, relative to control perfusions. Moreover, NMDAR blockade shifted local rhythmicsynchronization to higher frequencies and degradedits linkage to stimulus-outcome selective coding.These results demonstrate the importance of NMDARs for cue-outcome associative coding in OFC during learning and illustrate how NMDAR blockade disrupts network dynamics.

The role of dopamine in the accumbens core in the expression of Pavlovian‐conditioned responses

The role of dopamine in reward is a topic of debate. For example, some have argued that phasic dopamine signaling provides a prediction-error signal necessary for stimulus-reward learning, whereas others have hypothesized that dopamine is not necessary for learning per se, but for attributing incentive motivational value ('incentive salience') to reward cues. These psychological processes are difficult to tease apart, because they tend to change together. To disentangle them we took advantage of natural individual variation in the extent to which reward cues are attributed with incentive salience, and asked whether dopamine (specifically in the core of the nucleus accumbens) is necessary for the expression of two forms of pavlovian-conditioned approach behavior--one in which the cue acquires powerful motivational properties (sign-tracking) and another closely related one in which it does not (goal-tracking). After acquisition of these conditioned responses (CRs), intra-accumbens injection of the dopamine receptor antagonist flupenthixol markedly impaired the expression of a sign-tracking CR, but not a goal-tracking CR. Furthermore, dopamine antagonism did not produce a gradual extinction-like decline in behavior, but maximally impaired expression of a sign-tracking CR on the very first trial, indicating the effect was not due to new learning (i.e. it occurred in the absence of new prediction-error computations). The data support the view that dopamine in the accumbens core is not necessary for learning stimulus-reward associations, but for attributing incentive salience to reward cues, transforming predictive conditional stimuli into incentive stimuli with powerful motivational properties.

Neural Correlates of Human Associative Learning

Lesion studies in animals have associated the amygdala mainly with aversive conditioning, both cued and contextual. However, neurophysiological studies indicate that the amygdala has a role in the processing of positive emotions in some kinds of stimulus-reward learning. The aim of the current study was to extend these findings to humans. Functional magnetic resonance imaging was applied to investigate the neural basis of aversive and appetitive conditioning. In the first study, aversive electrical shocks were delivered as either cue or context related. The second study integrated appetitive and aversive conditioning using electric shocks as aversive and monetary rewards as appetitive reinforcers. Differential responses in the amygdala were observed during both cue and contextual conditioning, whereas context conditioning additionally recruited the hippocampus. Amygdala activity was higher for reinforced acoustic stimuli. The findings underline the importance of the amygdala in associative learning, irrespective of the stimulus valence and stimulus type.

A dopaminergic dichotomy

Dopaminergic signalling encodes the incentive salience of cues in a form of stimulus–reward learning.

A selective role for dopamine in stimulus–reward learning

Individuals make choices and prioritize goals using complex processes that assign value to rewards and associated stimuli. During Pavlovian learning, previously neutral stimuli that predict rewards can acquire motivational properties, whereby they themselves become attractive and desirable incentive stimuli. But individuals differ in whether a cue acts solely as a predictor that evokes a conditional response, or also serves as an incentive stimulus, and this determines the degree to which a cue might bias choice or even promote maladaptive behavior. Here we use rats that differ in the incentive motivational properties they attribute to food cues to probe the role of the neurotransmitter dopamine in stimulus-reward learning. We show that intact dopamine transmission is not required for all forms of learning in which reward cues become effective predictors. Rather, dopamine acts selectively in a form of reward learning in which “incentive salience” is assigned to reward cues. In individuals with a propensity for this form of learning, reward cues come to powerfully motivate and control behavior. This work provides insight into the neurobiology of a form of reward learning that confers increased susceptibility to disorders of impulse control.

Expression of a conditioned place preference or spatial navigation task following muscimol-induced inactivations of the amygdala or dorsal hippocampus: A double dissociation in the retrograde direction

Previous work indicates an essential role of the basolateral amygdala in stimulus-reward learning and the dorsal hippocampus in spatial learning and memory. The goal of the present, experiments was to examine the involvement of the amygdala and hippocampus in performance of tasks requiring stimulus-reward and spatial/relational learning and memory processes in the retrograde direction. Accordingly, this series of experiments tested the effects of temporary, inactivations directed at the basolateral nucleus of the amygdala or dorsal hippocampus on the, expression of a conditioned place preference (CPP) task or a spatial navigation water task. The results, of Experiments 1a and b showed that inactivations of the amygdala impaired the expression of a, previously acquired CPP but did not impair the expression of a learned spatial response required for, accurate performance of a spatial navigation task. The results of Experiments 2a and b showed that, inactivations of the dorsal hippocampus impaired expression of a learned response required for the, accurate performance of a spatial navigation task but did not impair the learned response required for, the expression of a CPP. Taken together, the results showed a functional dissociation between the, effects of amygdala or hippocampal dysfunction on the expression of these different classes of tasks.

Long‐term cognitive enrichment affects opioid receptor expression in the amygdala of domestic pigs

Enriching the housing environment by stimuli that challenge both reward and cognitive mechanisms may enhance behavioural experiences and can improve animal welfare, particularly in farm animals. A newly developed experimental feeding system for domestic pigs using food-rewarded learning of discriminatory and instrumental tasks enabled the animals to successfully master a cognitive challenge and to be rewarded ca. 30 times per day with small food portions. Reward-related behaviour is expected to be modulated by endogenous opioid systems. Furthermore, recent evidence supports a role for the amygdala in processing positive affects by stimulus-reward learning. Hence, the present study investigates mRNA expression of cerebral receptors, which are involved in these processes. In an initial step, reverse transcription-polymerase chain reaction (RT-PCR) provided the first evidence that transcripts of three different opioid receptors (MOR, DOR, KOR), as well as the neuropeptide Y 5 receptor (NPY5R), leptin receptor (LEPR) and proopiomelanocortin (POMC), are expressed in both the porcine amygdala and hypothalamus. Using real-time PCR we could show that the expression of two receptors of the opioid system (amygdala: KOR, DOR), in addition to the expression of NPY5R (hypothalamus) in eight enriched housed pigs was markedly downregulated compared to that of conventionally housed and fed pigs. Focusing on opioid receptors in the amygdala, the present study shows that long-term cognitive enrichment acts as a biologically relevant stimulus that causes modifications of gene expression of reward-sensitive cerebral receptors in domestic pigs.

Medial temporal lobe functioning and structure in the spontaneously hypertensive rat: Comparison with Wistar–Kyoto normotensive and Wistar–Kyoto hypertensive strains

The spontaneously hypertensive rat (SHR) is used as an animal model of attention deficit hyperactivity disorder (ADHD). It displays deficits in frontostriatal functioning, but it is unclear if medial temporal lobe functioning and structure are affected. We used behavioral tasks that evaluate functioning of the amygdala and hippocampus to compare male SHR to male rats from two inbred comparator strains, the normotensive Wistar-Kyoto (WKY) and the hypertensive Wistar-Kyoto (WKHT) rat (n = 8/strain). The three strains showed similar levels of amygdala-related stimulus-reward learning during conditioned cue preference testing. In the ambiguous T-maze task, which dissociates between spatial and habit learning, significantly more WKHT than SHR or WKY used a response (indicative of habit learning) versus a place (indicative of spatial learning) strategy during an early probe test on day 8. During a later probe test on day 24, WKY progressed significantly from using a place strategy to a response strategy. Throughout all probe tests, a place strategy was used predominately by SHR and a response strategy by WKHT. Thus, SHR exhibited deficits in dorsal striatum-related habit learning, whereas WKHT exhibited deficits in hippocampus-related spatial learning. Following behavioral testing, fluid attenuated inversion recovery (FLAIR) magnetic resonance imaging scans were conducted in subgroups of rats from each strain (n = 4/strain). FLAIR imaging detected bilateral hippocampal hyperintensities in three of four WKHT and unilateral hippocampal atrophy in one of four SHR. The association between response strategy use during the initial probe test to forage for food in the ambiguous T-maze task and bilateral hippocampal abnormalities was significant. Collectively, while medial temporal lobe functioning appears to be normal in SHR exhibiting an ADHD-like phenotype, WKHT rats display both hippocampal functioning deficits and signs of bilateral hippocampal cell loss. The latter characteristics might be used to develop a new animal model of age- or disease-related decline in hippocampal functioning.

Effects of self-administered cocaine in adolescent and adult male rats on orbitofrontal cortex-related neurocognitive functioning

Deficits in amygdala-related stimulus-reward learning are produced following 18 drug-free days of cocaine self-administration or its passive delivery in rats exposed during adulthood. No deficits in stimulus-reward learning are produced by cocaine exposure initiated during adolescence. To determine if age of initiating cocaine exposure differentially affects behavioral functioning of an additional memory system linked to cocaine addiction, the orbitofrontal cortex. A yoked-triad design (n = 8) was used. One rat controlled cocaine delivery and the other two passively received cocaine or saline. Rats controlling drug delivery (1.0 mg/kg) self-administered cocaine from either P37-P59 or P77-P99, and then underwent 18 drug-free days (P60-P77 vs. P100-P117). Rats next were tested for acquisition of odor-delayed win-shift behavior conducted over 15 sessions (P78-P96 vs. P118-P136). Cocaine self-administration did not differ between adults and adolescents. During the test phase of the odor-delayed win-shift task (relatively difficult task demands), rats from both drug-onset ages showed learning deficits. Rats with cocaine self-administration experience committed more errors and had longer session latencies compared to rats passively receiving saline or cocaine. Rats with adolescent-onset cocaine self-administration experience showed an additional learning deficit by requiring more sessions to reach criterion levels for task acquisition compared to same-aged passive saline controls or rats with adult-onset cocaine self-administration experience. Rats passively receiving cocaine did not differ from the passive saline control from either age group. Rats with adolescent-onset cocaine self-administration experience were more impaired in an orbitofrontal cortex-related learning task than rats with adult-onset cocaine self-administration experience.

Serial position learning in honeybees.

Learning of stimulus sequences is considered as a characteristic feature of episodic memory since it contains not only a particular item but also the experience of preceding and following events. In sensorimotor tasks resembling navigational performance, the serial order of objects is intimately connected with spatial order. Mammals and birds develop episodic(-like) memory in serial spatio-temporal tasks, and the honeybee learns spatio-temporal order when navigating between the nest and a food source. Here I examine the structure of the bees’ memory for a combined spatio-temporal task. I ask whether discrimination and generalization are based solely on simple forms of stimulus-reward learning or whether they require sequential configurations. Animals were trained to fly either left or right in a continuous T-maze. The correct choice was signaled by the sequence of colors (blue, yellow) at four positions in the access arm. If only one of the possible 4 signals is shown (either blue or yellow), the rank order of position salience is 1, 2 and 3 (numbered from T-junction). No learning is found if the signal appears at position 4. If two signals are shown, differences at positions 1 and 2 are learned best, those at position 3 at a low level, and those at position 4 not at all. If three or more signals are shown these results are corroborated. This salience rank order again appeared in transfer tests, but additional configural phenomena emerged. Most of the results can be explained with a simple model based on the assumption that the four positions are equipped with different salience scores and that these add up independently. However, deviations from the model are interpreted by assuming stimulus configuration of sequential patterns. It is concluded that, under the conditions chosen, bees rely most strongly on memories developed during simple forms of associative reward learning, but memories of configural serial patterns contribute, too.

Rewarding Morphine-Induced Synaptic Function of δ-Opioid Receptors on Central Glutamate Synapses

The rewarding effect of opioids, the driving force for compulsive behaviors of opioid abuse and addiction, is primarily mediated by the mu-opioid receptor. However, the role of the delta-opioid receptor (DOR) in opioid reward and addiction is still poorly understood. The recently discovered adaptive DOR property of exocytotic translocation in sensory neurons after chronic opioid exposure provides a new avenue of conceptual thoughts to exploring the DOR function in this psychoneurological disease. In this study, we investigated potential adaptive function of DOR in neurons of the central nucleus of the amygdala (CeA), a forebrain structure increasingly recognized for mediating stimulus reward learning in drug addiction. Using whole-cell recordings in CeA slices, we found that in rats displaying morphine-induced behavior of conditioned place preference, a behavioral measure of drug reward, the overall synaptic strength of glutamate synapses in CeA neurons was significantly enhanced. The selective DOR agonist [D-Pen(2),D-Pen(5)]-enkephalin, having no apparent effect on glutamatergic excitatory postsynaptic current (EPSC) in neurons from control rats, produced a significant, dose-dependent inhibition of the synaptic current in neurons from those morphine-treated rats. Detailed analyses of EPSC properties revealed that DOR activation inhibited the EPSC by reducing presynaptic release of glutamate, indicating functional DOR emerging on presynaptic glutamate terminals. The morphine treatment also significantly increased DOR proteins in CeA preparations of synaptosomes. These findings provide functional evidence for an adaptive modulation by presynaptic DOR of a key synaptic activity altered by morphine, thus implying likely important involvement of DOR in opioid reward and addiction.

Altered function of ventral striatum during reward-based decision making in old age

Normal aging is associated with a decline in different cognitive domains and local structural atrophy as well as decreases in dopamine concentration and receptor density. To date, it is largely unknown how these reductions in dopaminergic neurotransmission affect human brain regions responsible for reward-based decision making in older adults. Using a learning criterion in a probabilistic object reversal task, we found a learning stage by age interaction in the dorsolateral prefrontal cortex (dlPFC) during decision making. While young adults recruited the dlPFC in an early stage of learning reward associations, older adults recruited the dlPFC when reward associations had already been learned. Furthermore, we found a reduced change in ventral striatal BOLD signal in older as compared to younger adults in response to high probability rewards. Our data are in line with behavioral evidence that older adults show altered stimulus–reward learning and support the view of an altered fronto-striatal interaction during reward-based decision making in old age, which contributes to prolonged learning of reward associations.

The Contribution of Orbitofrontal Cortex to Action Selection

A number of recent findings suggest that the orbitofrontal cortex (OFC) influences action selection by providing information about the incentive value of behavioral goals or outcomes. However, much of this evidence has been derived from experiments using Pavlovian conditioning preparations of one form or another, making it difficult to determine whether the OFC is selectively involved in stimulus-outcome learning or whether it plays a more general role in processing reward value. Although many theorists have argued that these are fundamentally similar processes (i.e., that stimulus-reward learning provides the basis for choosing between actions based on anticipated reward value), several behavioral findings indicate that they are, in fact, dissociable. We have recently investigated the role of the OFC in the control of free operant lever pressing using tests that independently target the effect of stimulus-outcome learning and outcome devaluation on performance. We found that OFC lesions disrupted the tendency of Pavlovian cues to facilitate instrumental performance but left intact the suppressive effects of outcome devaluation. Rather than processing goal value, therefore, we hypothesize that the contribution of the OFC to goal-directed action is limited to encoding predictive stimulus-outcome relationships that can bias instrumental response selection.

Differential effects of self-administered cocaine in adolescent and adult rats on stimulus–reward learning

Adult cocaine addicts, abstinent at the time of testing, show a variety of neurocognitive impairments. Less clear is whether there are differences in the degree of impairment if cocaine use is initiated during adolescence rather than adulthood. Using a preclinical model, we evaluated if stimulus-reward learning was impacted differently in rats exposed to cocaine during adolescence (beginning on postnatal day 37) vs adulthood (beginning on postnatal days 74-79) and then tested after a drug-free period. A yoked-triad design of intravenous cocaine self-administration in adult (n = 8 triads) and adolescent (n = 8 triads) rats was used. Sets of three animals either contingently self-administered cocaine or received cocaine or saline in a noncontingent manner. Rats self-administering 1-mg/kg doses of cocaine responded under a fixed-ratio 5, timeout 20-s schedule of reinforcement. After 18 2-h drug or saline sessions, all rats (now adults) began the drug-free period in their home environments. Testing in a stimulus-reward learning task (conditioned cue preference) began 19 days later. Self-administration behavior was similar in adolescent and adult rats. Lever responses were not significantly different, and both age groups averaged approximately 20 infusions per session. Rats contingently self-administering cocaine or passively exposed to cocaine during adulthood showed stimulus-reward learning deficits in the conditioned cue preference task. Rats exposed to contingent or noncontingent cocaine during adolescence had normal learning, showing strong preferences for a Froot Loops-paired cue. These findings suggest that adolescents are insensitive to cocaine-induced impairment of learning related to amygdala memory system functioning.

Risk and protective haplotypes of the alpha‐synuclein gene associated with Parkinson’s disease differentially affect cognitive sequence learning

Alpha-synuclein (SNCA) is a key factor in the regulation of dopaminergic transmission and is related to Parkinson's disease. In this study, we investigated the effects of risk and protective SNCA haplotypes associated with Parkinson's disease on cognitive sequence learning in 204 healthy volunteers. We found that the 3'-block risk SNCA haplotypes are associated with less effective stimulus-reward learning of sequences and with superior context representation of sequences. In contrast, participants with protective haplotypes exhibit better stimulus-reward learning and worse context representation, which suggest that these functions are inversely affected by risk and protective haplotypes. The Rep1 promoter polymorphism does not influence cognitive sequence learning. Because stimulus-reward learning may be mediated by the basal ganglia and context learning may be related to the medial temporal lobe, our data raise the possibility that dopaminergic signals regulated by SNCA inversely affect these memory systems.

AMPA-Receptor GluR1 Subunits are Involved in the Control Over Behavior by Cocaine-Paired Cues

The learning processes underlying the formation of drug-cue associations involve changes in synaptic transmission mediated by AMPA receptors. Here, we examine the consequences of targeted deletion of the gene encoding GluR1 subunits of AMPA receptors (gria1 knockouts (KO)) on cocaine self-administration and on the ability of cocaine-paired cues to affect cocaine-seeking in mice. Cocaine self-administration was unaffected by gria1 deletion, as was the ability of a cocaine-paired cue to reinstate responding following extinction, following either a 3 or a 66 day delay. However, gria1 KOs over-responded during extinction. The KOs were unable initially to learn a new response to access a cue previously conditioned to cocaine (conditioned reinforcement (CR)), although a second test 2 months later revealed that this was a transient deficit. These studies indicate that GluR1-containing AMPA-receptors are not involved in cocaine self-administration, cue-induced reinstatement of cocaine seeking, or incubation of the cocaine seeking response. In order to understand the specificity of the deficits in CR responding, a parallel study was performed with food reward. As with cocaine, there were no effects of gria1 deletion on food self-administration or cue-induced reinstatement, and KOs over-responded during extinction. However, even immediately after instrumental training for food, KO mice demonstrated responding for CR, suggesting that the CR deficit observed with a cocaine cue is specific to drug reward. These data indicate that GluR1-containing AMPA receptors are important in stimulus reward learning, though the method of cue-reward association formation, the reward class, and the behavioral end point are critical variables in determining their involvement.

Lesions of the dorsolateral or dorsomedial striatum impair performance of a previously acquired simple discrimination task

Previous evidence has suggested a specific role for the dorsal striatum, especially the dorsolateral region of the dorsal striatum, in stimulus–response learning. In a previous study, we found an impairment in animals with dorsolateral striatal lesions on a simple discrimination task (CS+/CS−), thought to require the involvement of both stimulus–reward and stimulus–response learning. It is possible that the generally poor performance of dorsolateral lesioned animals on this experiment precluded adequate exposure to stimulus–reward pairings necessary for solving this task, and, thus, had little to do with stimulus–response learning. To test this hypothesis, the performance of animals with dorsolateral and dorsomedial striatal lesions was assessed on a previously acquired simple discrimination task. To independently assess the effects of each lesion on the performance of stimulus–reward learning, dorsolateral and dorsomedial lesioned animals were assessed on a previously acquired conditioned place preference task (CPP). In agreement with our earlier experiment, and the stimulus–response interpretation of dorsolateral striatal function, animals with dorsolateral striatal lesions were found to be impaired during post-lesion performance of the simple discrimination task, but not CPP learning. Additionally, dorsomedial lesioned animals were found to be impaired in performance of the simple discrimination task, but not on the CPP task. Possible explanations for the differences between the role of the dorsomedial striatum in acquisition and expression of the simple discrimination task are proposed.

Post-training intra-amygdala amphetamine injections given during acquisition of a stimulus–response (S–R) habit task enhance the expression of stimulus–reward learning: Further evidence for incidental amygdala learning

The effect of post-training intra-amygdala amphetamine injections was examined on the acquisition and expression of a visual discrimination task. Rats were trained to enter four lit arms for food (stimulus–response) and avoid unlit arms on an eight-arm radial maze visual discrimination task. Post-training intra-amygdala amphetamine injections (10 μg) were given for 4 consecutive days during the mid-point of training (days 20–23). The number of lit arm entries was used as a measure of stimulus–response habit learning 24 h after each injection. Twenty-four hours after the last injection, a transfer test was run to assess the effect of the same post-training manipulation. This transfer test assessed the amount of time spent in the lit arms and was used as a measure of stimulus–reward learning. Compared to saline-injected rats, rats that received post-training amphetamine spent more time in lit as opposed to dark arms during the transfer test. This occurred in the absence of an increase in the number of correct arm entries during visual discrimination training. This suggests that post-training amphetamine strengthened a stimulus–reward association that did not immediately affect behavioral output. This association may reflect a mnemonic representation stored in an ensemble of amygdala neurons.

Does bilateral stimulation of the subthalamic nucleus aggravate apathy in Parkinson’s disease?

Objective: High frequency stimulation of the subthalamic nucleus (STN) dramatically decreases motor disability in patients with Parkinson”s disease (PD), but has been reported to aggravate apathy. The aim of this...