Role In Reward-related Learning Research Articles

Muscarinic and NMDA Receptors in the Substantia Nigra Play a Role in Reward-Related Learning.

Reward-related learning, where animals form associations between rewards and stimuli (i.e., conditioned stimuli [CS]) that predict or accompany those rewards, is an essential adaptive function for survival. In this study, we investigated the mechanisms underlying the acquisition and performance of conditioned approach learning with a focus on the role of muscarinic acetylcholine (mACh) and NMDA glutamate receptors in the substantia nigra (SN), a brain region implicated in reward and motor processes. Using RNAscope in situ hybridization assays, we found that dopamine neurons of the SN express muscarinic (mACh5), NMDA2a, NMDA2b, and NMDA2d receptor mRNA but not mACh4. NMDA, but not mACh5, receptor mRNA was also found on SN GABA neurons. In a conditioned approach paradigm, rats were exposed to 3 or 7 conditioning sessions during which light/tone (CS) presentations were paired with delivery of food pellets, followed by a test session with CS-only presentations. Intra-SN microinjections of scopolamine (a mACh receptor antagonist) or AP-5 (a NMDA receptor antagonist) were made either prior to each conditioning session (to test their effects on acquisition) or prior to the CS-only test (to test their effects on expression of the learned response). Scopolamine and AP-5 produced dose-dependent significant reductions in the acquisition, but not performance, of conditioned approach. These results suggest that SN mACh and NMDA receptors are key players in the acquisition, but not the expression, of reward-related learning. Importantly, these findings redefine the role of the SN, which has traditionally been known for its involvement in motor processes, and suggest that the SN possesses attributes consistent with a function as a hub of integration of primary reward and CS signals.

Investigating dopamine and glucocorticoid systems as underlying mechanisms of anhedonia

Anhedonia, a deficit in reward processing, is an endophenotype of several neuropsychiatric conditions. Despite its prevalence and debilitating effects, treatments for anhedonia are lacking, primarily because its underlying mechanisms are poorly understood. Dopamine (DA) has been implicated in anhedonia through its role in reward-related learning; glucocorticoid systems may also be involved in that anhedonia is often preceded by chronic stress. This study investigated DA and glucocorticoid systems in anhedonia using a rat version of the probabilistic reward task (PRT). Adult male Wistar rats were trained on the PRT and then tested following: (1) activation or inhibition of DA activity induced by amphetamine (AMPH) or pramipexole (PRAMI) injections, (2) chronic mild stress (CMS), or (3) glucocorticoid system activation (dexamethasone (DEX)) or inhibition (mifepristone (MIFE)). AMPH increased and PRAMI decreased response bias, pointing to enhanced and diminished reward responsiveness with DA agonism and antagonism, respectively. CMS reduced response bias but only in a subpopulation of rats. DEX also decreased response bias, suggesting that glucocorticoid processes contribute to anhedonia, although glucocorticoid inhibition (MIFE) had no effect. None of the manipulations altered the ability to detect and respond to reward-paired stimuli. These results confirm a role of DA in anhedonia and elucidate the contribution of the glucocorticoid system to this effect. In addition, chronic stress may interfere with normal DA functioning, leading to impaired reward-related learning in some animals. These findings may direct future treatment of anhedonia by targeting DA and glucocorticoid systems, as well as a possible interaction between the two.

Regionally distinct phasic dopamine release patterns in the striatum during reversal learning

Striatal dopamine (DA) plays a central role in reward-related learning and behavioral adaptation to changing environments. Recent studies suggest that rather than being broadcast as a uniform signal throughout the entire region, DA release dynamics diverge between different striatal regions. In a previous study, we showed that phasic DA release patterns in the ventromedial striatum (VMS) rapidly adapt during reversal learning. However, it is unknown how DA dynamics in the dorsolateral striatum (DLS) are modulated during such adaptive behavior. Here, we used fast-scan cyclic voltammetry to measure phasic DA release in the DLS during spatial reversal learning. In the DLS, we observed minor DA release after the onset of a visual cue signaling reward availability, followed by more pronounced DA release during more proximal reward cues (e.g., lever extension) and execution of the operant response (i.e., lever press), both in rewarded and non-rewarded trials. These release dynamics (minor DA after onset of the predictive visual cue, prominent DA during the operant response) did not change significantly during or following a reversal of response-reward contingencies. Notably, the DA increase to the lever press did not reflect a general signal related to the initiation of any motivated motor response, as we did not observe DA release when rats initiated nose pokes into the food receptacle during inter-trial intervals. This suggests that DA release in the DLS occurs selectively during the initiation and execution of a learned operant response.Together with our previous results obtained in the VMS, these findings reveal distinct phasic DA release patterns during adaptation of established behavior in DLS and VMS. The VMS DA signal, which is highly sensitive to reversal of response-reward contingences, may provide a teaching signal to guide reward-related learning and facilitate behavioral adaptation, whereas DLS DA may reflect a ‘response execution signal’ largely independent of outcome, that may be involved in initiation and energizing of operant behavior.

Chapter 3 - The role of learning-related dopamine signals in addiction vulnerability

Dopaminergic signals play a mathematically precise role in reward-related learning, and variations in dopaminergic signaling have been implicated in vulnerability to addiction. Here, we provide a detailed overview of the relationship between theoretical, mathematical, and experimental accounts of phasic dopamine signaling, with implications for the role of learning-related dopamine signaling in addiction and related disorders. We describe the theoretical and behavioral characteristics of model-free learning based on errors in the prediction of reward, including step-by-step explanations of the underlying equations. We then use recent insights from an animal model that highlights individual variation in learning during a Pavlovian conditioning paradigm to describe overlapping aspects of incentive salience attribution and model-free learning. We argue that this provides a computationally coherent account of some features of addiction.

Double Dissociation of the Effects of Haloperidol and the Dopamine D3 Receptor Antagonist ABT-127 on Acquisition vs. Expression of Cocaine-Conditioned Activity in Rats

Dopamine receptors play a critical role in reward-related learning, but receptor subtypes may be differentially involved. D2-preferring receptor antagonists, e.g., haloperidol, attenuate acquisition of cocaine-conditioned motor activity at doses that fail to block expression. We compared haloperidol [4-[4-(4-chlorophenyl)-4-hydroxy-1-piperidyl]-1-(4-fluorophenyl)-butan-1-one] with the D3 receptor-preferring antagonist 2,3-di-tert-butyl-6-{4-[3-(4,5-dimethyl-4H-[1,2,4] triazol-3-yisulfanyl)-propyl]-piperazin-1-y1}-pyrimidine hydrochloride (ABT-127), given at D3 receptor-selective doses [i.e., no displacement of [(3)H]3,5-dichloro-N-[[(2S)-1-ethyl-2-pyrrolidinyl]methyl]-2-hydroxy-6-methoxybenzamide binding, no effects on γ-butyrolactone-induced striatal l-3,4-dihydroxyphenylalanine; haloperidol accumulation; no attenuation of apomorphine-induced stereotypy]. We hypothesized that haloperidol and ABT-127 will produce a doubly dissociable effect on acquisition versus expression of cocaine-conditioned activity. Rats received three 1-h habituation sessions to activity monitors followed by three 1-h cocaine (10 mg/kg) conditioning sessions. The expression phase (no cocaine injections) took place 48 h later. Haloperidol (50 μ/kg) given during the conditioning phase blocked the acquisition of conditioned activity but failed to block the expression of conditioning when given on the test day. In contrast, ABT-127 (1.0 mg/kg), when given during conditioning, failed to block the acquisition of conditioned activity but blocked the expression of conditioning when administered on the test day. Results suggest that D2 receptors are more critically involved in acquisition than initial expression and D3 receptors are more critically involved in expression than acquisition of conditioned activity based on cocaine.

VTA Dopamine Neuron Bursting is Altered in an Animal Model of Depression and Corrected by Desipramine

Ventral tegmental area (VTA) neuronal activity plays an important role in reward-related learning and motivation. Tracing the bursting signal is important for understanding neural state and understanding communication between individual neurons. The dopaminergic system, which projects from the VTA to other regions in the mesolimbic system, is involved in hedonia and motivation. However, the role of this system in the pathophysiology of depression and its manipulation for treatment of depression has received little attention. Inter-spike interval time series were recorded from the VTA of control Sprague-Dawley and Flinders sensitive line (FSL) rats with or without 14 days of desipramine (5 mg/kg) treatment. Comparison of the firing modes of control and desipramine-treated FSL rats reveals dissimilar patterns. Desipramine treatment normalized depressive-like behavior and elevated the dopaminergic mesolimbic activity, although not to control levels. Mesolimbic neuronal activity is known to occur either in burst or in single-spike firing mode. Herein, we suggest a third mode that is characterized as a "cluster" formed from burst and post-burst activity. A significant reduction in the activity of both bursts and cluster was detected in FSL rats, which was restored by desipramine treatment.

High-Frequency Afferent Stimulation Induces Long-Term Potentiation of Field Potentials in the Ventral Tegmental Area

Excitatory synapses on dopamine neurons in the VTA can undergo both long-term potentiation and depression. Additionally, drug-induced plasticity has been found at VTA synapses, and is proposed to play a role in reward-related learning and addiction by modifying dopamine cell firing. LTP at these synapses is difficult to generate experimentally in that it requires an undisturbed intracellular milieu and is often small in magnitude. Here, we demonstrate the induction of LTP as a property of evoked field potentials within the VTA. Excitatory field potentials were recorded extracellularly from VTA neurons in acute horizontal midbrain slices. Using extracellular and intracellular recording techniques, we found that evoked field potentials originate within the VTA itself and are largely composed of AMPA receptor-mediated EPSPs and action potentials triggered by activation of glutamatergic synapses on both dopamine and GABA neurons. High-frequency afferent stimulation (HFS) induced LTP of the field potential. The induction of this LTP was blocked by application of the NMDAR antagonist, d-APV, prior to HFS. As reported previously, glutamatergic synapses on GABA neurons did not express LTP while those on dopamine neurons did. We conclude that the potentiation of glutamatergic synapses on dopamine neurons is a major contributor to NMDA receptor-dependent LTP of the field potential. Field potential recordings may provide a convenient approach to explore the basic electrophysiological properties of VTA neurons and the development of addiction-related processes in this brain region.

Model-based reward prediction in the primate prefrontal cortex

Dopaminergic signals play a mathematically precise role in reward-related learning, and variations in dopaminergic signaling have been implicated in vulnerability to addiction. Here, we provide a detailed overview of the relationship between theoretical, mathematical, and experimental accounts of phasic dopamine signaling, with implications for the role of learning-related dopamine signaling in addiction and related disorders. We describe the theoretical and behavioral characteristics of model-free learning based on errors in the prediction of reward, including step-by-step explanations of the underlying equations. We then use recent insights from an animal model that highlights individual variation in learning during a Pavlovian conditioning paradigm to describe overlapping aspects of incentive salience attribution and model-free learning. We argue that this provides a computationally coherent account of some features of addiction.

Synaptic plasticity and drug addiction

Recent studies have suggested that the development of addictive behaviours shares common features with traditional learning models. Synaptic plasticity, a possible substrate for learning, has been demonstrated in neural reward circuits and might contribute to the learning of addictive behaviours. Changes in the strength of synaptic connections have been investigated in dopaminergic cells of the ventral tegmental area in response to several addictive drugs. Rapid and persistent forms of synaptic plasticity (specifically, long-lasting synaptic potentiation) have been demonstrated to accompany some of the behavioural effects of addictive drugs. We hypothesize that drug-induced synaptic plasticity might play a role in reward-related learning and addiction by modifying the fine tuning of dopaminergic cell firing.

Dopamine D1-like Receptors and Reward-related Incentive Learning

BENINGER, R. J., R. MILLER. Dopamine D1-like receptors and reward-related incentive learning. NEUROSCI BIOBEHAV REV 22(2) 335–345, 1998.—There now is general agreement that dopaminergic neurons projecting from ventral mesencephalic nuclei to forebrain targets play a critical role in reward-related incentive learning. Many recent experiments evaluate the role of dopamine (DA) receptor subtypes in various paradigms involving this type of learning. The first part of this paper reviews evidence from these studies that use antagonists or agonists relatively specific for D1- or D2-like receptors in operant paradigms with food, brain stimulation, self-administered stimulant or conditioned rewards or place conditioning. The focus is on studies that directly compare agents acting at the two DA receptor families, especially those studies where the agents produce differential actions. Results support the conclusion that D1-like receptors play a more critical role in reward-related learning than D2-like receptors. D1-like receptors initiate a cascade of intracellular events including cyclic adenosine monophosphate (cAMP) formation and activation of cAMP-dependent protein kinase (PKA). The final section of this paper reviews evidence from a wide range of neuroscience experiments that implicates the cAMP/PKA pathway in learning in general and in reward-related incentive learning in particular. We conclude that the molecular mechanism underlying DA-mediated incentive learning may involve DA release in association with reward, stimulation of D1-like receptors, activation of the cAMP/PKA cascade and additional intracellular events leading to modification of cortico-striatal glutamatergic synapses activated by stimuli encountered in close temporal contiguity with reward. Thus, when reward-related incentive learning takes place, it may be the action of DA acting at D1-like receptors that leads to plastic changes in the striatum that form the substrate of that learning.

The effects of systemic and intracerebral injections of D 1 and D 2 agonists on brain stimulation reward

That dopamine (DA) plays a role in reward-related learning is well documented but the mechanisms through which it acts are not well understood. The present set of experiments investigated the role of DA receptor subtypes within DA-innervated forebrain regions in brain stimulation reward (BSR). Thirty-two rats were implanted with electrodes in the ventral tegmental area (VTA) and cannulae aimed at the caudal nucleus accumbens (NAcc), the caudate-putamen (CP) or cortex. Rate-frequency functions were determined by logarithmically decreasing the number of cathodal pulses in a stimulation train from a value that sustained maximal responding to one that did not sustain responding (thresholds). After BSR thresholds stabilized rats received treatments with DA agonists and their effects on thresholds were analyzed. Systemic treatments consisted of injections of (+)-amphetamine (1.0 mg/kg, i.p., 10 min before testing), the D 2 agonist quinpirole (1.0 mg/kg, i.p., 10 min before testing), the novel D 1 agonist A-77636 (3.0 mg/kg, s.c., 90 min before testing) or their vehicle (distilled H 2O). Central treatments consisted of microinjections of quinpirole (0.3 – 10.0 μg/0.5 μl) directly into the caudal NAcc, CP or cortex or A-77636 (30 μg/0.5 μl) into the caudal NAcc or CP. Results showed that all three agonists, when injected systemically, significantly reduced the threshold frequency required for VTA BSR, indicating a potentiative effect on reward. Central injections of quinpirole in the caudal NAcc, CP or cortex produced significant increases in BSR thresholds indicative of reduced rewarding efficacy of stimulation. Central injections of A-77636 into the caudal NAcc, but not the CP, were associated with a reduction in VTA BSR thresholds, suggesting an increase in reward. These results suggest that stimulation of D 1 or D 2 receptors enhances the rewarding effect of brain stimulation. In the case of the systemic quinpirole enhancement of reward, the present results suggest that this may not occur in the caudal NAcc, CP or cortex. Finally, the present results suggest that D 1 receptor stimulation in the caudal NAcc can facilitate reward-related learning.