Schedule Of Reinforcement Research Articles

Differential Roles of Oxytocin Receptors in the Prefrontal Cortex and Nucleus Accumbens on Cocaine Self-Administration and Reinstatement of Cued Cocaine Seeking in Male Rats.

Little is known about the specific roles of cortical and accumbal oxytocin receptors in drug use disorders. To better understand the importance of the endogenous oxytocin system in cocaine relapse behavior, we developed an adeno-associated viral vector-expressing short hairpin (sh) RNAs to selectively degrade the rat oxytocin receptor (OxyR) mRNA in vivo. Male (Sprague-Dawley) rats received bilateral infusions of the shRNA for the oxytocin receptor (shOxyR) or an shRNA control virus into the prefrontal cortex (PFC) or the nucleus accumbens core (NAc). Rats self-administered cocaine on an escalating FR ratio for 14 days, lever responding was extinguished, and rats were tested for cued and cocaine-primed reinstatement of drug seeking. OxyR knockdown in the PFC delayed the acquisition of lever pressing on an fixed ratio 1 schedule of reinforcement. All rats eventually acquired the same level of lever pressing and discrimination, and there were no differences in extinction. OxyR knockdown in the NAc had no effect during acquisition. In both the PFC and NAc, the shOxyR decreased cued reinstatement relative to shRNA control virus but was without effect during drug-primed reinstatement. OxyR knockdown in the PFC increased chamber activity during a social interaction task. This study provides critical new information about how endogenous OxyRs function to affect drug seeking in response to different precipitators of relapse. The tool developed to knockdown OxyRs in rat could provide important new insights that aid development of oxytocin-based therapeutics to reduce return-to-use episodes in people with substance use disorder and other neuropsychiatric disorders.

Partial reinforcement optimizer: An evolutionary optimization algorithm

In this paper, a novel evolutionary optimization algorithm, named Partial Reinforcement Optimizer (PRO), is introduced. The major idea behind the PRO comes from a psychological theory in evolutionary learning and training called the partial reinforcement effect (PRE) theory. According to the PRE theory, a learner is intermittently reinforced to learn or strengthen a specific behavior during the learning and training process. The reinforcement patterns significantly impact the response rate and strength of the learner during a reinforcement schedule, achieved by appropriately selecting a reinforcement behavior and the time of applying reinforcement process. In the PRO algorithm, the PRE theory is mathematically modeled to an evolutionary optimization algorithm for solving global optimization problems. The efficiency of the proposed PRO algorithm is compared to well-known Meta-heuristic Algorithms (MAs) using Wilcoxon and Friedman statistical tests to analyze results from 75 benchmarks of the CEC2005, CEC2014, and CEC-BC-2017 test suits, which include unimodal, multimodal, hybrid, and composition functions. Additionally, the proposed PRO algorithm is applied to optimize a Federated Deep Learning Electrocardiography (ECG) classifier, as a real case study, to investigate the robustness and applicability of the proposed PRO. The experimental results demonstrate that the PRO algorithm outperforms existing meta-heuristic optimization algorithms by providing a more accurate and robust solution.

Acute but not long-lasting antidepressant-like effect of psilocybin in differential reinforcement of low-rate 72 schedule in rats.

In clinical studies, psychedelics including psilocybin and D-lysergic acid diethylamide (LSD) demonstrate rapid and persistent antidepressant effects. Since the effective treatment with psychedelics is usually provided with psychotherapy, it is debatable whether their prolonged efficacy can be observed in infrahuman species. Preclinical reports on psychedelics' effects most often address their acute actions, and different tests and models provide inconsistent results. The goal of this study was to examine whether the treatment with psilocybin and/or LSD would demonstrate immediate and/or sustained antidepressant-like effects in the differential reinforcement of low-rate responding (DRL) schedule in rats. In contrast to the antidepressant screening tools, the DRL 72s test is known to detect antidepressants with high predictive validity as it differentiates clinically effective antidepressants from other psychoactive drugs in non-stressed animals. Adult male Sprague Dawley rats were injected over three consecutive days with psilocybin (1 mg/kg), LSD (0.08 mg/kg), or saline and then tested in DRL 72s for the following 4 weeks. Treatment with psilocybin but not LSD demonstrated an immediate antidepressant-like effect, manifested as an increased number of reinforced presses and response efficiency. By contrast, neither of the drugs showed a long-term (up to 4 weeks following administration) antidepressant-like effect. Using DRL 72s schedule of reinforcement, we demonstrated the acute antidepressant-like effect of psilocybin but not of LSD, and failed to detect their persistent antidepressant-like efficacy. The present study suggests that the detection of long-lasting antidepressant-like activity in rats could be challenging and may require entirely novel behavioral methods.

Updated Recommendations for Reinforcement Schedule Thinning following Functional Communication Training.

Schedule thinning is a necessary treatment procedure following the acquisition of a communication response during functional communication training. In this article, we update and extend the Hagopian et al. Behavior Analysis in Practice, 4, 4-16, (2011) review and recommendations on schedule-thinning procedures following functional communication training. Since their publication, substantial research has been published on the efficacy, efficiency, and social validity of schedule-thinning methods. We provide updated recommendations for schedule thinning based on contemporary literature that has been published since 2011, as well as discuss key areas for future research.

Using synchronous reinforcement to increase mask wearing in young children: A replication and extension

AbstractIn the current study, experimenters implemented synchronous schedules of reinforcement to increase mask wearing for up to 30 min for six children under the age of 5 years. Additionally, for a subset of children, we evaluated whether mask wearing would continue under baseline conditions in their classroom with staff during 30 min sessions (treatment extension), and later throughout the day (all‐day probes). Results showed the intervention increased mask wearing for all children for up to 30 min. Additionally, treatment‐extension sessions and all‐day probes, conducted with some children, showed mask wearing maintained in their classroom with staff.

Clozapine N-oxide, compound 21, and JHU37160 do not influence effortful reward-seeking behavior in mice

RationaleClozapine N-oxide (CNO) has been developed as a ligand to selectively activate designer receptors exclusively activated by designer drugs (DREADDs). However, previous studies have revealed that peripherally injected CNO is reverse-metabolized into clozapine, which, in addition to activating DREADDs, acts as an antagonist at various neurotransmitter receptors, suggesting potential off-target effects of CNO on animal physiology and behaviors. Recently, second-generation DREADD agonists compound 21 (C21) and JHU37160 (J60) have been developed, but their off-target effects are not fully understood.ObjectivesThe present studies assessed the effect of novel DREADD ligands on reward-seeking behavior.MethodsWe first tested the possible effect of acute i.p. injection of low-to-moderate (0.1, 0.3, 1, 3 mg/kg) of CNO, C21, and J60 on motivated reward-seeking behavior in wild-type mice. We then examined whether a high dose (10 mg/kg) of these drugs might be able to alter responding.ResultsLow-to-moderate doses of all drugs and a high dose of CNO or C21 did not alter operant lick responding for a reward under a progressive ratio schedule of reinforcement, in which the number of operant lick responses to obtain a reward increases after each reward collection. However, high-dose J60 resulted in a total lack of responding that was later observed in an open field arena to be due to a sedative effect.ConclusionsThis study provides definitive evidence that commonly used doses of CNO, C21, and J60 have negligible off-target effects on motivated reward-seeking but urges caution when using high doses of J60 due to sedative effects.

Conscious expectancy rather than associative strength elicits brain activity during single-cue fear conditioning.

The neurocognitive processes underlying Pavlovian conditioning in humans are still largely debated. The conventional view is that conditioned responses (CRs) emerge automatically as a function of the contingencies between a conditioned stimulus (CS) and an unconditioned stimulus (US). As such, the associative strength model asserts that the frequency or amplitude of CRs reflects the strength of the CS-US associations. Alternatively, the expectation model asserts that the presentation of the CS triggers conscious expectancy of the US, which is responsible for the production of CRs. The present study tested the hypothesis that there are dissociable brain networks related to the expectancy and associative strength theories using a single-cue fear conditioning paradigm with a pseudo-random intermittent reinforcement schedule during functional magnetic resonance imaging. Participants' (n = 21) trial-by-trial expectations of receiving shock displayed a significant linear effect consistent with the expectation model. We also found a positive linear relationship between the expectancy model and activity in frontoparietal brain areas including the dorsolateral prefrontal cortex (PFC) and dorsomedial PFC. While an exploratory analysis found a linear relationship consistent with the associated strength model in the insula and early visual cortex, our primary results are consistent with the view that conscious expectancy contributes to CRs.

A new formulation of dezocine, Cyc-dezocine, reduces oxycodone self-administration in female and male rats

Dezocine is a partial mu opioid receptor agonist previously used as an analgesic for perioperative acute pain in the US and is now the most used perioperative analgesic in China. In general, dezocine is well-tolerated, with relatively minimal risk of fatal respiratory depression. To our knowledge, there are no reports of dezocine addiction, which suggests that the abuse liability of dezocine is low. The overarching goal of this study was to determine the efficacy of a novel formulation of dezocine (Cyc-dezocine), developed for intraperitoneal or intranasal administration, to reduce voluntary opioid taking in rats. One cohort of male rats self-administered intravenous oxycodone on a fixed-ratio 5 schedule of reinforcement. Once oxycodone taking stabilized, rats were pretreated with systemic injections of vehicle or Cyc-dezocine. Cyc-dezocine dose-dependently reduced intravenous oxycodone self-administration. A second cohort of male and female rats self-administered oral oxycodone from drinking water. Once oxycodone taking stabilized, rats were pretreated with intra-nasal Cyc-dezocine. Consistent with the effects of i.p. Cyc-dezocine in our intravenous oxycodone studies, intra-nasal Cyc-dezocine attenuated oral oxycodone self-administration. Together, these findings support the need for further studies investigating the therapeutic potential of Cyc-dezocine for treating opioid use disorder.

Modulation of cue value and the augmentation of heroin seeking in chronically food-restricted male rats under withdrawal

Food restriction augments drug seeking in abstinent rats. The underlying motivational mechanisms, however, remain unclear. We hypothesized that caloric restriction enhances the incentive value attributed to drug-associated cues and, in turn, augments drug seeking. Male rats were trained to lever-press for heroin, and then moved to the animal colony for a forced-abstinence period. Rats were maintained on free access to food (Sated) or subjected to 14 days of food restriction (FDR). In a series of experiments, we assessed the effect of food-restriction on the incentive value of heroin-associated cues. Tests included performance under a progressive ratio (PR) schedule of reinforcement maintained by heroin-associated cues, acquisition of a novel operant response reinforced by drug-associated cues, effect of food-restriction on operant response reinforced by neutral cues, acquisition of a novel operant response reinforced by drug-associated or neutral cues, and the effect of food-restriction on operant response reinforced by drug-associated or neutral cues, under a discrete choice procedure.Food-restriction did not change breakpoints in PR maintained by heroin-associated cues. FDR rats acquired the novel response at a greater level compared to the Sated group. Food-restriction-induced increase in novel-response rate was observed for both heroin-paired and the neutral cue. Responding for a heroin-associated cue was greater than for the neutral cue in both Sated and FDR groups. Response rate for the neutral cue, however, was greater in the FDR versus Sated group.Our findings suggest that food restriction increases the conditioned motivational properties of environmental stimuli, including, but not exclusive to, heroin-paired cues.

A comparison of the reinforcing strength of cocaethylene and cocaine in monkeys responding under progressive-ratio and concurrent choice schedules of reinforcement

BackgroundIndividuals who use cocaine have high rates of co-morbid alcohol use and when ethanol and cocaine are administered concurrently, the metabolite cocaethylene is formed. Cocaethylene is equipotent to cocaine in blocking dopamine reuptake and substitutes for cocaine in drug discrimination studies. However, no previous work has directly compared the reinforcing strength of cocaine to cocaethylene. MethodsIn Experiment 1, three individually-housed adult male rhesus macaques self-administer cocaine under a progressive-ratio (PR) schedule of reinforcement, during daily 4-hr sessions. Under this schedule, the primary dependent variable is the number of injections received, or the break point (BP). Saline, cocaine (0.001–0.3mg/kg/injection) and cocaethylene (0.0003–0.1mg/kg/injection) dose-response curves were determined. In Experiment 2, two female cynomolgus and one rhesus macaque responded under a concurrent schedule of drug (cocaine or cocaethylene) vs. 1.0-g banana-flavored food pellets, during daily 1-hr sessions. ResultsBoth cocaine and cocaethylene functioned as reinforcers under the PR and concurrent choice schedules of reinforcement. Under the PR schedule, peak BPs were not significantly different, nor were ED50 values on the ascending limb, suggesting that cocaethylene has equal reinforcing strength and potency to cocaine. Under the concurrent drug-food choice procedure, cocaethylene was also equally potent to cocaine. ConclusionsUnder two schedules of reinforcement designed to assess reinforcing strength, cocaethylene and cocaine were equipotent and of equal reinforcing strength. Because cocaethylene has a longer duration of action, it is important for studies designed to evaluate treatments for cocaine use to also consider the effects of these interventions on cocaethylene.

Food Restriction Level and Reinforcement Schedule Differentially Influence Behavior during Acquisition and Devaluation Procedures in Mice.

Behavioral strategies are often classified based on whether reinforcer value controls reinforcement. Value-sensitive behaviors, in which animals update their actions when reinforcer value is changed, are classified as goal-directed; conversely, value-insensitive actions, where behavior remains consistent when the reinforcer is removed or devalued, are considered habitual. Basic reinforcement schedules can help to bias behavior toward either process: random ratio (RR) schedules are thought to promote the formation of goal-directed behaviors while random intervals (RIs) promote habitual control. However, how the schedule-specific features of these tasks interact with other factors that influence learning to control behavior has not been well characterized. Using male and female mice, we asked how distinct food restriction levels, a strategy often used to increase task engagement, interact with RR and RI schedules to control performance during task acquisition and devaluation procedures. We determined that food restriction level has a stronger effect on the behavior of mice following RR schedules compared with RI schedules, and that it promotes a decrease in response rate during devaluation procedures that is best explained by the effects of extinction rather than devaluation. Surprisingly, food restriction accelerated the decrease in response rates observed following devaluation across sequential extinction sessions, but not within a single session. Our results support the idea that the relationships between schedules and behavioral control strategies are not clear-cut and suggest that an animal's engagement in a task must be accounted for, together with the structure of reinforcement schedules, to appropriately interpret the cognitive underpinnings of behavior.

Cue-induced reinstatement of seeking behavior in male rats is independent from the rewarding value of the primary reinforcer: Effect of mGluR5 blockade

Environmental conditioning factors have a profound impact on alcohol-seeking behavior and the maintenance of alcohol use in individuals with alcohol dependence. Cues associated with alcohol, depending on the perceived value of the primary reinforcer, gain salience and can trigger relapse.This study investigates the correlation between the reward magnitude of the primary reinforcer and the reinstatement evoked by cues predictive of their availability in male rats. Rat self-administration procedures were used to test reinstatement, with reinforcers consisting of 10% alcohol, 10% sucrose, or 2% sodium chloride (NaCl) experienced under need-state conditions. The effect of MTEP ([(2-methyl-1,3-thiazol-4-yl) ethynyl] pyridine), a selective metabotropic glutamate receptor 5 (mGluR5) antagonist, on motivation and reinstatement behaviors was also evaluated. Resultsdemonstrate that under Fixed Ratio 1 (FR1) schedule, the three reinforcers maintain operant responding with the following order of magnitude 10% sucrose >2% NaCl >10% alcohol > water. Under a progressive ratio (PR) schedule of reinforcement, rats exhibit a significantly higher breakpoint for 2% NaCl (under Na-depletion), followed by 10% sucrose and 10% alcohol. After extinction, a significant reinstatement is observed with the magnitude order of 10% sucrose >10% alcohol >2% NaCl. However, only re-exposure to alcohol-paired cues induced significant reinstatement of alcohol-seeking after 4 and 8 months. Treatment with MTEP significantly reduces reinstatement of responding across all reinforcers, with the strongest effect observed on alcohol-seeking. These findings suggest that mGluR5 plays a general role in controlling cue-reactivity, but the effect is prominent in the case of alcohol compared to natural rewards.In conclusion, the results demonstrate a remarkable dissociation between the rewarding magnitude of the primary reinforcer and its ability to trigger relapse upon presentation of a cue previously associated with it. Importantly, alcohol, despite having lower intrinsic motivational value compared to a natural reward (sucrose) or a consummatory stimulus experienced under need state conditions (NaCl), can elicit more robust and longer-term reinstatement of seeking responses. Finally, our data demonstrate a significant involvement of the mGluR5 system in the regulation of seeking behavior.

Environmental enrichment accelerates the acquisition of schedule-induced drinking in rats

Environmental enrichment (EE) provides an improvement in the housing conditions of experimental animals, such as laboratory rats, with greater physical and social stimulation through toys and company in the home cages. Its use is known to influence performance of experimental protocols, but these effects have not been well determined in the schedule-induced drinking (SID) procedure. The main goal of this study was to investigate the effects of EE on the acquisition of SID in 24 12-week-old male Wistar rats, divided into two groups, a group with EE housed with toys and companions, and a group without enrichment in individual housing conditions without toys (social isolation and no environmental enrichment, INEE). A total of 25 sessions, under a fixed time 30 s food reinforcement schedule and with access to water in the experimental chambers were carried out. Sessions lasted 30 min. The results showed that the EE group developed faster the excessive drinking pattern of SID, and drank to higher levels, than the INEE group. The greater development of SID in the EE group contradicts the view of schedule-induced behavior as linked to stress reduction and better suits with the conception of induction related to positive reinforcement.

No influence of threat uncertainty on fear generalization.

Fear overgeneralization and perceived uncertainty about future outcomes have been suggested as risk factors for clinical anxiety. However, little is known regarding how they influence each other. In this study, we investigated whether different levels of threat uncertainty influence fear generalization. Three groups of healthy participants underwent a differential fear conditioning protocol followed by a generalization test. All groups learned to associate one female face (conditioned stimulus, CS+) with a female scream (unconditioned stimulus, US), whereas the other face (CS-) was not associated with the scream. In order to manipulate threat uncertainty, one group (low uncertainty, n = 26) received 80%, the second group (moderate uncertainty, n = 32) received 60%, and the third group (high uncertainty, n = 30) 40% CS-US contingency. In the generalization test, all groups saw CS+ and CS- again along with four morphs resembling the CSs in steps of 20%. Subjective (expectancy, valence, and arousal ratings), psychophysiological (skin conductance response, SCR), and visuocortical (steady-state visual evoked potentials, ssVEPs) indices of fear were registered. Participants expected the US according to their reinforcement schedules and the discriminative responses to CS+/CS- increased with more uncertainty in skin conductance. However, acquisition of conditioned fear was not evident in ssVEPs. During the generalization test, we found no effect of threat uncertainty in any of the measured variables, but the strength of generalization for threat expectancy ratings was positively correlated with dispositional intolerance of uncertainty. This study suggests that mere threat uncertainty does not modulate fear generalization.

Chronic administration of a norepinephrine antagonist prevents and partially reverses escalation of cocaine self-administration.

Anxiety is a critical component of the development and maintenance of drug addiction; however, anti-anxiety medications such as benzodiazepines and beta-blockers (β-adrenergic receptor antagonists) are not used for the treatment of substance use disorder, except for the management of acute withdrawal syndrome. Preclinical studies have shown that beta-blockers may reduce stress-induced relapse; however, the effect of beta blockers on the escalation and maintenance of drug intake has not been tested. To address this issue, we chronically administered the β-adrenergic receptor antagonist propranolol during the escalation or maintenance of cocaine intake in a model of extended access (6h) to cocaine self-administration (0.5mg/kg). The behavioural specificity of propranolol was tested using a non-drug reward (saccharin). Daily administration of propranolol (15 mg/kg) prevented the development of escalation of cocaine self-administration and partially reversed self-administration after the establishment of escalation of intake. Moreover, propranolol dose-dependently decreased the motivation for cocaine tested under a progressive ratio schedule of reinforcement during the development of escalation and after maintenance. Finally, propranolol administration had no effect on the escalation and maintenance of saccharin self-administration. These results demonstrate that chronic treatment with propranolol provides therapeutic efficacy in reducing cocaine self-administration during the development and after the establishment of escalation of cocaine self-administration in an animal model relevant to cocaine use disorder. These results suggest that beta blockers should be further investigated as a target for medication development for the treatment of cocaine use disorder.

Functional communication training and a chained schedule of reinforcement to reduce escape‐maintained challenging behavior related to a daily routine

AbstractChildren with autism spectrum disorder often exhibit challenging behavior to avoid exposure to common daily tasks or routines. There is a need to investigate methods to teach appropriate communication to replace challenging behavior and increase tolerance for daily routines. However, the data to support its use to decrease challenging behavior associated with an activity or routine is limited. In the present study, the authors evaluated the use of parent‐implemented functional communication training (FCT) with a modified chained schedule of reinforcement to reduce challenging behavior associated with a nonpreferred daily routine. One 6‐year‐old male diagnosed with autism participated in this study. The functional analysis indicated that he engaged in challenging behavior to terminate throwing trash away. The authors then conducted a treatment evaluation to assess the efficacy of the FCT with a modified chained schedule of reinforcement in reducing challenging behavior and increasing the number of items that could be thrown in the trash. Results indicated that FCT with a modified chained schedule of reinforcement was effective in decreasing challenging behavior.

PYY3-36 infused systemically or directly into the VTA attenuates fentanyl seeking in male rats

More effective treatments for fentanyl use disorder are urgently needed. An emerging literature indicates that glucagon-like peptide-1 receptor (GLP-1R) agonists attenuate voluntary opioid taking and seeking in rodents. However, GLP-1R agonists produce adverse malaise-like effects that may limit patient compliance. Recently, we developed a dual agonist of GLP-1Rs and neuropeptide Y2 receptors (Y2Rs) that attenuates fentanyl taking and seeking at doses that do not produce malaise-like effects in opioid-experienced rats. Whether activating Y2Rs alone is sufficient to reduce opioid taking and seeking, however, is not known. Here, we investigated the efficacy of the Y2R ligand PYY3-36 to reduce fentanyl self-administration and the reinstatement of fentanyl-seeking behavior, a model of relapse in humans. Male rats were allowed to self-administer fentanyl (2.5 μg/kg, i.v.) for 21 days on a fixed-ratio 5 (FR5) schedule of reinforcement. Rats were then pretreated with vehicle or PYY3-36 (50 μg/kg s.c.; 0.1 and 1.0 μg/100 nL intra-VTA) prior to fentanyl self-administration test sessions. There were no effects of systemic or intra-VTA PYY3-36 on intravenous fentanyl self-administration. Opioid taking was then extinguished. Prior to subsequent reinstatement test sessions, rats were pretreated with vehicle or PYY3-36 (50 μg/kg s.c.; 0.1 and 1.0 μg/100 nL intra-VTA). Both systemic and intra-VTA administration of PYY3-36 attenuated fentanyl reinstatement in male rats at doses that did not affect food intake or produce adverse malaise-like effects. These findings indicate that Y2R agonism alone is sufficient to decrease fentanyl-seeking behavior during abstinence in opioid-experienced rats and further support strategies aimed at targeting Y2Rs for treating opioid use disorders.

Isradipine, an L-type calcium channel inhibitor, attenuates cue-associated methamphetamine-seeking in mice

Methamphetamine (meth) is an addictive psychostimulant and there are no FDA-approved treatment options for patients suffering from meth use disorders. In addition to being addictive, meth is also neurotoxic and chronic administration results in degeneration of substantia nigra pars compacta (SNc) dopamine and locus coeruleus (LC) norepinephrine neurons in mice. Optimal treatment strategies for meth use disorders would attenuate maladaptive meth-seeking behavior as well as provide neuroprotection. The L-type calcium channel inhibitor isradipine and the monoamine oxidase (MAO) inhibitor rasagiline both prevent chronic meth-induced SNc and LC degeneration but effects on meth-seeking are unknown. To test whether these clinically available compounds can mitigate meth-seeking, mice were implanted with chronic indwelling jugular vein catheters and allowed to self-administer meth (0.1 mg/kg/infusion) for 10 consecutive days (2-hrs/day) on a fixed ratio (FR) 1 schedule of reinforcement with meth infusions paired to a cue light. One day after the last self-administration session mice were tested for cue-associated meth-seeking behavior wherein the meth-associated cue light was contingently presented but meth reinforcement withheld. Isradipine (3 mg/kg) attenuated cue-associated meth-seeking in both male and female mice. In contrast, rasagiline (1 mg/kg) had no effect on seeking in either sex. These results suggest that isradipine may have the potential to serve as a dual-purpose pharmacotherapy for meth use disorders by attenuating seeking behavior and providing neuroprotection.

Unpredictable, intermittent access to sucrose or water promotes increased reward pursuit in rats

Reward uncertainty can sensitize reward pathways, promoting increased reward-seeking and -taking behaviours. This is relevant to human conditions such as pathological gambling, eating disorders and drug addiction. In the context of addiction, preclinical self-administration procedures have been developed to model the intermittency of human drug use. These intermittent-access (IntA) procedures involve intermittent but predictable access to drug during self-administration sessions. However, human drug use typically involves intermittent and unpredictable drug access. We introduce a new procedure modeling unpredictable, intermittent access (UIntA) to a reinforcer, and we compare it to procedures that provide predictable reinforcer availability; continuous (ContA) or intermittent (IntA) access. Female rats self-administered water or liquid sucrose in daily hour-long sessions. IntA and ContA rats had access to a fixed volume of water or sucrose (0.1 ml), under a fixed ratio 3 schedule of reinforcement. IntA rats had predictable 5-min reinforcer ON and 25-min reinforcer OFF periods. ContA rats had 60 min of reinforcer access during each session. For UIntA rats, variation in the length of ON and OFF periods (1, 5 or 9 min/period), response requirement (variable ratio 3 schedule of reinforcement), reinforcer probability (50%) and quantity (0, 0.1 or 0.2 ml) introduced reward uncertainty. Following 14 daily self-administration sessions, UIntA rats showed the highest levels of responding for water or sucrose under progressive ratio conditions, responding under extinction conditions, and cue-induced reinstatement of sucrose seeking. Thus, unpredictable, intermittent reward access promotes increased reward pursuit. This has implications for modeling addiction and other disorders of increased reward seeking.

The motivational role of the ventral striatum and amygdala in learning from gains and losses.

The ventral striatum (VS) and amygdala are two structures often implicated as essential structures for learning. The literature addressing the contribution of these areas to learning, however, is not entirely consistent. We propose that these inconsistencies are due to learning environments and the effect they have on motivation. To differentiate aspects of learning from environmental factors that affect motivation, we ran a series of experiments with varying task factors. We compared monkeys (Macaca mulatta) with VS lesions, amygdala lesions, and unoperated controls on reinforcement learning (RL) tasks that involve learning from both gains and losses as well as from deterministic and stochastic schedules of reinforcement. We found that for all three groups, performance varied by experiment. All three groups modulated their behavior in the same directions, to varying degrees, across the three experiments. This behavioral modulation is why we find deficits in some experiments, but not others. The amount of effort animals exhibited differed depending on the learning environment. Our results suggest that the VS is important for the amount of effort animals will give in rich deterministic and relatively leaner stochastic learning enivornments. We also showed that monkeys with amygdala lesions can learn stimulus-based RL in stochastic environments and environments with loss and conditioned reinforcers. These results show that learning environments shape motivation and that the VS is essential for distinct aspects of motivated behavior. (PsycInfo Database Record (c) 2023 APA, all rights reserved).