Schedule Of Reinforcement Research Articles

Contingency management for monosubstance use disorders: Systematic review and assessment of predicted versus obtained effects.

Contingency management (CM) is notably successful as a substance use disorder treatment and is most effective when targeting monosubstance use. Evidence suggests the effects of CM exceed predictions based on the value of the incentives delivered for monosubstance abstinence. In this systematic review, we examine common variations of CM interventions applied to a single substance to determine what factors may contribute to the larger effect. Our results show that CM produced moderate to large effect sizes when single drugs were targeted, with stable effects over time. We also found that interventions targeting cocaine abstinence overwhelmingly outperformed their predicted effect, whereas interventions for smoking cessation did not. Thus, incentives alone may not account for the success of CM, at least when applied to stimulant use disorder. We propose other potential sources of the effect including social reinforcement and the specific parameters of the reinforcement schedule.

Caregiver-Implemented Interventions to Improve Daily Living Skills for Individuals With Developmental Disabilities: A Systematic Review.

Many individuals with intellectual and developmental disabilities (DD) encounter difficulties in performing daily living skills, which limits their self-sufficiency and autonomy. Caregivers, such as parents, are often the individuals who interact with individuals with DD the most during daily living skills routines. Therefore, it is critical to teach caregivers to implement daily living skills interventions with their children. A systematic review of the literature related to caregiver-implemented daily living skills interventions yielded 38 articles. Of those, 20 articles, with 21 experiments, were rated as high quality based on What Works Clearinghouse design standards. Behavior skill training was the most common strategy used to teach caregivers to implement interventions. Common daily living skills interventions included prompting, reinforcement, and task analysis. The most common form of prompting was verbal prompting, and the most common schedule of reinforcement was a continuous schedule of reinforcement. Of the 21 high-quality experiments, 86% demonstrated caregiver-implemented interventions yielded improvements in daily living skills. Several studies demonstrated generalization and maintenance of daily living skills. Implications for research and practice are discussed.

Delineating anticipatory arousal from reward consumption: evaluating fixed-intervals in cocaine seeking-taking response chains.

Anticipation is a critical antecedent to drug use, in which the prospect of imminent drug availability can potently motivate instrumental actions directed to procure it. Models that capture the behavioral dynamics that precede drug access may allow for the dissociation of key neural mechanisms underlying appetitive or consummatory processes in drug self-administration. We aimed to isolate measurements attributed to the procurement and consumption of a reward by defining distinct actions for each using a chain-schedule of reinforcement. Male Long-Evans rats were trained to self-administer cocaine or saccharin under a chained schedule of reinforcement (FI-FR) in order to dissociate appetitive ('seeking') from consummatory ('taking') behaviors. Completion of a fixed-interval (5min) was followed by 5min of continuously reinforced responding (FR1) on another lever. The FI-FR chain procedure appears to provide sensitive and dissociable dimensions of cocaine self-administration within a single experimental session. Importantly, we demonstrate that responding during the FI (i.e., seeking) link tracks with the incentive value of anticipated reward access - whereby response rates corresponded to expected reward magnitude, degree of reward-specific satiety, and general motivational state. The FI component is a sensitive and reliable index of motivational changes induced by either the extrinsic incentive value of reinforcement (i.e., anticipated dose) or intrinsic motive states (i.e., satiety or deprivation). This procedure provides a valuable tool for interrogating the neural dynamics of drug-seeking and -taking behavior, in isolation.

Demand management of plug-in electric vehicle charging station considering bidirectional power flow using deep reinforcement learning

The recent development in plug-in electric vehicle technology has increased its popularity. Plug-in electric vehicles are widely used for their environment-friendly nature and they contribute to the reduction in global warming. With the increasing number of plug-in electric vehicles, charging coordination becomes essential for managing the charging station demand. The random charging behaviour of plug-in electric vehicles makes it a difficult task. This paper proposes a novel demand management strategy of charging stations. The proposed strategy supports the grid and reduces its burden during peak load. Deep-Q network based deep reinforcement learning is used in the proposed strategy. It is a value based deep reinforcement learning algorithm that approximates the Q value function using deep neural network. The deep reinforcement schedules the charging and discharging of plug-in electric vehicles to optimize the cost and manage the charging station load. Deep reinforcement learning enhances charging coordination by dynamically optimizing charging schedules according to real-time conditions and user preferences, thereby increasing efficiency and better integration with the grid. The reward function in deep reinforcement learning is designed based on the power price and demand of the charging station. A discount factor is introduced based on the service time to make the charging coordination efficient. A case study using dynamic pricing is carried out to validate the proposed strategy. The results prove that the proposed strategy optimizes charging and discharging costs and manages charging station demand efficiently. It is also observed that the proposed strategy is fast and incurs less computational cost.

Selective dopamine D3 receptor partial agonist (±)VK4-40 reduces the reinforcing strength of d-amphetamine but not cocaine in rhesus monkeys responding under a progressive-ratio schedule of reinforcement

Selective dopamine D3 receptor partial agonist (±)VK4-40 reduces the reinforcing strength of d-amphetamine but not cocaine in rhesus monkeys responding under a progressive-ratio schedule of reinforcement

The impact of voluntary wheel-running exercise on hippocampal neurogenesis and behaviours in response to nicotine cessation in rats.

The literature indicates that nicotine exposure or its discontinuation impair adult hippocampal neurogenesis in rats, though the impact of exercise on this process remains unclear. We have previously shown that disturbances in the number of doublecortin (DCX, a marker of immature neurons)-positive (DCX+) cells in the dentate gyrus (DG) of the hippocampus during nicotine deprivation may contribute to a depression-like state in rats. This study aimed to investigate the effect of running on hippocampal neurogenesis, depression-like symptoms, and drug-seeking behaviour during nicotine deprivation. The rats were subjected to nicotine (0.03mg/kg/inf) self-administration via an increasing schedule of reinforcement. After 21 sessions, the animals entered a 14-day abstinence phase during which they were housed in either standard home cages without wheels, cages equipped with running wheels, or cages with locked wheels. Wheel running increased the number of Ki-67+ and DCX+ cells in the DG of both nicotine-deprived and nicotine-naive rats. Wheel-running exercise evoked an antidepressant effect on abstinence Day 14 but had no effect on nicotine-seeking behaviour on abstinence Day 15 compared to rats with locked-wheel access. In summary, long-term wheel running positively affected the number of immature neurons in the hippocampus, which corresponded with an antidepressant response in nicotine-weaned rats. One possible mechanism underlying the positive effect of running on the affective state during nicotine cessation may be the reduction in deficits in DCX+ cells in the hippocampus.

Between Scylla and Charybdis: Fixed-Ratio Avoidance Response Effort and Unavoidable Shock Extinction in Humans

Avoidance of potential threat may become maladaptive when it is indiscriminate and resistant to change. Here, we investigated the resistance to change of high and low avoidance response effort when avoidance extinction involved unavoidable presentations of the aversive event (shock) in humans. Following fear conditioning, participants prevented upcoming shock delivery by responding on high (i.e., fixed ratio, FR-20) and low (FR-5) negative reinforcement schedules. Next, noneliminable shock was used for an avoidance extinction procedure whereby responding was followed by, rather than prevented, shock. During a subsequent standard extinction and response prevention test phase, we found that High effort (FR-20) avoidance would be more readily extinguished than Low effort (FR-5) avoidance. It was also predicted that fear, threat expectancy, and psychophysiological (skin conductance) responses would decrease on avoidable trials and increase on unavoidable trials before extinguishing to low levels. It was found that in the final extinction re-test phase when avoidance was possible, responding increased, particularly for low effort cues. Both fear and expectancy remained high. Individual differences on clinically relevant measures of trait anxiety, intolerance of uncertainty and experiential avoidance were associated with greater levels of fear and threat expectancy. Overall, unavoidable shock extinction may hold promise for further translational investigations of avoidance learning, extinction, and clinical treatment development.

The role of schedules of reinforcement in the experimental analysis of social behavior

The role of schedules of reinforcement in the experimental analysis of social behavior

Alternative Reinforcers Enhance the Effects of Opioid Antagonists, but Not Agonists, on Oxycodone Choice Self-Administration in Nonhuman Primates.

Clinical reports suggest that the most effective strategies for managing opioid use disorder comprise a comprehensive treatment program of both pharmacological and nonpharmacological approaches. However, the conditions under which these combinations are most effective are not well characterized. This study examined whether the presence of an alternative reinforcer could alter the efficacy of Food and Drug Administration-approved opioid antagonist or agonist medications, as well as the nonopioid flumazenil, in decreasing oxycodone choice self-administration in nonhuman primates. Adult squirrel monkeys (n = 7; four females) responded under concurrent second-order fixed-ratio (FR)-3(FR5:S);TO45s schedules of reinforcement for intravenous oxycodone (0.1 mg/kg) or saline on one lever and 30% sweetened condensed milk or water on the other. Doses of naltrexone (0.00032-1.0 mg/kg), nalbuphine (0.32-10 mg/kg), buprenorphine (0.0032-0.032 mg/kg), methadone (0.32-1.0 mg/kg), or flumazenil (1-3.2 mg/kg) were administered intramuscularly prior to oxycodone self-administration sessions that occurred with either milk or water as the alternative. Naltrexone, a μ-opioid receptor antagonist, was >30-fold more potent when milk was available compared with water and abolished oxycodone intake (injections/session) while concomitantly increasing milk deliveries at the highest dose tested. Pretreatment with the low-efficacy μ-agonist nalbuphine was most effective in the presence of milk compared with water, decreasing oxycodone preference to <50% of control values. The higher efficacy μ-agonists, methadone and buprenorphine, and the benzodiazepine antagonist flumazenil did not appreciably alter the reinforcing potency of oxycodone under either condition. These results suggest that antagonist medications used in combination with alternative reinforcers may be an effective strategy to curtail opioid abuse-related behaviors. SIGNIFICANCE STATEMENT: Clinical treatment programs for opioid use disorder use a combination of pharmacological and nonpharmacological approaches. However, the conditions under which these combinations are most effective have not been fully characterized. This study examined whether the effectiveness of μ-opioid medications to decrease oxycodone self-administration is altered in the presence of an alternative reinforcer. The results suggest that alternative reinforcers enhance the effects of antagonist or low-efficacy partial agonists, suggesting they may be a more effective strategy to curtail opioid use.

Using Programmed Schedules of Reinforcement to Increase the Variability of Reinforcer Delivery in Classroom Assistant-Implemented Variable-Ratio Schedules

Using Programmed Schedules of Reinforcement to Increase the Variability of Reinforcer Delivery in Classroom Assistant-Implemented Variable-Ratio Schedules

Negative allosteric modulation of CB1 cannabinoid receptor signalling decreases intravenous morphine self-administration and relapse in mice.

The endocannabinoid system interacts with the reward system to modulate responsiveness to natural reinforcers, as well as drugs of abuse. Previous preclinical studies suggested that direct blockade of CB1 cannabinoid receptors (CB1R) could be leveraged as a potential pharmacological approach to treat substance use disorder, but this strategy failed during clinical trials due to severe psychiatric side effects. Alternative strategies have emerged to circumvent the side effects of direct CB1 binding through the development of allosteric modulators. We hypothesized that negative allosteric modulation of CB1R signalling would reduce the reinforcing properties of morphine and decrease behaviours associated with opioid misuse. By employing intravenous self-administration in mice, we studied the effects of GAT358, a functionally-biased CB1R negative allosteric modulator (NAM), on morphine intake, relapse-like behaviour and motivation to work for morphine infusions. GAT358 reduced morphine infusion intake during the maintenance phase of morphine self-administration under a fixed ratio 1 schedule of reinforcement. GAT358 also decreased morphine-seeking behaviour after forced abstinence. Moreover, GAT358 dose dependently decreased the motivation to obtain morphine infusions under a progressive ratio schedule of reinforcement. Strikingly, GAT358 did not affect the motivation to work for food rewards in an identical progressive ratio task, suggesting that the effect of GAT358 in decreasing opioid self-administration was reward specific. Furthermore, GAT58 did not produce motor ataxia in the rotarod test. Our results suggest that CB1R NAMs reduced the reinforcing properties of morphine and could represent a viable therapeutic route to safely decrease misuse of opioids.

Kv7 channel opener retigabine reduces self-administration of cocaine but not sucrose in rats.

The increasing rates of drug misuse highlight the urgency of identifying improved therapeutics for treatment. Most drug-seeking behaviours that can be modelled in rodents utilize the repeated intravenous self-administration (SA) of drugs. Recent studies examining the mesolimbic pathway suggest that Kv7/KCNQ channels may contribute to the transition from recreational to chronic drug use. However, to date, all such studies used noncontingent, experimenter-delivered drug model systems, and the extent to which this effect generalizes to rats trained to self-administer drugs is not known. Here, we tested the ability of retigabine (ezogabine), a Kv7 channel opener, to regulate instrumental behaviour in male Sprague Dawley rats. We first validated the ability of retigabine to target experimenter-delivered cocaine in a conditioned place preference (CPP) assay and found that retigabine reduced the acquisition of place preference. Next, we trained rats for cocaine-SA under a fixed-ratio or progressive-ratio reinforcement schedule and found that retigabine pretreatment attenuated the SA of low to moderate doses of cocaine. This was not observed in parallel experiments, with rats self-administering sucrose, a natural reward. Compared with sucrose-SA, cocaine-SA was associated with reductions in the expression of the Kv7.5 subunit in the nucleus accumbens, without alterations in Kv7.2 and Kv7.3. Therefore, these studies reveal a reward-specific reduction in SA behaviour and support the notion that Kv7 is a potential therapeutic target for human psychiatric diseases with dysfunctional reward circuitry.

Use of Transdermal Alcohol Sensors in Conjunction With Contingency Management to Reduce Alcohol Consumption in People With Alcohol Dependence Attending Alcohol Treatment Services: Protocol for a Pilot Feasibility Randomized Controlled Trial.

Wearable technology for objective, continuous, and reliable alcohol monitoring has been developed. These are known as transdermal alcohol sensors (TASs). They can be worn on the wrist or ankle with the sensor pressed against the skin and can measure sweat vapors being emitted from the skin, to record transdermal alcohol concentration (TAC). Previous studies have investigated the accuracy and acceptability of the available TAS brands, but there has been little research into their use in people with alcohol use disorders (AUD). This feasibility randomized controlled trial aims to explore the feasibility, strengths, and limitations of using a TAS to monitor alcohol consumption in individuals in treatment for AUD with or without contingency management (CM) to promote abstinence or low-level alcohol consumption. The target sample size is 30 (15 randomized to each group). Participants will be recruited through poster adverts at alcohol services. Both groups (control and CM) will wear the TAS (BACtrack Skyn) for 2 weeks in the context of their usual treatment, meeting with the researcher every other weekday. In the last meeting, the participants will complete a postwear survey on their experience of wearing the TAS. The CM group will also receive small financial incentives for low or no alcohol consumption, as measured by the TAS. On days where the TAC peak is below a set threshold (<115.660 g/L), CM group participants will be rewarded with a £5 (US $6.38) voucher. There are financial bonuses if this target is achieved on consecutive days. The researcher will monitor TAC for each day of the study at each research visit and allocate financial incentives to participants according to a set reinforcement schedule. The first participant was enrolled in June 2023, and the last in December 2023. Data analysis is underway and is estimated to be completed by June 2024. A total of 32 participants were enrolled. Most TAS brands have had limited application in clinical settings, and most studies have included healthy adults rather than people with AUD. TAS has the potential to enhance treatment outcomes in clinical alcohol treatment. The accuracy, acceptability, and feasibility of TAS for people with AUD in clinical settings need to be investigated. This is the first study to use TAS in specialized alcohol services with diagnosed AUD individuals currently receiving treatment from a south London alcohol service. ISRCTN Registry ISRCTN46845361; https://www.isrctn.com/ISRCTN46845361. DERR1-10.2196/57653.

Random interval schedule of reinforcement influences punishment resistance for cocaine in rats

In an animal model of compulsive drug use, a subset of rats continues to self-administer cocaine despite footshock consequences and is considered punishment resistant. We recently found that punishment resistance is associated with habits that persist under conditions that typically encourage a transition to goal-directed control. Given that random ratio (RR) and random interval (RI) schedules of reinforcement influence whether responding is goal-directed or habitual, we investigated the influence of these schedules on punishment resistance for cocaine or food. Male and female Sprague Dawley rats were trained to self-administer either intravenous cocaine or food pellets on a seeking-taking chained schedule of reinforcement, with the seeking lever requiring completion of either an RR20 or RI60 schedule. Rats were then given four days of punishment testing with footshock administered at the completion of seeking on a random one-third of trials. For cocaine-trained rats, the RI60 schedule led to greater punishment resistance (i.e., more trials completed) than the RR20 schedule in males and females. For food-trained rats, the RI60 schedule led to greater punishment resistance (i.e., higher reward rates) than the RR20 schedule in female rats, although male rats showed punishment resistance on both RR20 and RI60 schedules. For both cocaine and food, we found that seeking responses were suppressed to a greater degree than reward rate with the RI60 schedule, whereas response rate and reward rate were equally suppressed with the RR20 schedule. This dissociation between punishment effects on reward rate and response rate with the RI60 schedule can be explained by the nonlinear relation between these variables on RI schedules, but it does not account for the enhanced resistance to punishment. Overall, the results show greater punishment resistance with the RI60 schedule as compared to the RR20 schedule, indicating that schedules of reinforcement are an influencing factor on resistance to negative consequences.

The positive reinforcing effects of cocaine and opposite-sex social contact: roles of biological sex and estrus.

Preclinical studies report that drug use and social contact mutually influence the reinforcing effects of one another. Most of these studies have used same-sex dyads exclusively, and the role of factors related to biological sex and hormonal fluctuations are not well understood. The purpose of this study was to examine the reinforcing effects of cocaine and social contact with an opposite-sex partner in male and female rats, and how these effects are modulated by ovarian hormones. Male and female rats were trained in a nonexclusive choice procedure in which cocaine and social contact with an opposite-sex partner were simultaneously available on concurrent progressive ratio schedules of reinforcement. To examine the effects of ovarian hormones related to estrous cycling, Experiment 1 used naturally cycling, gonadally intact females, whereas Experiment 2 used ovariectomized females, and estrus was artificially induced with exogenous hormones. In both experiments, cocaine and social contact functioned as robust reinforcers, and there were no significant effects of biological sex or estrus status of the females. The positive reinforcing effects of both cocaine and social contact increased as a function of cocaine dose, indicating that contingent cocaine administration increases the reinforcing effects of social contact. These data suggest that cocaine use among opposite-sex partners may enhance factors that contribute to social bonding.

The Effects of Reinforcing Task Alternations on Voluntary Task Selection

Everyday life often requires us to switch between states of cognitive flexibility and stability. However, little is known about what drives their regulation at the meta-control level. Based on current theories of cognitive control, we tested whether the regulation of cognitive flexibility and stability is guided by reinforcement learning. Using a task-switching paradigm with a double registration procedure – where people need to choose which task to perform before seeing the target, we systematically reinforced cued task switches or repetitions to test whether this led to more voluntary task switching (flexibility) or repeating (stability) on interspersed unrewarded free choice trials, respectively. While we did not find the hypothesized effect in an uninstructed version of the experiment (n=97), informing participants (n=58) on the reinforcement schedule on cued trials induced more switching or repeating on unrewarded free-choice trials. We speculate that people adapt their control strategies if they are aware of the respective benefits, even when they cease to be rewarded. Interestingly, choices were predominantly driven by task preference in both experiments and strongly correlated with task performance costs. This finding suggests that, when using an interspersed design with a double registration procedure, people mostly choose tasks based on task difficulty rather than switch avoidance. Together, these experiments help determine whether and when people can learn about the value of task selection strategies beyond the scope of a single task, and provide a self-regulating system to understand putative higher-order control processes.

Comparison of synchronous reinforcement and accumulated reinforcement for increasing on-task behavior in preschoolers.

In synchronous-reinforcement schedules, the duration of behavior directly controls the duration of reinforcement on a moment-to-moment basis. We replicated and extended Diaz de Villegas et al. (2020) by comparing the effects of synchronous reinforcement with two accumulated-reinforcement schedules for increasing on-task behavior for seven preschoolers. One accumulated schedule was the same as the one used in Diaz de Villegas et al. and did not include tokens, whereas the other accumulated schedule included the delivery of tokens within session. Furthermore, we assessed preference for the three reinforcement schedules. The results showed that synchronous reinforcement was effective for increasing on-task behavior for all seven participants. Furthermore, it was most effective for increasing on-task behavior for three out of seven participants and preferred by all participants. For some participants, other schedules were also similarly effective. The results are discussed with respect to implications for application.

A reinforcement schedule with two types of temporal information

Decades of research on reinforcement schedules have demonstrated that temporal information regarding the arrival or nonarrival of biologically significant events controls animal behavior. The fixed interval (FI) schedule, which is a time-based reinforcement schedule, suggests that responses are regulated by the time elapsed since the last reinforcement. This raises the question of how behavior is controlled when two distinct temporal cues regarding the availability of reinforcers are simultaneously presented. We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information: the time elapsed since the last reinforcement, and the inter-reinforcement interval (IRI). Through long-term operant conditioning in rats, we examined how temporal information controls responses. When two temporal cues were available, we found that behavior was influenced by each cue. Moreover, we discovered that the effects of these cues could be analytically separated within the dynamics of response rates. The findings revealed that in controlling behavior, living organisms can utilize two temporal cues rather than relying on a single, clear cue.•We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information.•When two temporal cues were available, we found that behavior was influenced by each cue.•Rats can utilize two temporal cues rather than relying on a single, clear cue.

Impact of Serine Racemase Deletion on Nicotine Discrimination.

The high comorbidity between schizophrenia and cigarette smoking points to a possible shared heritable factor predisposing individuals with schizophrenia to nicotine addiction. The N-methyl-D-aspartate (NMDA) receptor has been highly implicated in both schizophrenia and nicotine addiction. In the present study, we used mice with a null mutation on the serine racemase gene (srr), an established risk gene for schizophrenia, which encodes the enzyme to produce the NMDA receptor co-agonist D-serine, to model the pathology of schizophrenia and to determine whether NMDA receptor hypofunction reduced the ability of srr-/- mice to identify nicotine's subjective effects. Established nicotine discrimination procedures were used to train srr-/- and wild-type (WT) mice to discriminate 0.4mg/kg nicotine under a 10-response fixed-ratio (FR10) schedule of food reinforcement. Results show that WT mice reliably acquired 0.4mg/kg nicotine discrimination in about 54 training session, whereas srr-/- mice failed to acquire robust 0.4mg/kg nicotine discrimination even after extended (>70) training sessions. These results show that NDMA receptor hypofunction in srr-/- mice decreased sensitivity to the interoceptive effects of nicotine. Projected to humans, NMDA receptor hypofunction caused by mutations to the serine racemase gene in schizophrenia may reduce sensitivity to nicotine's subjective effects leading to increased nicotine consumption to produce the same effects as those unaffected by schizophrenia. There is high comorbidity between schizophrenia and nicotine dependence as well as possible shared genetic risk factors between the two. The serine racemase knockout mouse (srr-/-) with NMDA receptor hypofunction has been developed a model for schizophrenia. We found that srr-/- mice were unable to acquire 0.4mg/kg nicotine discrimination, whilst wild-type mice readily discriminated nicotine. These results show that decreased NMDA receptor function present in srr-/- mice and patients with schizophrenia may result in reduced sensitivity to nicotine's interoceptive effects, leading to increased nicotine consumption to produce the same subjective effects as those unaffected by schizophrenia.

Effects of psilocybin on uncertain punishment learning

Psilocybin may provide a useful treatment for mood disorders including anxiety and depression but its mechanisms of action for these effects are not well understood. While recent preclinical work has begun to assess psilocybin’s role in affective behaviors through innate anxiety or fear conditioning, there is scant evidence for its role in conflict between reward and punishment. The current study was designed to determine the impact of psilocybin on the learning of reward-punishment conflict associations, as well as its effects after learning, in male and female rats. We utilized a chained schedule of reinforcement that involved execution of safe and risky reward-guided actions under uncertain punishment. Different patterns of behavioral suppression by psilocybin emerged during learning versus after learning of risky action-reward associations. Psilocybin increased behavioral suppression in female rats as punishment associations were learned. After learning, psilocybin decreased behavioral suppression in both sexes. Thus, psilocybin produces divergent effects on action suppression during approach-avoidance conflict depending on when the conflict is experienced. This observation may have implications for its therapeutic mechanism of action.