Reinforcement Of Low Rate Schedule Research Articles

Implementing a differential reinforcement of low rates schedule to alter vocal stereotypy and task engagement in two adolescents with autism spectrum disorder

AbstractVocal stereotypy (VS) is often observed in individuals with autism spectrum disorder (ASD) which at high rates can interfere with socialization or functioning in structured settings. There are multiple effective interventions available; yet, many procedures target the complete omission of the behavior or are only assessed at short intervals, making it unclear how they will generalize in applied settings. One intervention yet to be assessed as an individual intervention for automatically reinforced VS is differential reinforcement of low rates of behavior (DRL). In the present study, a functional analysis determined that the VS of two female adolescents with ASD was maintained by automatic reinforcement. A DRL procedure was implemented which incorporated: (a) a specified interval for reinforcement; (b) the behavioral expectations; (c) the permissible instances of VS within the interval; (d) learner feedback; and (e) the reset/non‐reset aspect of the schedule. As the targeted behavior decreased across sessions, the DRL interval was systematically increased in order to thin out the schedule of reinforcement. The intervention reduced VS and increased untargeted task engagement for both participants. Applied and theoretical implications of the study as well as social validity, limitations, and future research are discussed.

Modeling VI and VDRL feedback functions: Searching normative rules through computational simulation.

We present the mathematical description of feedback functions of variable interval and variable differential reinforcement of low rates as functions of schedule size only. These results were obtained using an R script named Beak, which was built to simulate rates of behavior interacting with simple schedules of reinforcement. Using Beak, we have simulated data that allow an assessment of different reinforcement feedback functions. This was made with unparalleled precision, as simulations provide huge samples of data and, more importantly, simulated behavior is not changed by the reinforcement it produces. Therefore, we can vary response rates systematically. We've compared different reinforcement feedback functions for random interval schedules, using the following criteria: meaning, precision, parsimony, and generality. Our results indicate that the best feedback function for the random interval schedule was published by Baum (1981). We also propose that the model used by Killeen (1975) is a viable feedback function for the random differential reinforcement of low rates schedule. We argue that Beak paves the way for greater understanding of schedules of reinforcement, addressing still open questions about quantitative features of simple schedules. Also, Beak could guide future experiments that use schedules as theoretical and methodological tools.

Sugar consumption and behavioural inhibition in the rat

The metabolic effects of sugary drinks have been extensively studied, whereas the effects on psychological processes have received relatively limited attention. Several studies have found that high-sugar diets can produce impaired performance by rats on tests assessing spatial learning and memory. In contrast, despite claims that weakened inhibitory control underlies many sugar-induced deficits, evidence supporting this proposal has been limited. The aim of the present study was to assess the impact of high-sugar diets on response inhibition, as measured by rats' performance on a differential reinforcement of low rates schedule (DRL) in Experiments 1 and 2 and on a Pavlovian discrimination reversal task in Experiment 3. In all three experiments a 30-day diet stage, in which Sugar groups were given unrestricted access to 10% sucrose solution and Control groups had access to water only, was followed by behavioural tests. In Experiment 1 the Sugar group performed poorly on a spatial memory task, but no difference was detected between the performances of the two groups in the DRL test. In Experiment 2 longer DRL training was given and post-diet performance was assessed both before and after access to sugar was withdrawn. Null results were obtained under both conditions. In Experiment 3 rats’ performance on a discrimination learned prior to the diet intervention was not affected by the high-sugar diet, but neither was performance once the discrimination was reversed. The implications of these results for understanding of sugar-induced psychological deficits are discussed.

Response duration is sensitive to both immediate and delayed reinforcement.

We investigated the duration of lever pressing by rats when the delivery of appetitive reinforcers was contingent upon response duration. In the first experiment, response durations increased when duration requirements were imposed, and they decreased when duration requirements were removed. This effect occurred whether reinforcers were immediate or delayed by 8 s. In order to maintain the integrity of the delay intervals, reinforcer delivery was dependent upon both lever depression and release. In a second experiment, lever depression only and a response duration of at least 4 s were required for reinforcer delivery. Compared to immediate reinforcement conditions, delayed reinforcers increased both variability and the length of the maximum response durations. In a third experiment, immediate reinforcers were delivered contingent upon lever depression and release under a variety of duration requirements. Median lever-press durations tracked the contingencies rapidly. Across all three experiments, rats emitted numerous response durations that were too short to satisfy the reinforcer requirements, and bimodal distributions similar to those produced by differential reinforcement of low rate schedules were evident for most rats. In many aspects, response duration responds to reinforcement parameters in a fashion similar to rate of discrete responding, but an examination of this continuous dimension of behavior may provide additional information about environment-behavior relationships.

Ethanol reinforcement elicits novel response inhibition behavior in a model of ethanol dependence

Reduced impulse control is a known risk factor for drug abuse vulnerability. Chronic experience with drugs of abuse is suggested to enhance impulsivity and thereby perpetuate addiction. However, the nature of this relationship (directionality, causality) with respect to alcohol use disorder is unclear. The current study aims to evaluate changes in impulsivity during the development of ethanol dependence and subsequent protracted abstinence from excessive ethanol. A widely validated chronic intermittent ethanol-vapor exposure (CIE; 14 h vapor-ON/10 h vapor-OFF) was used to generate ethanol dependence and impulsivity was tested using a differential reinforcement of low rates 15 second (DRL15) schedule using either SUCROSE (palatable modified sucrose pellets) or ETHANOL (10%v/v with 0.125% w/v saccharine sodium, 1.5% w/v sucrose). Reduced efficiency of earning reinforcers (expressed as % reinforcers/responses) is indicative of reduced response inhibition or increased impulsivity. Neither CIE, nor forced abstinence altered efficiency in SUCROSE-CIE rats compared to matched SUCROSE-NonCIE. In contrast, efficiency of reinforcement increased in ETHANOL-CIE compared to ETHANOL-NonCIE during during protracted abstinence. Interestingly, ETHANOL-CIE rats escalated the ETHANOL (not SUCROSE) intake during the weeks of CIE exposure; this effect attenuated over the weeks of protracted abstinence. Taken together, the reinforcer-type is key to identifying changes due to ethanol dependence. In contrast to other drugs of abuse, evidence for reduced impulsivity was obtained in the ethanol dependence model. Finally, the behavioral adaptations under DRL15 appear to be targeted towards maximization of ethanol intake and thus, may overlap with neurological underpinnings of escalation and relapse. Support or Funding Information AA020098, AA06420 and DA034140 Ethanol dependent rodents exhibit escalation of ethanol intake under a differential reinforcement of low rates schedule. This abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

Differential Reinforcement of Low Rate Schedules Reduce Severe Problem Behavior.

Differential-reinforcement-of-low-rate (DRL) schedules are reinforcement contingencies designed to reduce response rates. A common variation of the DRL arrangement is known as full-session DRL ( f-DRL), in which a reinforcer is presented at the end of an interval if the response rate during that interval is below a predetermined criterion. Prior human operant research involving arbitrary mouse clicks has shown that the f-DRL is likely to reduce target responding to near zero rates. Similarly, applied research has shown that the f-DRL is likely to reduce minimally disruptive classroom behavior. There are, however, relatively few successful applications of the f-DRL to severe forms of problem behavior (e.g., self-injurious behavior). Thus, the purpose of this study was to examine the effects of f-DRL on the severe problem behavior of individuals with intellectual and developmental disabilities. For four participants, the f-DRL reduced severe problem behavior by clinically significant levels. Furthermore, results of a contingency strength analysis showed a strong negative contingency strength between target responding and reinforcer delivery for all participants.

Temporal control of behaviour in children with differential reinforcement of low rates schedule: the role of age, language and cognitive functioning on temporal regulation

Research on temporal regulation in children has been prolific until early 1990s and has received a very limited attention since then. However, the studies focussed mainly on very short durations, and many questions raised at that time remain unanswered (Clément et al., 2007). The scope of this study was to evaluate temporal control in children with differential reinforcement of low-rates (DRL) schedule. Objectives were (a) to evaluate the performance in DRL with two distinct durations; (b) to evaluate the relationship between performance, IQ and language; and (c) to observe children’s response patterns across the sessions. Eleven children aged from 2.6 to 7 years old were exposed to a DRL 5s and a DRL 20s schedule. No significant correlation was observed between language, IQ and the performance in DRL. In DRL 5s, seven children adjusted their responses and six in DRL 20s. Age was positively correlated to performance in DRL 5s, while the response patterns in DRL 20s were hardly predictable. In both conditions, children aged from 4.6 years old showed a lower proportion of bursting responses, a lower rate of response, a larger proportion of reinforced responses and a higher optimisation coefficient.

Effects of the δ opioid agonist AZD2327 upon operant behaviors and assessment of its potential for abuse

AZD2327 is a brain-penetrant agonist at δ opioid receptors which has antidepressant and anxiolytic properties in a wide array of animal models. As part of the preclinical safety pharmacology assessment, a number of studies were conducted in order to characterize its behavioral effects and its potential for abuse, in order to enable testing in humans. AZD2327 produced only modest effects when tested in a multiple fixed-ratio differential reinforcement of low rate schedule in rats, and did not enhance the rate-suppressing effects of ethanol in the procedure. In a suppressed responding test, AZD2327 only reduced rates of unpunished responding. In drug discrimination studies, AZD2327 produced partial or no generalization from known drugs of abuse. In primates trained to self-administer cocaine, substitution with AZD2327 did not result in appreciable self-administration of AZD2327, indicating that it does not behave as a positive reinforcer under the present conditions. Following termination of repeated administration of AZD2327, no signs of physical dependence (withdrawal) were noted. Overall, the data suggest that AZD2327 does not possess a high potential for abuse, and appears to have only subtle behavioral effects as measured by operant behaviors.

AQUISIÇÃO DE REPERTÓRIO DISCRIMINATIVO AUDITIVO EM ESQUEMA MÚLTIPLO: EXPLORANDO PROCEDIMENTO PARA ESTUDO DA FORMAÇÃO DE CLASSES FUNCIONAIS EM RATOS

Estudos sobre a formação de classes de estímulos usualmente requerem o treino de discriminações de linha de base que incluem múltiplas discriminações simples e/ou condicionais inter-relacionadas. O objetivo do presente estudo foi replicar a fase de aquisição de discriminações de linha de base de um estudo anterior com ratos como sujeitos, com mudanças de procedimento objetivando tornar mais eficiente o treino de linha de base. Seis ratos albinos (Rattus norvegicus) foram submetidos a treino de discriminações simples sucessivas, com estímulos auditivos, em sessões de longa duração. Dois estímulos auditivos (A1 e B1) foram correlacionados a esquema de razão variável (VR) e dois outros estímulos (A2 e B2) foram correlacionados a esquema de reforço diferencial de taxas baixas (DRL). Cinco dos seis ratos demonstraram aquisição das discriminações, desenvolvendo dois padrões distintos de responder (taxas altas no esquema VR e taxas baixas no esquema DRL) sob controle dos estímulos auditivos. Este trabalho sugere que o emprego de esquemas múltiplos com estímulos auditivos em sessões de longa duração pode representar uma alternativa rápida e eficaz na formação de uma linha de base necessária aos estudos de formação de classes funcionais com estímulos auditivos e com ratos como sujeitos. Palavras-chave: razão variável, reforço diferencial de taxas baixas, estímulos auditivos, Rattus novergicus

A comparison of adult and adolescent rat behavior in operant learning, extinction, and behavioral inhibition paradigms.

Poor self-control, lack of inhibition, and impulsivity contribute to the propensity of adolescents to engage in risky or dangerous behaviors. Brain regions (e.g., prefrontal cortex) involved in impulse-control, reward-processing, and decision-making continue to develop during adolescence, raising the possibility that an immature brain contributes to dangerous behavior during adolescence. However, very few validated animal behavioral models are available for behavioral neuroscientists to explore the relationship between brain development and behavior. To that end, a valid model must be conducted in the relatively brief window of adolescence and not use manipulations that potentially compromise development. The present experiments used three operant arrangements to assess whether adolescent rats differ from adults in measures of learning, behavioral inhibition, and impulsivity, within the aforementioned time frame without substantial food restriction. In Experiment 1, separate squads of rats were trained to lever-press and then transitioned to two types of extinction. Relative to their baselines, adolescent rats responded more during extinction than adults, suggesting that they were less sensitive to the abolishment of the reinforcement contingency. Experiment 2 demonstrated similar age-related differences during exposure to a differential reinforcement of low rates schedule, a test of behavioral inhibition. Lastly, in Experiment 3, adolescent's responding decreased more slowly than adults during exposure to a resetting delay of reinforcement schedule, suggesting impaired self-control. Results from these experiments suggest that adolescents exhibit impaired learning, behavioral inhibition and self-control, and in concert with recent reports, provide researchers with three behavioral models to more fully explore neurobiology of risk-taking behavior in adolescence.

Early temporal short-term memory deficits in double transgenic APP/PS1 mice

We tested single APP (Tg2576) transgenic, PS1 (PS1dE9) transgenic, and double APP/PS1 transgenic mice at 3 and 6 months of age on the acquisition of a hippocampal-dependent operant "differential reinforcement of low rate schedule" (DRL) paradigm. In this task mice are required to wait for at least 10 seconds (DRL-10s) between 2 consecutive nose poke responses. Our data showed that while single APP and PS1 transgene expression did not affect DRL learning and performance, mice expressing double APP/PS1 transgenes were impaired in the acquisition of DRL-10s at 6 months, but not at 3 months of age. The same impaired double transgenic mice, however, were perfectly capable of normal acquisition of signaled DRL-10s (SDRL-10s) task, a hippocampal-independent task, wherein mice were required to emit responses when the end of the 10-second delay was signaled by a lighting of the chamber. The age-dependent and early deficits of APP/PS1 mice suggest that the appetitive DRL paradigm is sensitive to the amyloid pathology present in double APP/PS1 mice, and that this mouse line represents a good model with which to study the efficacy of therapeutic strategies against Alzheimer's disease.

Generalization of causal efficacy judgments after evaluative learning

In three experiments, we examined the effect of response-outcome relations on human ratings of causal efficacy and demonstrated that such efficacy ratings transfer to novel situations through derived stimulus relations. Causal efficacy ratings were higher, and probability of an outcome given a response was lower, for a differential reinforcement of high rate schedule than for either a differential reinforcement of low rate schedule (Experiment 1) or a variable interval schedule (Experiment 2). In Experiment 3, we employed schedules that were equated for outcome probability and noted that ratings of causal efficacy and the rate of response were higher on a variable ratio than on a variable interval schedule. For participants in all three experiments, causal efficacy ratings transferred to the stimulus present during each schedule and generalized to novel stimuli through derived relations. The results corroborate the view that schedules are a determinant of both response rates and causal efficacy ratings. In addition, the novel demonstration of a mechanism of generalization of these ratings via derived relations has clinical implications.

Evidence for impulsivity in the Spontaneously Hypertensive Rat drawn from complementary response-withholding tasks

BackgroundThe inability to inhibit reinforced responses is a defining feature of ADHD associated with impulsivity. The Spontaneously Hypertensive Rat (SHR) has been extolled as an animal model of ADHD, but there is no clear experimental evidence of inhibition deficits in SHR. Attempts to demonstrate these deficits may have suffered from methodological and analytical limitations.MethodsWe provide a rationale for using two complementary response-withholding tasks to doubly dissociate impulsivity from motivational and motor processes. In the lever-holding task (LHT), continual lever depression was required for a minimum interval. Under a differential reinforcement of low rates schedule (DRL), a minimum interval was required between lever presses. Both tasks were studied using SHR and two normotensive control strains, Wistar-Kyoto (WKY) and Long Evans (LE), over an overlapping range of intervals (1 – 5 s for LHT and 5 – 60 s for DRL). Lever-holding and DRL performance was characterized as the output of a mixture of two processes, timing and iterative random responding; we call this account of response inhibition the Temporal Regulation (TR) model. In the context of TR, impulsivity was defined as a bias toward premature termination of the timed intervals.ResultsThe TR model provided an accurate description of LHT and DRL performance. On the basis of TR parameter estimates, SHRs were more impulsive than LE rats across tasks and target times. WKY rats produced substantially shorter timed responses in the lever-holding task than in DRL, suggesting a motivational or motor deficit. The precision of timing by SHR, as measured by the variance of their timed intervals, was excellent, flouting expectations from ADHD research.ConclusionThis research validates the TR model of response inhibition and supports SHR as an animal model of ADHD-related impulsivity. It indicates, however, that SHR's impulse-control deficit is not caused by imprecise timing. The use of ad hoc impulsivity metrics and of WKY as control strain for SHR impulsivity are called into question.

Response rate is not an effective mediator of learned stimulus equivalence in pigeons

We explored response rate as a possible mediator of learned stimulus equivalence. Five pigeons were trained to discriminate four clip art pictures presented during a 10-sec discrete-trial fixed interval (FI) schedule: two paired with a one-pellet reinforcer, which supported a low rate of responding, and two paired with a nine-pellet reinforcer, which supported a high rate of responding. After subjects associated one stimulus from each of these pairs with a discriminative choice response, researchers presented two new clip art stimuli during a 10-sec FI: one trained with a differential reinforcement of low rate schedule (DRL) after the FI and the other trained with a differential reinforcement of high rate schedule (DRH) after the FI. Each of the stimuli that were withheld during choice training was later shown to see if the choice responses would transfer to these stimuli. The results suggest that response rate alone does not mediate learned stimulus equivalence.

Behavioral pharmacology of AR-A000002, a novel, selective 5-hydroxytryptamine(1B) antagonist.

The present review summarizes the behavioral pharmacology conducted to profile the anxiolytic and antidepressant potential of the selective 5-hydroxytryptamine (HT)(1B) antagonist (R)-N-[5-methyl-8-(4-methylpiperazin-1-yl)-1,2,3,4-tetrahydro-2-naphthyl]-4-morpholinobenzamide (AR-A000002). AR-A000002 functions as a 5-HT(1B) antagonist in vivo, which was shown by the antagonism of the discriminative stimulus effects in the guinea pig of the 5-HT(1B) agonist 3-(N-methylpyrrolidin-2R-ylmethyl)-5-(3-nitropyrid-2-ylamino)-lH-indole (CP135,807). Anxiolytic activity of AR-A000002 was demonstrated in the separation-induced vocalization paradigm in guinea pig pups, and in a suppressed responding procedure in pigeons and guinea pigs, but only a weak trend was noted in a suppressed responding procedure in squirrel monkeys. Antidepressant efficacy was shown in a number of paradigms. In pigeons and guinea pigs responding under a differential reinforcement of low rates schedule of reinforcement (DRL), AR-A000002 increased the number of reinforcers earned without altering the number of responses made. In guinea pigs trained under a response duration differentiation paradigm, AR-A000002 increased mean lever-press duration. Finally, AR-A000002 was shown to block escape failures in guinea pigs submitted to a learned helplessness paradigm. Taken together, these data suggest utility for 5-HT(1B) antagonists in the treatment of both anxiety and affective disorders.

Post-injection delays in experimental chambers, but not in home cages, produce both sensitization and tolerance of operant behaviour to midazolam: relation to pharmacokinetics.

The effects of post-injection delay time and environmental context on behaviour after subcutaneous administration of 3 mg/kg midazolam were investigated under a differential reinforcement of low rate schedule (i.e. DRL 45 s) in 3 h sessions. Post-injection delays were varied (0-120 min) for two groups of rats placed in either the experimental chamber (group 1) or home cage (group 2) during the pre-session delay times. Midazolam increased shorter-response (inter-response times < 45 s) rates and decreased reinforcement rates in a time-related manner. Reinforcement rate-time profiles were also integrated with parallel pharmacokinetics. Post-injection delays in either environment yielded performances that mirrored the pharmacokinetic profile operative at the corresponding time-delay points. At higher concentrations (> 0.12 microg/ml) the pharmacokinetics of midazolam largely determined the behavioural effects in both groups, regardless of post-injection delays. However, at lower drug concentrations, longer post-injection delays (> 60 min) in the experimental chamber produced both sensitization and tolerance, as measured by greater increases in shorter-response rates and a more rapid return of the reinforcement rate, respectively. Interaction of the discriminative stimulus effects of midazolam with the context probably alters the magnitude of behavioural effects when the delay occurs in the experimental chambers, whereas no such interaction is present in group 2. The DRL schedule with post-injection delays in experimental chambers provides a useful behavioural paradigm for studying both sensitization and tolerance.

Cocaine pharmacodynamics after intravenous and oral administration in rats: relation to pharmacokinetics.

To design optimal dose regimes for oral cocaine, it is essential to characterize pharmacokinetics (PK) of cocaine after IV and PO administration. To investigate the absolute bioavailability of oral cocaine, its effectiveness and the relation between PK and PD in a within-subject design. We used the effects of IV and PO cocaine on a contingency-controlled timing behavior, the differential reinforcement of low rate schedule (DRL 45-s) in 3-h sessions, as the PD measures [i.e., the shorter-response rate (srr) and the reinforcement rate (rr)]. Cocaine PK parameters were determined by simultaneous modeling of the concentration-time profiles (CTPs) after IV 2 mg/kg and PO 20 mg/kg cocaine administration. The absolute oral cocaine bioavailability was determined pharmacokinetically (F) and pharmacodynamically (Fsrr and Frr). IV and PO cocaine increased the shorter response rate and decreased the reinforcement rate in a dose- and time-related fashion, which mirrored the respective prototypical serum cocaine CTPs. After the absorption phase, the serum cocaine CTP of PO cocaine paralleled that of IV cocaine. The duration of action for PO cocaine was longer than that for IV cocaine owing to its larger mean residence time. The active metabolite, norcocaine, was not detected after IV but after PO cocaine administration. The value of F was 4.66% which was significantly lower than the values of Fsrr (13.67%) and Frr (32.63%). Furthermore, the concentration-effect relations for the reinforcement rate revealed that PO cocaine was more potent than IV cocaine. Oral cocaine is more effective behaviorally than from predictions made in terms of its PK. The differences in active metabolite profiles as well as the rate and extent of acute tolerance for IV versus PO cocaine may account for the greater potency observed for oral cocaine.

Caffeine Has Similar Pharmacokinetics and Behavioral Effects via the IP and PO Routes of Administration

Caffeine administered intraperitoneally (IP) or orally (PO) decreased the reinforcement rate and increased the nonreinforced response rate in a dose-related fashion under a differential reinforcement of low rate schedule (DRL 45-s) in 3-h sessions. These effects were similar following both routes of caffeine administration. The parallel pharmacokinetics for IP and PO caffeine were each determined and related to the respective effects of caffeine on reinforcement rate. Serum caffeine concentrations were similar across the session after the absorption phase for a given dose. Consequently, the effect remained in approximately the same range within a dose, and no single dose possessed a full concentration–effect relation for the two routes. The effects of IP and PO caffeine on reinforcement rate plateaued at doses higher than 40 mg/kg, which produced a serum caffeine concentration of approximately 25 μg/ml regardless of the route of administration. The EC 50 values were 7.34 and 9.93 μg/ml for IP and PO caffeine, respectively. This study as well as our previous studies demonstrated that the IP route is dependable for studying caffeine dose–response relations but not for studying other drugs (e.g., midazolam). The possible mechanism accounting for this difference is discussed.

Lack of consistent individual differences in rats on tasks that require response inhibition

Hungry rats performed four different tasks which all required responding to be withheld for food reinforcement to be obtained. The tasks were the following: an omission schedule, a go/no-go procedure, a differential reinforcement of low rate schedule, and a choice task in which the subject could respond for a small immediate reward or a large delayed reward. Each rat experienced each task twice. For each task, the rank ordering of the subjects’ performance on the test and retest was highly consistent, indicating the reliability of the task for measuring a particular “ability.” However, there were no consistencies in the performance of individual rats between tasks. This suggests that claims regarding consistent individual differences in brain-lesioned nonhumans across a range of such tasks are not supported in intact subjects.

Effects of imipramine and mirtazapine on operant performance in rats

AbstractThe effects of the tricyclic antidepressant imipramine and the atypical antidepressant mirtazapine were compared on the performance of rats in three operant procedures: a differential reinforcement of low rates schedule (DRL), a delayed matching to position (DMTP), and simultaneous visual discrimination tasks. Both compounds improved performance in the DRL task in a similar dose‐related manner. Imipramine, but not mirtazapine, disrupted performance in the visual discrimination task. Imipramine reduced accuracy and increased response latencies and missed trials. Neither compound effected accuracy in the DMTP; both compounds caused some slowness of responding and imipramine caused several animals to fail to respond in a manner similar to that observed in the visual discrimination task. These data suggest that although imipramine and mirtazapine are similarly effective in putative tests of antidepressant activity, imipramine has a greater tendency to disrupt other aspects of cognitive performance, as well as exert generally depressive effects on operant responding.