Variable Ratio Schedule Of Reinforcement Research Articles

An optimized procedure for robust volitional cocaine intake in mice.

Substance use disorder (SUD) is a behavioral disorder characterized by volitional drug consumption. Mouse models of SUD allow for the use of molecular, genetic, and circuit-level tools, providing enormous potential for defining the underlying mechanisms of this disorder. However, the relevance of results depends on the validity of the mouse models used. Self-administration models have long been the preferred preclinical model for SUD as they allow for volitional drug consumption, thus providing strong face validity. While previous work has defined the parameters that influence intravenous cocaine self-administration in other species-such as rats and primates-many of these parameters have not been explicitly assessed in mice. In a series of experiments, we showed that commonly used mouse models of self-administration, where behavior is maintained on a fixed-ratio schedule of reinforcement, show similar levels of responding in the presence and absence of drug delivery-demonstrating that it is impossible to determine when drug consumption is and is not volitional. To address these issues, we have developed a novel mouse self-administration procedure where animals do not need to be pretrained on sucrose and behavior is maintained on a variable-ratio schedule of reinforcement. This procedure increases rates of reinforcement behavior, increases levels of drug intake, and results in clearer delineation between drug-reinforced and saline conditions. Together, these data highlight a major issue with fixed-ratio models in mice that complicates subsequent analysis and provide a simple approach to minimize these confounds with variable-ratio schedules of reinforcement. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Context affects resurgence of negatively reinforced human behavior

The effects of context on the resurgence of negatively reinforced (escape) responding was studied in an experiment with undergraduate students in which resurgence and renewal procedures were combined. Across conditions, in baseline (BL), key-pressing produced 3-s timeouts from pressing a force cell on a variable-ratio schedule of reinforcement; in the Alternative-Reinforcement and Test phases, a differential-reinforcement-of-other-behavior schedule and extinction were in effect, respectively. Conditions differed according to the context (the computer-screen color) in effect in each phase: ABA vs. ABB (with each letter representing a context; order of exposure to conditions was counterbalanced across participants). For each of six participants, independent of (a) order of exposure to conditions, (b) slight differences in BL reinforcement, and (c) differences in BL key-pressing rates, resurgence of greater magnitude occurred in the ABA than in the ABB condition. These results replicate and extend to contingencies of negative reinforcement previous findings with nonhumans and humans showing that context modulates the magnitude of resurgence.

Equidad en Intercambios de Esfuerzo y Ganancias: Efectos de la Información

Este trabajo aborda experimentalmente el estudio de los intercambios equitativos en una tarea en la que doce díadas de estudiantes de licenciatura en psicología asignaban a su compañero uno de tres posibles ―ujegos‖ asociados con requisitos diferenciales de un programa de reforzamiento de razón variable (RV: 5, 10 ó 20). En una segunda condición, los participantes asignaban también la cantidad de puntos que obtendría su pareja (1, 2 ó 4 puntos) con cada requisito de razón. Se programaron dos clases de sesiones: con información o sin información de la asignación de juego que hacían los compañeros en cada ensayo. Para evaluar las estrategias utilizadas se empleó el modelo Aristotélico de la Equidad (Adams, 1965; Anderson, 1976), mismo que describió adecuadamente la razón de esfuerzo y ganancias en los intercambios bajo la condición de información. Se discuten aquellas estrategias diádicas que resultaron en una distribución de mayor cantidad de ganancias con el menor esfuerzo posible, así como aquellas que produjeron distribuciones asimétricas de ganancias y esfuerzo, lo que desencadenó ganancias relativamente reducidas a lo largo de las sesiones. Especial énfasis recibe la discusión del mecanismo de reciprocidad en el intercambio social.

Uncertainty exposure causes behavioural sensitization and increases risky decision-making in male rats: toward modelling gambling disorder.

An animal model of gambling disorder, previously known as pathological gambling, could advance our understanding of the disorder and help with treatment development. We hypothesized that repeated exposure to uncertainty during gambling induces behavioural and dopamine (DA) sensitization - similar to chronic exposure to drugs of abuse. Uncertainty exposure (UE) may also increase risky decision-making in an animal model of gambling disorder. Male Sprague Dawley rats received 56 UE sessions, during which animals responded for saccharin according to an unpredictable, variable ratio schedule of reinforcement (VR group). Control animals responded on a predictable, fixed ratio schedule (FR group). Rats yoked to receive unpredictable reward were also included (Y group). Animals were then tested on the Rat Gambling Task (rGT), an analogue of the Iowa Gambling Task, to measure decision-making. Compared with the FR group, the VR and Y groups experienced a greater locomotor response following administration of amphetamine. On the rGT, the FR and Y groups preferred the advantageous options over the risky, disadvantageous options throughout testing (40 sessions). However, rats in the VR group did not have a significant preference for the advantageous options during sessions 20-40. Amphetamine had a small, but significant, effect on decision-making only in the VR group. After rGT testing, only the VR group showed greater hyperactivity following administration of amphetamine compared with the FR group. Reward uncertainty was the only gambling feature modelled. Actively responding for uncertain reward likely sensitized the DA system and impaired the ability to make optimal decisions, modelling some aspects of gambling disorder.

Selective lesions of the cholinergic neurons within the posterior pedunculopontine do not alter operant learning or nicotine sensitization.

Cholinergic neurons within the pedunculopontine tegmental nucleus have been implicated in a range of functions, including behavioral state control, attention, and modulation of midbrain and basal ganglia systems. Previous experiments with excitotoxic lesions have found persistent learning impairment and altered response to nicotine following lesion of the posterior component of the PPTg (pPPTg). These effects have been attributed to disrupted input to midbrain dopamine systems, particularly the ventral tegmental area. The pPPTg contains a dense collection of cholinergic neurons and also large numbers of glutamatergic and GABAergic neurons. Because these interdigitated populations of neurons are all susceptible to excitotoxins, the effects of such lesions cannot be attributed to one neuronal population. We wished to assess whether the learning impairments and altered responses to nicotine in excitotoxic PPTg-lesioned rats were due to loss of cholinergic neurons within the pPPTg. Selective depletion of cholinergic pPPTg neurons is achievable with the fusion toxin Dtx-UII, which targets UII receptors expressed only by cholinergic neurons in this region. Rats bearing bilateral lesions of cholinergic pPPTg neurons (>90% ChAT+ neuronal loss) displayed no deficits in the learning or performance of fixed and variable ratio schedules of reinforcement for pellet reward. Separate rats with the same lesions had a normal locomotor response to nicotine and furthermore sensitized to repeated administration of nicotine at the same rate as sham controls. Previously seen changes in these behaviors following excitotoxic pPPTg lesions cannot be attributed solely to loss of cholinergic neurons. These findings indicate that non-cholinergic neurons within the pPPTg are responsible for the learning deficits and altered responses to nicotine seen after excitotoxic lesions. The functions of cholinergic neurons may be related to behavioral state control and attention rather than learning.

Reinforcer devaluation by extinction depends on the food restriction protocol

A common feature of reinforcer devaluation studies is that new learning induces the devaluation. The present study used extinction to induce new learning about the conditioned reinforcer in a heterogeneous chain schedule. Rats pressed a lever in a heterogeneous chain schedule to produce a conditioned reinforcer (light) associated with the opportunity to obtain an unconditioned reinforcer (food) by pulling a chain. The density of food reinforcement correlated with the conditioned reinforcer was varied in a comparison of continuous and variable-ratio reinforcement schedules of chain pulling; this had no noticeable effect on conditioned reinforcer devaluation produced by extinction of chain pulling. In contrast, how rats were deprived appeared to matter very much. Restricting meal duration to 1h daily produced more lever pressing during baseline training and a greater reductive effect of devaluation on lever pressing than restricting body weight to 80% of a control rat's weight, which eliminated the devaluation effect. Further analysis suggested that meal-duration restriction may have produced devaluation effects because it was more effective than weight restriction in reducing rats' body weights. Our results exposed an important limitation on the devaluation of conditioned reinforcers: slight differences in food restriction, using two commonly employed food-restriction procedures, can produce completely different interpretations of reinforcer devaluation while leaving reinforcer-based learning intact.

Orexin Signaling Via the Orexin 1 Receptor Mediates Operant Responding for Food Reinforcement

Orexin (hypocretin) signaling is implicated in drug addiction and reward, but its role in feeding and food-motivated behavior remains unclear. We investigated orexin's contribution to food-reinforced instrumental responding using an orexin 1 receptor (Ox1r) antagonist, orexin -/- (OKO) and littermate wildtype (WT) mice, and RNAi-mediated knockdown of orexin. C57BL/6J (n = 76) and OKO (n = 39) mice were trained to nose poke for food under a variable ratio schedule of reinforcement. After responding stabilized, a progressive ratio schedule was initiated to evaluate motivation to obtain food reinforcement. Blockade of Ox1r in C57BL/6J mice impaired performance under both the variable ratio and progressive ratio schedules of reinforcement, indicating impaired motivational processes. In contrast, OKO mice initially demonstrated a delay in acquisition but eventually achieved levels of responding similar to those observed in WT animals. Moreover, OKO mice did not differ from WT mice under a progressive ratio schedule, indicating delayed learning processes but no motivational impairments. Considering the differences between pharmacologic blockade of Ox1r and the OKO mice, animals with RNAi mediated knockdown of orexin were then generated and analyzed to eliminate possible developmental effects of missing orexin. Orexin gene knockdown in the lateral hypothalamus in C57BL/6J mice resulted in blunted performance under both the variable ratio and progressive ratio schedules, resembling data obtained following Ox1r antagonism. The behavior seen in OKO mice likely reflects developmental compensation often seen in mutant animals. These data suggest that activation of the Ox1r is a necessary component of food-reinforced responding, motivation, or both in normal mice.

Bar pressing for food: differential consequences of lesions to the anterior versus posterior pedunculopontine

The pedunculopontine tegmental nucleus (PPTg) is in a key position to participate in operant reinforcement via its connections with the corticostriatal architecture and the medial reticular formation. Indeed, previous work has demonstrated that rats bearing lesions of the whole PPTg are impaired when learning to make two bar presses for amphetamine reinforcement. Anterior and posterior portions of the PPTg make different anatomical connections, including preferential projections by the anterior PPTg to substantia nigra pars compacta dopamine neurons and by the posterior PPTg to ventral tegmental area dopamine neurons. We wanted to assess the effects of anterior and posterior PPTg ibotenate lesions on rats learning simple and more complex schedules of natural reinforcement. We trained rats with lesions to the anterior PPTg (n = 11) and the posterior PPTg (n = 5) [and appropriate controls (n = 15)] to bar press for food on a variety of fixed-ratio and variable-ratio reinforcement schedules and then during extinction. We found that posterior PPTg-lesioned rats bar pressed at lower rates, were slower to learn to bar press, and often had deficits characteristic of impaired learning and/or motivation. In contrast, anterior PPTg-lesioned rats learned to bar press for reinforcement at normal rates. However, they made errors of perseveration and anticipation throughout many schedules, and pressed at a higher rate than controls during extinction, deficits best characterized as reflecting disorganized response control. Together, these data suggest that the anterior PPTg and posterior PPTg (and their related circuits) contribute differently to reinforcement learning, incentive motivation, and response control, processes that are considered to malfunction in drug addiction.

Response perseveration and adaptation in heavy marijuana-smoking adolescents

The present study examined two behavioral processes – response perseveration and response adaptation – in adolescents who were heavy marijuana smokers and control adolescents. Testing took place in a controlled laboratory setting, using customized software and either a computer keyboard or a custom built response panel for response input. Adolescents age 14–18 were recruited into a heavy smoking (near daily) group ( N = 22) or a control group ( N = 31) with < 15 lifetime uses of marijuana and no history of substance abuse or dependence. Marijuana use was verified by daily quantification of urinary cannabinoids and self-reports. Participants completed laboratory tasks designed to measure response perseveration (Wisconsin Card Sort Task, WCST) and response adaptation (concurrent variable-ratio reinforcement schedule with changing contingencies). Data were analyzed via ANOVA, controlling for multiple factors including: gender, age, nicotine use, presence of conduct disorder, and subscales of the Youth Self Report. After controlling for these compared to controls marijuana-using participants made significantly more perseverative and total errors on the WCST and showed significantly impaired (e.g., less adaptive) response allocation to the changing reinforcement contingencies on the concurrent-reinforcement task. Within the constraints of the study's limitations in controlling for alternative sources of between-subject variability, the data suggest that individuals who regularly smoke marijuana during adolescence show measurable perturbations in important basic behavioral processes. The data are also consistent with a previous laboratory study demonstrating reduced motivation in marijuana-smoking adolescents versus controls [Lane, S.D., Cherek, D.R., Pietras, C.J., and Steinberg, J.L. (2005). Performance of heavy marijuana-smoking adolescents on a laboratory measure of motivation. Addictive Behaviors, 30, 815–828].

FROM DISABLING TO ENABLING THE PUBLIC INTEREST: NATURAL TRANSITIONS FROM GAMBLING EXPOSURE TO ADAPTATION AND SELF‐REGULATION

FROM DISABLING TO ENABLING THE PUBLIC INTEREST: NATURAL TRANSITIONS FROM GAMBLING EXPOSURE TO ADAPTATION AND SELF‐REGULATION

Altered reward value of carbohydrate snacks for female smokers withdrawn from nicotine

Discontinuing nicotine intake usually results in weight gain partially due to heightened energy intake from between-meal snacks. This experiment tested the hypothesis that the reinforcing value of palatable carbohydrate-rich snacks increases for female smokers during nicotine deprivation. Eighteen smokers and 18 nonsmokers completed a concurrent-schedules operant computer task on two separate days. Smokers were bioverified abstinent at the second testing. The operant task allowed participants to earn points redeemable for either carbohydrate snacks or money on concurrent variable-ratio schedules of reinforcement. There were five different probabilities of earning points redeemable for snacks (8%, 16%, 25%, 50%, 75%), while the probability of earning points redeemable for money remained fixed at 25%. Reward value of snacks was measured by switch point: the reinforcement ratio at which the effort required to earn snacks exceeded their value to the respondent, as signified by a shift to working for money. Results showed that smokers undergoing nicotine deprivation persisted in working for snacks into leaner reinforcement schedules than nonsmokers ( P=.026). Furthermore, nicotine deprivation increased smokers' allocation of effort to earn snack foods relative to their own behavior when smoking ( P=.006). Variation in palatability or hunger did not explain these differences in snack reward value. Findings indicate that nicotine deprivation is associated with a heightened reward value of appealing snack foods for female smokers.

An apparatus for studying operant activity of captive coyotes

We describe a portable apparatus designed to examine the free-operant food preferences of captive coyotes in their home kennels. Because lever-pressing for food access was the dependent variable, we measured food preference independently of food ingestion. Using successive approximation, we trained 8 out of 19 coyotes (42%) to use the apparatus. This percentage is similar to training rates for dogs. We used fixed and variable ratio schedules of reinforcement to further test 4 of the trained coyotes. All 4 produced response curves similar to those of other species on similar schedules of reinforcement.

Potentiation of responding on a VR schedule by a stimulus correlated with reinforcement: Effects of diffuse and localized signals

In Experiment 1, rats were trained to leverpress on a variable ratio (VR) 30 schedule with a 500-msec delay between the reinforced response and food delivery. Subjects that experienced a signal during the delay responded faster than did control subjects that received the stimulus un-correlated with reinforcement. Higher response rates were obtained when the stimulus used to signal reinforcement was auditory rather than visual. Experiments 2 and 3 compared the effects of signaling reinforcement with either a localized or a diffuse light on responding maintained by VR schedules of reinforcement. Elevated response rates were observed with the diffuse stimulus, but the localized stimulus failed to produce such potentiation. Experiment 3 also examined the conditioned reinforcing power of localized and diffuse visual stimuli. These results are discussed with reference to (1) theories of selective association and sign tracking and (2) their implications for current theories of signaling reinforcement.

Discriminative stimulus properties of methylxanthines and their metabolites in rats

Rats were trained to discriminate methylxanthines from saline under a two-lever concurrent variable ratio schedule of reinforcement. One group was trained to discriminate between saline and 32 mg/kg caffeine. A second group was trained to discriminate between 56 mg/kg theophylline and saline. Rats reliably discriminated between saline and the training methylxanthine, displaying graded generalization curves across training-drug doses. Caffeine-trained rats demonstrated caffeine-appropriate responding when tested with theophylline, paraxanthine, and 3-methylxanthine. Theobromine failed to generalize to the caffeine cue at test doses up to 75 g/kg. In contrast to the caffeine group, rats trained to discriminate theophylline from saline were less sensitive (higher ED50) to the effects of caffeine and paraxanthine test doses. Only partial generalization to the theophylline cue occured at paraxanthine doses up to 100 mg/kg. Based upon these data, it is suggested that the underlying substrate(s) for the caffeine cue is in some respects different from the substrate(s) for the theophylline cue.

Interval and ratio reinforcement contingencies as determinants of methadone's effects

The effects of methadone hydrochloride on lever pressing rats maintained under multiple fixed-interval and fixed-ratio, or multiple variable-interval and variable-ratio reinforcement schedules equated for reinforcement density were examined. Under a multiple fixed-interval, fixed-ratio schedule overall response rate was decreased during both components but was most affected under the ratio schedule. Response rate decreases were due primarily to changes in running rate rather than pause time. Under a multiple variable-interval, variable-ratio schedule, overall response rate was also decreased by methadone, with the greatest decrease again occurring during the ratio schedule. These schedule-specific methadone effects are not due to differences in reinforcement frequency. Evidence for rate-dependency with methadone is not consistent across subjects.

Effects of relative proximity to reinforcement of a stimulus intruded on fixed- and variable-ratio schedules of reinforcement

Pigeons were trained to respond on either a fixed- or a variable-ratio schedule of reinforcement. After responding had stabilized, a visual stimulus change was presented after either 20% or 80% of the ratio had been completed. The duration of the stimulus was equal to the time it took the subjects to complete an additional 10% of the ratio. Presentation of the stimulus early in the ratio inhibited responding on the variable, but not on the fixed-ratio schedule. A stimulus change after 80% of the ratio had been completed, only produced response enhancement on the variable-ratio schedule. The effects of stimulus intrusion on ratio schedules were determined by a) the temporal proximity between stimulus intrusion and reinforcement presentation, and b) the contingencies of the operant baseline.

Undernutrition of weanling and adult rats: Effects on operant responding

In the rat, brain growth is most vulnerable to undernutrition during the suckling period. Undernutrition at that stage also produces lasting effects on behaviour and it is often assumed that these are due to disturbances of brain growth. The proposal that this may not necessarily be so was explored by testing the behaviour of rats which had been undernourished at later stages of life and which, therefore, would be expected to show little or no deficit in brain growth. Rats were undernourished either immediately after weaning (25–67 days) or in adulthood (80–134 days) and were tested 3–4 months later on variable interval and variable ratio schedules of reinforcement with food as the reward. Their behaviour on these schedules was similar to that of rats undernourished during the suckling period: both groups responded or tended to respond at a higher rate than controls. Hence, it is possible that undernutrition at any stage in life may make animals more responsive to food when deprived subsequently. A cognitive mechanism for this change in behaviour is suggested.

Learning deficits after lesions of dentate gyrus granule cells

The effect of destroying granule cells in the dentate gyrus on learning was examined with a behavioral testing protocol. These neurons were destroyed by microinjections of the selective neurotoxin colchicine in the hippocampal formation of rats. After a 30-day recovery period, the animals were trained in an operant chamber with an appetitive conditioning paradigm. The learning abilities of the animals with lesions were compared with two control groups—naive, unoperated rats and those with control injections of saline. The basic task required the animal to discriminate between two spatially separate visual stimuli which represented positive and negative cues. Testing and training was separated into four progressively more difficult phases with various stimulus schedules, contingencies of reinforcement, and stimulus positions. Colchicine-treated animals demonstrated significantly poorer performance than naive animals and those receiving saline control injections. None of the colchicine-treated animals achieved criterion performance in the stimulus position reversal paradigm, and half had difficulty with variable ratio schedules of reinforcement. Our experiments suggested that granule cells in the dentate gyrus played a pivotal role in certain learning tasks.

The “bathroom game”: A systematic program for the elimination of encopretic behavior

The present investigation utilized a unique, variable ratio schedule of reinforcement (the “bathroom game”) to treat a 10-year-old encopretic male. Dependent measures included confirmed incidents of (a) soiling and (b) appropriate bowel movements monitored across an ABAB design (Baseline 1, “Bathroom Game 1”, Baseline 2, “Bathroom Game 2”) with one-year follow-up. During “bathroom game” conditions, contingent monetary rewards were provided for non-instances of soiling and appropriate bowel movements. Such rewards were progressively and systematically leaned-out over the course of treatment on a pre-determined variable ratio schedule. Results indicated a clear demonstration of functional control and clinically significant treatment effects during both experimental periods. These findings are discussed with regard to the positive features of the “bathroom game” procedure and recommendations are made for future investigations in the area.

Employee reaction to continuous and variable ratio reinforcement schedules involving a monetary incentive.

Employee reaction to continuous and variable ratio reinforcement schedules involving a monetary incentive.