Intermittent Reinforcement Schedules Research Articles

Conscious expectancy rather than associative strength elicits brain activity during single-cue fear conditioning.

The neurocognitive processes underlying Pavlovian conditioning in humans are still largely debated. The conventional view is that conditioned responses (CRs) emerge automatically as a function of the contingencies between a conditioned stimulus (CS) and an unconditioned stimulus (US). As such, the associative strength model asserts that the frequency or amplitude of CRs reflects the strength of the CS-US associations. Alternatively, the expectation model asserts that the presentation of the CS triggers conscious expectancy of the US, which is responsible for the production of CRs. The present study tested the hypothesis that there are dissociable brain networks related to the expectancy and associative strength theories using a single-cue fear conditioning paradigm with a pseudo-random intermittent reinforcement schedule during functional magnetic resonance imaging. Participants' (n = 21) trial-by-trial expectations of receiving shock displayed a significant linear effect consistent with the expectation model. We also found a positive linear relationship between the expectancy model and activity in frontoparietal brain areas including the dorsolateral prefrontal cortex (PFC) and dorsomedial PFC. While an exploratory analysis found a linear relationship consistent with the associated strength model in the insula and early visual cortex, our primary results are consistent with the view that conscious expectancy contributes to CRs.

Effects of contingent and noncontingent reinforcement on the emotional behavior of domestic dogs (Canis familiaris)

Intermittent schedules of reinforcement are frequently used in dog training. During fixed-time (FT) schedules, reinforcement delivery is time-based and response-independent. In contrast, during fixed-interval (FI) schedules, a response produces a reinforcer if a certain amount of time has elapsed since the last reinforcer was delivered (i.e., FI schedules are time-based and response-dependent). FT schedules are typically used in noncontingent-reinforcement interventions with the aim of decreasing an undesired behavior. Although the effectiveness of these interventions has been reliably shown with human and nonhuman animal learners, little is known about the emotional behavior that is related to these contingencies. In the present study, the target behavior (stepping on a mat) of six dogs was initially trained (shaped) and maintained on a FI 15s schedule. In addition, dogs were exposed to a FT 15s schedule, a combination of FT 15s and FI 15s schedules, and an extinction-only condition (no reinforcement was delivered for the target response). We found that rates of emotional responses (e.g., lip/nose licking, vocalizing) varied in prevalence across dogs and phases. Although lip/nose licking was displayed in high rates across all dogs and phases, it was generally lower in baseline and extinction phases. This suggests that these phases produced less frustration-related behavior. The effect-size calculations for dogs engaging in pacing, sneezing, and vocalizing underscored the varied prevalence of the behaviors across phases and dogs and yielded mostly small- to medium-size effects. These results are especially relevant for the training of companion and working dogs and their well-being during training sessions. Future research should continue investigating emotional behavior in dogs across various schedules of reinforcement (e.g., variable schedules).

Coordinated and individual responding in pairs of rats under variable interval schedules

The coordination of responses in pairs of rats was investigated by parametrically manipulating the duration of variable-interval (VI) schedule mean inter-reinforcer intervals (IRIs). Two conditions required coordination for a group of three pairs of rats (VI 5 s, 10 s, 20 s, 40 s, 80 s, 120 s, 160 s, 60 s, and 243 s) and individual responding for another group of three pairs of rats (VI 5, 10 s, 20 s, 40 s and 80 s). An inverted U-shaped function related responding measures and reinforcement rates: Overall response rates, coordinated episode rates and percentage of coordination increased with increasing reinforcement rates, and decreased at the highest rates of reinforcement in both groups. Rats exposed to VI schedules that required coordination generally showed higher rates and higher percentages of coordinated responses when compared to rats that responded under the individual schedules. Local rates of coordinated episodes showed that coordination occurred at a steady rate across the VI IRIs for the Coordination Group. The results extend previous research on the patterns of coordinated responses maintained by intermittent schedules of reinforcement.

Efectos de variar el nivel de privación de comida sobre el valor reforzante del agua en el beber inducido por el programa

Scheduled-induced drinking (SID) occurs when food-deprived rats are exposed to intermittent schedules of food reinforcement. In recent years, SID has been considered as operant behavior reinforced directly by water and it has been suggested that food deprivation may function as a motivating operation that increases the reinforcing value of water. The purpose of the study was to determine the effects of varying food-deprivation levels on water reinforcing efficacy during SID sessions. In successive phases, three rats were maintained at their free-feeding weights, or were food deprived at 90 or 80% of their free-feeding weights. Rats were exposed to SID sessions in which food was delivered according to a tandem fixed- time 176- s, differential-reinforcement-of-other-behavior 4-s. Lever presses were reinforced with water according to a progressive-ratio 5 schedule. In general, response rates for water and break points increased with increases in food-deprivation level. The results complement earlier findings showing that SID is operant behavior reinforced directly by water.

Intermittent access to oxycodone decreases dopamine uptake in the nucleus accumbens core during abstinence.

A major obstacle in treating opioid use disorder is the persistence of drug seeking or craving during periods of abstinence, which is believed to contribute to relapse. Dopamine transmission in the mesolimbic pathway is posited to contribute to opioid reinforcement, but the processes by which dopamine influences drug seeking have not been completely elucidated. To examine whether opioid seeking during abstinence is associated with alterations in dopamine transmission, female and male rats self-administered oxycodone under an intermittent access schedule of reinforcement. Following self-administration, rats underwent a forced abstinence period, and cue-induced seeking tests were conducted to assess oxycodone seeking. One day following the final seeking test, rats were sacrificed to perform ex vivo fast scan cyclic voltammetry and western blotting in the nucleus accumbens. Rats displayed reduced dopamine uptake rate on abstinence day 2 and abstinence day 15, compared to oxycodone-naïve rats. Further, on abstinence day 15, rats had reduced phosphorylation of the dopamine transporter. Additionally, local application of oxycodone to the nucleus accumbens reduced dopamine uptake in oxycodone-naïve rats and in rats during oxycodone abstinence, on abstinence day 2 and abstinence day 15. These observations suggest that abstinence from oxycodone results in dysfunctional dopamine transmission, which may contribute to sustained oxycodone seeking during abstinence.

Nominally acceptable integrity failures negatively affect interventions involving intermittent reinforcement.

The finding that differential reinforcement of alternative behavior (DRA) is efficacious at 80% integrity when continuous reinforcement is programmed for alternative responding may have contributed to a perception that integrity at 80% or above is acceptable. However, research also suggests that other interventions (e.g., noncontingent reinforcement) may not remain effective at 80% integrity. The conditions under which 80% integrity is acceptable for common behavioral interventions remains unclear. Therefore, we conducted two human-operant studies to evaluate effects of 80% integrity for interventions with contingent or noncontingent intermittent reinforcement schedules. During Experiment 1, we compared noncontingent reinforcement (NCR) and DRA when implemented with 80% integrity. During Experiment 2, we compared 2 variations of DRA, which included either a ratio or interval schedule to reinforce alternative behavior. Results replicated previous research showing that DRA with a FR-1 schedule programmed for alternative responding resulted in consistent target response suppression, even when integrity was reduced to 80%. In contrast, neither NCR nor interval-based DRA were consistently effective when implemented at 80% integrity. These results demonstrate that 80% integrity is not a uniformly acceptable minimal level of integrity.

The use of an intermittent schedule of reinforcement to evaluate detection dogs' generalization from smokeless-powder.

Odor generalization is essential for detection dogs. Regardless of its importance, limited research is available on detection dog odor generalization. The objectives of this study were (1) evaluate the use of an intermittent schedule of reinforcement to assess generalization in dogs and (2) evaluate olfactory generalization from a single exemplar of smokeless powder (SP). Dogs (N = 5) were trained to detect SP in an automated olfactometer under an intermittent schedule of reinforcement where only 60% of correct responses were reinforced. After training, eight non-reinforced probe trials were inserted within a session. A total of 15 testing odors were evaluated across 15 consecutive sessions (one odor/session). Six of the testing odors were control and the remaining testing odors were objects indirectly exposed to SP, objects that contained or were directly exposed to SP, single-base SP and diphenylamine (the main volatile present in the headspace of SP). Dogs' response rate to all testing odors except for the cotton gauzes and t-shirt cloths exposed to the headspace of SP, the simulated IED, and Getxent tubes exposed to direct contact with SP were statistically lower than their response rate of actual SP. The response rate to SP was not different across all 15 testing sessions suggesting that the intermittent schedule of reinforcement, maintained dog motivation and performance. Data show that the outlined method is a good approach to study generalization in detection dogs. These results also highlight dog generalization to SP varieties and associated odors.

Effect of Schedule-Induced Behavior on Responses of Spontaneously Hypertensive and Wistar-Kyoto Rats in a Delay-Discounting Task: A Preliminary Report.

Delay discounting is the loss of the subjective value of an outcome as the time to its delivery increases. It has been suggested that organisms can become more tolerant of this delay when engaging in schedule-induced behaviors. Schedule-induced behaviors are those that develop at a high rate during intermittent reinforcement schedules without the need of arranged contingency to the reinforcer, and they have been considered as a model of compulsivity. There is evidence that relates compulsivity to greater delay discounting. The rate of delay discounting represents how impulsive the subject is, as the rate of discounting increases the higher the impulsivity. Thus, the main purpose of this study was to undertake a preliminary evaluation of whether developing schedule-induced behaviors affects performance in a delay-discounting task, by comparing spontaneously hypertensive rats (SHRs) and Wistar–Kyoto (WKY) rats. The rats were exposed to a task that consisted of presenting the subjects with two levers: one produced a small, immediate food reinforcer while the other one produced a larger, delayed reinforcer. During Condition A, the levers were presented, and a water bottle and a running wheel were available in the conditioning chambers; during Condition B, only the levers were presented. SHR and WKY rats developed schedule-induced behaviors during Condition A and showed no difference in discounting rates, contradicting previous reports. Lick allocation during response-reinforcer delays and the inter-trial interval (ITI) showed, respectively, pre- and post-food distributions. Discounting rates during Condition B (when rats could not engage in schedule-induced behaviors) did not reach statistical significance difference among strains of animals, although it was observed a tendency for WKY to behave more self-controlled. Likewise it was not found any effect of schedule-induced behavior on discounting rates, however, a tendency for WKY rats to behave more impulsive during access to drink and run seems to tentatively support the idea of schedule-induced behavior as a model of compulsivity in those rats, being impulsivity simply defined as an excess in behavior.

O ESTUDO DA PUNIÇÃO COM O JATO DE AR QUENTE: EFEITO DE DIFERENTES HISTÓRIAS DE REFORÇAMENTO

Uma das variáveis que tem influência sobre os efeitos de contingências punitivas é a história de reforçamento. Estudos com choque elétrico demonstram que esquemas de reforçamento contínuo produzem menor resistência à punição, do que esquemas de reforçamento intermitentes. O objetivo desse estudo foi verificar o efeito de um estímulo punidor alternativo - o jato de ar quente (JAQ) - apresentado de acordo com um esquema FR1, sobre a frequência de respostas operantes fortalecidas por diferentes histórias de reforçamento. Seis ratos foram expostos a diferentes histórias de reforçamento durante oito sessões (reforçamento contínuo, razão variável e intervalo variável). Em seguida, os animais passaram por seis sessões em que cada resposta de pressão à barra produzia um JAQ e uma gota de água. Por fim, os animais passaram por seis sessões de recondicionamento, com os mesmos esquemas utilizados na fase de fortalecimento. Durante as sessões de punição, todos sujeitos, independente da história de reforçamento apresentaram total supressão do responder. Quando a punição foi descontinuada todos sujeitos retornaram aos níveis prévios de resposta compatíveis com suas respectivas histórias de reforçamento, os sujeitos do grupo de reforçamento contínuo, contudo, levaram mais tempo para recuperar as taxas de respostas. Limitações metodológicas que comprometeram as conclusões do estudo são discutidas no intuito de aperfeiçoamento.Palavras-chave: punição; história de reforçamento; jato de ar quente; controle aversivo.

連続強化スケジュール,間欠強化スケジュール,消去がハトのキーつつき反応の位置次元および距離次元における行動変動性に及ぼす効果(特集号 行動変動性の実験研究とその応用可能性22巻2号追加論文)

連続強化スケジュール,間欠強化スケジュール,消去がハトのキーつつき反応の位置次元および距離次元における行動変動性に及ぼす効果(特集号 行動変動性の実験研究とその応用可能性22巻2号追加論文)

Pausing as an operant: Choice and discriminated responding

The effects of intermittent schedules of reinforcement for pausing were evaluated in two experiments. In Experiment 1, across a series of conditions, a variable-interval (VI) baseline schedule, in which pigeons' key pecks produced food, alternated with conditions in which food was delivered according to a concurrent VI (for key pecking) tandem variable-time differential-reinforcement-of-other-behavior (DRO) 5-s schedule. Time spent pausing within a session was proportional to the reinforcement rates associated with the tandem schedule. To examine the control of pausing by antecedent events, Experiment 2 arranged a multiple schedule in which pecking and pausing in either component were maintained according to concurrent schedules like those used in the first experiment. The availability of reinforcement for pausing was signaled in one component while signals uncorrelated with reinforcement were presented in the other. Signaled reinforcement for pausing, relative to the presentation of uncorrelated signals, decreased time spent pausing, a finding consistent with existing research on the effects of signaled VI reinforcement for key pecking in pigeons. The results of the two experiments show that pausing functions as an operant in much the same way that discrete responses, like key pecks, do, and that pausing and other operants are similarly affected by both antecedent and consequent events.

Resistance to extinction and behavioral momentum

In the metaphor of behavioral momentum, reinforcement is assumed to strengthen discriminated operant behavior in the sense of increasing its resistance to disruption, and extinction is viewed as disruption by contingency termination and reinforcer omission. In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.

Interval, blocking, and marking effects during the development of schedule-induced drinking in rats.

Schedule-induced drinking (SID) can occur when food-deprived rats are given access to water while receiving pellets on an intermittent reinforcement schedule. These conditions can increase water intake excessively. The possible role of adventitious reinforcement of postpellet drinking was assessed by testing whether response-reinforcer contiguity, the relative predictiveness of a response, and whether it is marked are important in the development of SID. Rats exposed to a short interpellet interval acquired SID most rapidly, with this acquired drinking response maintained when animals were transferred to a longer interpellet interval, thus indicating an easy-to-hard effect (Experiment 1). Further experiments demonstrated that a stimulus (a brief-flashing house light) occurring prior to pellet delivery could block the acquisition of SID (Experiment 2), while a lick-contingent tone, intended to increase the associability of this response, produced more rapid acquisition of SID (Experiment 3). Analysis of lick distributions revealed that licking became concentrated in the first half of an interpellet interval only after several sessions. Overall, the results indicated that similar factors affect the acquisition of both SID and instrumental conditioning with delayed reinforcement, as is consistent with a superstitious conditioning account of SID development.

Using differential reinforcement to improve equine welfare: Shaping appropriate truck loading and feet handling

Inappropriate behavior during common handling procedures with horses is often subject to aversive treatment. The present study replicated and extended previous findings using differential reinforcement to shape appropriate equine handling behavior. In Study 1, a multiple baseline across subjects design was used with four horses to determine first the effects of shaping target-touch responses and then successive approximations of full truck loading under continuous and intermittent schedules of reinforcement. Full loading responses were shaped and maintained in all four horses and occurrences of inappropriate behaviors reduced to zero. Generalization of the loading response was also observed to both a novel trainer and trailer. In Study 2, a changing criterion design was used to increase the duration of feet handling with one horse. The horse's responding reached the terminal duration criterion of 1min and showed consistent generalization and one-week maintenance. Overall, the results of both studies support the use of applied equine training systems based on positive reinforcement for increasing appropriate behavior during common handling procedures.

EXAMINING THE GENERALITY OF CHILDREN'S PREFERENCE FOR CONTINGENT REINFORCEMENT VIA EXTENSION TO DIFFERENT RESPONSES, REINFORCERS, AND SCHEDULES

Studies that have assessed whether children prefer contingent reinforcement (CR) or noncontingent reinforcement (NCR) have shown that they prefer CR. Preference for CR has, however, been evaluated only under continuous reinforcement (CRF) schedules. The prevalence of intermittent reinforcement (INT) warrants an evaluation of whether preference for CR persists as the schedule of reinforcement is thinned. In the current study, we evaluated 2 children's preference for contingent versus noncontingent delivery of highly preferred edible items for academic task completion under CRF and INT schedules. Children (a) preferred CR to NCR under the CRF schedule, (b) continued to prefer CR as the schedule of reinforcement became intermittent, and (c) exhibited a shift in preference from CR to NCR as the schedule became increasingly thin. These findings extend the generality of and provide one set of limits to the preference for CR. Applied implications, variables controlling preferences, and future research are discussed.

Increasing Eye Contact Toward Learning Materials in a Toddler with Autism

Individuals with Autism Spectrum Disorder often display deficits in eye contact that limit communicating with others, completing tasks, and attending to relevant stimuli. The participant in this study was a two-and-a-half-year-old boy with autism who seldom looked at materials while attempting three different tasks—assembling the pieces of a puzzle, stacking objects, and looking at pictures. This study compared rates of eye contact toward three target materials under two conditions: when food reinforcers were provided when he made eye contact with task materials and when they were withheld. An initial baseline phase was followed by a continuous reinforcement schedule in the first experimental phase that subsequently was thinned to an intermittent reinforcement schedule in the final phase. Results demonstrated an increase in eye contact toward target materials under both continuous and intermittent reinforcement conditions.

EFFECTS OF DIFFERING RESPONSE‐FORCE REQUIREMENTS ON FOOD‐MAINTAINED RESPONDING IN CD‐1 MICE

The effect of force requirements on response effort was examined using outbred (CD-1) mice trained to press a disk with their snout. Lateral peak forces greater than 2 g were defined as threshold responses (i.e., all measured responses). Different force requirements were used to define criterion responses (a subclass of threshold responses) that exceeded the requirement. The reinforcer was sweetened, condensed milk, and it was delivered upon response termination. All mice were exposed to two ascending series of criterion force requirements (2, 4, 8, 16, and 32 g). Increasing the force requirement decreased criterion response rates, but increased threshold response rates. The time-integral of force (area under the force-time curve for individual responses, which is proportional to energy expenditure for each response) increased with the increase in the force requirement. These results conflict with the hypothesis that higher force requirements have aversive qualities and suggest that increased force requirements are more analogous to intermittent schedules of reinforcement. These data suggest that estimations of effort or energy expenditure should be measured independently of the force requirement. Individual differences in responding were found for the CD-1 outbred stock.

Schedule-Induced and Operant Mechanisms that Influence Response Variability: A Review and Implications for Future Investigations

Response variability, a fundamental characteristic of behavior, may be in some cases an induced effect of reinforcement schedules. Research on schedule-induced response variability has shown that continuous reinforcement results in less variability than intermittent reinforcement schedules. Studies on the effects of intermittency of reinforcement, periodicity of reinforcement, and type of schedule have resulted in mixed findings. Contingencies have also been arranged to directly influence operant response variability. These include lag reinforcement schedules, differential reinforcement of novel behavior, differential reinforcement of less frequent behavior, and percentile reinforcement schedules. These procedures are discussed in terms of practicality and implications for use in applied settings. Contingencies and treatment packages that indirectly influence response variability are addressed in terms of response allocation, response generalization, and response covariation. Studies on the effects of a variety of other variables on response variability are also reported, such as levels of food deprivation and drugs. Finally, directions for applied research in response variability are suggested.

Schedule‐induced electrodermal responding in children

The current studies were designed to determine whether different intermittent schedules of reinforcement would have a differential effect on the skin conductance responses (SCRs) of children in the 8-12-year-old age range. Results of two experiments indicated that the amplitude of children's SCRs immediately following the occurrence of reinforcement were a function of the schedule of reinforcement, with larger SCRs associated with longer inter-reinforcement intervals. The findings are consistent with the research literature suggesting that long inter-reinforcement intervals can be evocative of aversively motivated emotions, as well as with research on the effects of interstimulus intervals on SCRs. Possible implications for these findings are discussed.

Response Inhibition During Differential Reinforcement of Low Rates (DRL) Schedules May Be Sensitive to Low-Level Polychlorinated Biphenyl, Methylmercury, and Lead Exposure in Children

BackgroundAnimal studies have shown that exposure to common, low-level environmental contaminants [e.g., polychlorinated biphenyls (PCBs), lead] causes excessive and inappropriate responding on intermittent reinforcement schedules. The Differential Reinforcement of Low Rates task (DRL) has been shown to be especially sensitive to low-level PCB exposure in monkeys.ObjectivesWe investigated the relationships between prenatal PCB and postnatal Pb exposure performance on a DRL schedule in children. We predicted that a) prenatal PCB exposure would reduce interresponse times (IRTs) and reinforcements earned, and b) postnatal Pb exposure would reduce IRTs and reinforcements earned.MethodsWe tested 167 children on a DRL20 (20 sec) reinforcement schedule, and recorded IRTs and the number of reinforced responses across the session. We measured prenatal PCB exposure (cord blood), methylmercury (MeHg) (maternal hair), and postnatal Pb exposure (venous blood), and > 50 potentially confounding variables.ResultsResults indicated impaired performance in children exposed to PCBs, MeHg, and Pb. Children prenatally exposed to PCBs responded excessively, with significantly lower IRTs and fewer reinforcers earned across the session. In addition, exposure to either MeHg or Pb predicted statistically significant impairments of a similar magnitude to those for PCBs, and the associated impairments of all three contaminants (PCB, MeHg, and Pb) were statistically independent of one another.ConclusionsThese results, taken with animal literature, argue the high sensitivity of DRL performance to low-level PCB, MeHg, and Pb exposure. Future research should employ behavioral tasks in children, such as DRL, that have been demonstrably sensitive to low-level PCB, MeHg, and Pb exposure in animals.