Variable-time Schedule Research Articles

The effects of noncontingent reinforcement on an arbitrary response in domestic dogs (Canis lupus familiaris)

Noncontingent reinforcement (NCR) procedures entail the response-independent, time-based delivery of a reinforcer (e.g., delivery of a reinforcer on a fixed-time 15 s schedule – FT 15 s). Although NCR has been demonstrated to be effective for the treatment of undesired behaviours in human learners, very few studies have examined it in nonhuman animals. We investigated how NCR with and without extinction (another well-studied behavioural intervention) affected an arbitrary response in companion dogs. The ‘stepping on a rug’ response was shaped and established in lieu of a problem behaviour in six dogs. Stepping was then exposed to fixed-interval (FI 15 s), fixed-time (FT 15 s), a combination of these schedules of reinforcement (FT 15 s + FI 15 s), and an extinction-only condition. The results indicated that NCR was effective in reducing the target behaviour in three out of the six dogs. These effects were greater in the NCR-with-extinction procedure than in the NCR-without-extinction condition. Future research should investigate the effects of schedule thinning and other schedules of reinforcement (e.g., variable-time schedules) on the behaviour of companion dogs.

The ideal free distribution accounts for free‐operant domesticated dog foraging

The Ideal Free Distribution suggests the relative allocation of the number of organisms foraging across two or more resource sites will match the relative density of reinforcement patches in those resource sites. This initial project sought to develop a new method of foraging research for dogs by using a commercially available treat dispenser in basic behavioral research. The purpose of this study was to determine whether the Ideal Free Distribution equation could describe the behavior of the domesticated dog in a daycare setting. Toward this end, we recorded dog behavior in a free operant arrangement on various variable-time schedules of reinforcement. Results indicated matching occurred in the dog sample, which corresponded with the Ideal Free Distribution equation, with a slight visual appearance toward undermatching, consistent with extant literature on group foraging. Furthermore, there was a lack of bias in the current sample of foraging dogs. Implications and future directions involve extending foraging research in the domesticated dog. That the study occurred in a local dog daycare specifically highlights how practical concerns in animal welfare and/or training can be addressed in rigorous ways with strong conceptual foundations in the experimental analysis of behavior.

Kappa‐Opioid Receptors Modulate Innate and Learned Responses to Positive and Negative Reinforcers

There are few effective and approved treatments for Substance Use Disorders (SUDs) but clinical and preclinical research has shown therapeutic potential for Kappa-opioid receptor (KOR) antagonists. The leading theory of the KOR system's role in SUD posits that chronic substance use augments expression and release of the aversive KOR ligand, dynorphin, which produces a dysphoric state. In this model, upregulation of dynorphin/KOR activity drives continued substance use through a negative reinforcement mechanism whereby substances are consumed to alleviate the negative affective state. While systemic KOR antagonists have been shown to reduce drug consumption in several preclinical models, direct effects on positive and negative reinforcement schedules have not been systematically investigated. In the present study we investigate the effects of systemic antagonism of KORs by nor-Binaltorphimine (NorBNI) on acquisition and maintenance of operant responding reinforced by presentation of sucrose (positive reinforcement) or removal of footshock (negative reinforcement). Based on the prevailing theory that KOR activation is aversive and drives negative reinforcement in addiction, we hypothesized that antagonism of the system would result in decreased responding under the negative reinforcement contingency and have no effect on positive reinforcement. Male C57BL/6J mice received 10 mg/kg intraperitoneal injections of the long-acting KOR antagonist NorBNI or 10 ml/kg saline 24 hours prior to their first session. Animals acquired a discriminated operant response for a positive or negative reinforcer (separate cohorts) whereby a cue light above the active nose-poke (presented on a variable time schedule) served as a discriminative stimulus indicating when a nose-poke would result in the receipt of sucrose or the removal of a series of shocks. Contrary to our hypothesis, mice that received NorBNI showed increased performance during acquisition of both positive and negative reinforcement contingencies compared to saline control mice. These results suggest a novel role of the KOR system in learning that is independent of the valence of the stimulus. We also found differential innate responses to sucrose and footshock prior to operant conditioning suggesting a key role of the system in encoding information about stimuli both aversive and non-aversive. These results call for a reevaluation of the role of KOR signaling in behavioral control and how this system influences drug taking behaviors.

Ecological Location of a Water Source and Spatial Dynamics of Behavior Under Temporally Scheduled Water Deliveries in a Modified Open-Field System: An Integrative Approach.

It has been reported in non-contingent schedules that the variety of patterns of behavior is affected by the temporal variation of water deliveries. While temporal variation is accomplished by delivering water at fixed or variable times, spatial variation is usually accomplished by varying the number of dispensers and distance among them. Such criteria do not consider the possible ecological relevance of the location of water dispensers. Nevertheless, it is plausible to suppose that the intersection of the programed contingencies (e.g., time-based schedules), the ecological differentiated space (e.g., open vs. closed zones), and the relative location of relevant objects and events (e.g., location of the water source—peripherical vs. center zone) could set up an integrated system with the behavioral patterns of the organism. In the present study, we evaluated the eco-functional relevance of two locations of the dispensers upon behavioral dynamics in Wistar rats using fixed and variable time schedules in a modified open-field system. In Experiment 1, three subjects were exposed to a fixed time 30-s water delivery schedule. In the first condition, the water dispenser was located at the center of the experimental chamber. In the second condition, the water dispenser was located at the center of a wall of the experimental chamber. Each location was present for 20 sessions. In Experiment 2, conditions were the same, but a variable time schedule was used. Routes, distance to the dispenser, recurrence patterns, time spent in zones, entropy, and divergence were analyzed. Our findings suggest a robust differential relevance of the location of the dispensers that should be considered in studies evaluating behavioral dynamics. Results are discussed from an integrative, ecological-parametric framework.

The appearance of matching

A computerized version of Houston and McNamara’s (1981) model of concurrent variable time schedules was used to generate large data sets consisting of behavior inputs and the corresponding reinforcement outputs. The data was analyzed to assess the hypothesis that matching behavior observed with CONCVTVT (or CONCVIVI) schedules is an inevitable product of the operation of the schedule. The results suggest that the generalized matching law does not provide the best description of the results obtained with CONCVTVT schedules; that the parameter value of schedule components determine the framework of the reinforcement efficiency maps of behavior choices; and that by random chance alone matching-like behavior is likely to appear if the reinforcement generation efficiency of the behavior is high.

The role of adventitious reinforcement during differential reinforcement of other behavior: A systematic replication

Differential reinforcement of other behavior (DRO) is commonly used to decrease problem behavior by presenting reinforcers contingent upon the absence of a target response. Although it is well demonstrated that DROs decrease response rates, the processes producing these decreases are not well understood. The present study systematically replicated previous research assessing whether adventitious reinforcement of alternative behavior contributes to the effectiveness of DRO. We presented university students with two options on a computer and reinforced target responding on a variable-ratio schedule. Next, we compared decreases in target-response rates and any increases in alternative responding during DRO schedules versus yoked variable-time schedules or extinction probes. DRO schedules resulted in the lowest target-response rate and highest alternative-response rate. These findings generally provide some support for the adventitious reinforcement of "other" behavior.

DOES “MUTUAL REINFORCEMENT” FUNCTION AS REINFORCEMENT?

A contingency under which a response by an individual yields a reinforcer delivered to another individual, and vice versa, sometimes has been called mutual reinforcement. The present experiments examined whether such so-called mutual reinforcement increases and maintains response rates. Following hopper training, two pairs of pigeons were exposed to a schedule of mutual reinforcement: a peck of the key by one pigeon permitted another pigeon to access food, and vice versa (Experiment 1). The results provided no evidence that the mutual reinforcement increased the response rates. In Experiment 2, a fixed-ratio 1 schedule was followed by a schedule of mutual reinforcement for two pairs of pigeons. Although response rates decreased to near zero across the 70 sessions with the mutual reinforcement in effect, the rates were higher than those under a variable-time schedule, suggesting that mutual reinforcement did not maintain a stable rate of responding,but it did postpone extinction of the response.

Discrimination reversal facilitates subsequent acquisition of temporal discriminations in rats' appetitive conditioning.

Three experiments with rats assessed the effects of introducing predictive ambiguity by reversing a Pavlovianly trained discrimination on subsequent context and temporal conditioning. The experience of discrimination reversal did not facilitate context conditioning when the food was presented on a variable time schedule (Experiment 1a). However, in Experiment 1b, discrimination reversal enhanced subsequent learning of a fixed temporal interval associated with unsignaled food presentation in comparison with consistent training. In Experiment 2, temporal discrimination after reversal and consistent training was compared with a naïve control. The experience of discrimination facilitated subsequent temporal conditioning with respect to the naïve control, and discrimination reversal enhanced temporal conditioning even further. In Experiment 3, reversal enhanced learning of the fixed temporal interval, regardless of whether it was relatively short or long (i.e., 30 s or 60 s). Results are discussed in terms of current associative theories of human and nonhuman conditioning and attention. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Behavior and illusions: a model to study superstition in a participant replacement experiment

The notion of superstitious behavior can provide a basic background for understanding such notions as illusions and beliefs. The present study investigated the social mechanism of the transmission of superstitious behavior in an experiment that utilized participant replacement. The sample was composed of a total of 38 participants. Participants performed a task on a computer: they could click a colored rectangle using the mouse. When the rectangle was in a particular color, the participants received points independently of their behavior (variable time schedule). When the color of the rectangle was changed, no points were presented (extinction). Under an Individual Exposure condition, ten participants worked alone on the task. Other participants were exposed to the same experimental task under a Social Exposure condition, in which each participant first learned by observation and then worked on the task in a participant replacement (chain) procedure. The first participant in each chain in the Social Exposure condition was a confederate who worked on the task “superstitiously,” clicking the rectangle when points were presented. Superstitious responding was transmitted because of the behavior of the confederate. This also influenced estimates of personal control. These findings suggest that social learning can facilitate the acquisition and maintenance of superstitious behavior and the illusion of control. Our data also suggest that superstitious behavior and the illusion of control may involve similar learning principles.

Response-reinforcer dependency and resistance to change.

The effects of the response-reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two-component multiple schedule. Across conditions, in the fixed component, all reinforcers were response-dependent; in the alternative component, the percentage of response-dependent reinforcers was 100, 50 (i.e., 50% response-dependent and 50% response-independent) or 10% (i.e., 10% response-dependent and 90% response-independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three-component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response-dependent or independent reinforcers. In the other component, response-dependent and -independent reinforcers were programmed by superimposing a variable-time schedule on an independent variable-interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple-schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response-reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.

Clinical translation of the ABA renewal model of treatment relapse

ABSTRACTAn 18-year-old male’s attention-maintained problem behavior was treated with an alternative (ALT) variable-interval, variable-time (VI VT) schedule arrangement with two therapists each delivering attention at different rates in a multiple schedule of reinforcement. Despite the differing rates of VI VT attention, the procedures resulted in similar rates of problem behavior in the last three treatment sessions reducing problem behavior by an average of 45%. Following treatment, the two therapists conducted separate extinction sessions, initially in a different context until problem behavior extinguished and then returning to the original treatment context (i.e., the ABA renewal test). Extinction took 4.4 times longer in the renewal test for the therapist who implemented the high-rate VI VT treatment compared with the therapist who implemented the low-rate VI VT treatment. Although these results are preliminary, the implications for clinicians is that treatment designs that use low-rate treatment may be more effective than high-rate treatment in preventing relapse.

Temporal distributions of schedule-induced licks, magazine entries, and lever presses on fixed- and variable-time schedules.

In this article, schedule-induced drinking (SID) refers to increased drinking by hungry rats exposed to intermittent delivery of food pellets. Two major accounts of SID differ in their explanation of why such drinking tends be concentrated soon after pellet delivery. Temporal discrimination theories propose that drinking is a form of displacement activity that occurs when a pellet is least likely. Adventitious reinforcement theories propose that drinking is displaced to early in an interpellet interval (IPI) by magazine-directed behavior that occurs toward the end of an IPI. The main aim of this study was to examine the latter response-competition account by recording distributions of both licking and magazine entries as SID developed when pellets were delivered to different groups either on a fixed-time (FT 30 s) or on a variable-time schedule (VT 30 s), as in Experiment 1. Although VT 30-s schedules produced essentially flat distributions of magazine entries, licking still tended to be concentrated early in an IPI. Furthermore, there was no indication (Experiments 1 and 2) that magazine entry distributions developed ahead of licking distributions. Experiment 3 examined distributions of lever presses instead of licks: Initially high rates of lever pressing declined both with response-independent schedules (FT and VT) and when a minimal response-dependency was introduced (recycling conjunctive schedule), yet this response also tended to be most frequent soon after pellet delivery. Overall, the data were generally consistent with temporal conditioning theories.

Effectiveness of Time-Based Attention Schedules on Students in Inclusive Classrooms in Turkey

This study examines the effectiveness of fixed-time (FT) and variable-time (VT) schedules and attention on the problem behaviors and on-task behaviors of students with and without intellectual disabilities in inclusive classrooms in Turkey. Three second-grade students with intellectual disabilities, three students without intellectual disabilities, and three teachers participated in this study. The ABAC reversal design of the singlesubject design was used. A1 and A2 were the baseline conditions; in the B condition, an FT schedule of attention (every four minutes) was implemented, and in the C condition, a VT schedule of attention (average four minutes) was utilized. The results demonstrated that FT and VT schedules of attention decreased problem behaviors, and increased on-task behaviors during lessons. Moreover, in the VT schedule of attention, gains were noted to be faster and more consistent, and teachers performed VT more easily

Reply to Pellón and Killeen's (2015) commentary on Boakes, Patterson, Kendig, and Harris (2015).

This reply notes the importance of data from variable-time schedules for resolving the issue of whether competition from magazine approach behavior is a major factor responsible for the concentration of schedule-induced drinking early in interpellet intervals. Pellón and Killeen propose that this temporal distribution of schedule-induced drinking reflects the effectiveness of adventitious reinforcement of this behavior over a delay, as detailed in their model. However, there is no evidence to support either of their key assumptions: (a) that schedule-induced behaviors are maintained by adventitious reinforcement ("are operants") and (b) that behaviors vary in their sensitivity to a delay before the subsequent occurrence of a reinforcer.

Clinical translation of animal models of treatment relapse

Behavioral Momentum Theory (BMT) has inspired animal models of treatment relapse. We translated the models of reinstatement and resurgence into clinical procedures to test whether relapse tests would replicate behavior pattern found in basic research. Following multiple schedule baseline reinforcement of a 16-year-old male's problem behavior at equal rates by two therapists, treatment was introduced using a variable-interval, variable-time (VI VT) schedule arrangement with therapists delivering reinforcers at different rates. Despite the differing rates of VI VT reinforcers, the treatment produced comparable reductions in problem behavior. Following successful treatment, the two therapists discontinued treatment and resumed reinforcement of problem behavior at equal rates that constituted a reinstatement of baseline conditions. As predicted by BMT, reinstatement resulted in an immediate return of high rates of problem behavior but was 2.6 times higher for the therapist using the higher rate VI VT treatment. A second treatment phase was implemented followed by a test of resurgence in a single extended extinction session conducted separately for each therapist. The unequal VI VT treatment rates by therapists resulted in 2.1 times greater responding in the resurgence test for the therapist who implemented the higher rate VI VT procedure. These results are consistent with basic research studies and BMT.

Observing of chain-schedule stimuli

A classical-conditioning account of the processes maintaining behavior under chained schedules entails a backward transmission of conditioned-reinforcement effects. Assessing this process in traditional chain schedules is limited because the response maintained by stimulus onset accompanied by each link in a chain schedule may also be maintained by the primary reinforcer. In the present experiment, an observing response was used to measure the conditioned-reinforcing effects of stimuli associated with a three-link chain variable-time (VT) food schedule, and resistance-to-change tests (extinction and prefeeding) were implemented to examine if a backward transmission of reinforcement effects occur. Four pigeons served as subjects. Observing was maintained by the production of stimuli correlated with links of a three-link chain VT schedule with the middle-link stimulus maintaining the highest rate of observing, followed by the initial-link stimulus and the terminal-link stimulus maintaining the lowest observing rate. Results from resistance-to-change tests of extinction and prefeeding were not supportive of a backward transmission of reinforcement effects and in general, the pattern of resistance-to-change was forward. Based on past and current research, it appears that a backward pattern of relative rate decreases in responses maintained by stimuli correlated with a chain schedule due to disruption (i.e., extinction and prefeeding) is not a ubiquitous process that is evident within different chain-schedule arrangements.

The impact of range anxiety and home, workplace, and public charging infrastructure on simulated battery electric vehicle lifetime utility

Battery electric vehicles (BEVs) offer the potential to reduce both oil imports and greenhouse gas emissions, but have a limited utility due to factors including driver range anxiety and access to charging infrastructure. In this paper we apply NREL's Battery Lifetime Analysis and Simulation Tool for Vehicles (BLAST-V) to examine the sensitivity of BEV utility to range anxiety and different charging infrastructure scenarios, including variable time schedules, power levels, and locations (home, work, and public installations). We find that the effects of range anxiety can be significant, but are reduced with access to additional charging infrastructure. We also find that (1) increasing home charging power above that provided by a common 15 A, 120 V circuit offers little added utility, (2) workplace charging offers significant utility benefits to select high mileage commuters, and (3) broadly available public charging can bring many lower mileage drivers to near-100% utility while strongly increasing the achieved miles of high mileage drivers.

EVALUATION OF RESISTANCE TO CHANGE UNDER DIFFERENT DISRUPTER CONDITIONS IN CHILDREN WITH AUTISM AND SEVERE INTELLECTUAL DISABILITY

Translational research inspired by behavioral momentum theory in the area of developmental disabilities has shown effects in individuals over a range of functioning levels. In the current study, behavioral momentum was assessed in 6 children diagnosed with autism and severe intellectual disability. In a repeated measures design, participants were exposed to relatively rich versus lean reinforcement contingencies in a multiple schedule with food reinforcers. This was followed by exposure to each of four disrupting conditions: prefeeding, presentation of a concurrent alternative stimulus, presentation of a movie, and the presence of a researcher dispensing response-independent reinforcers on a variable-time schedule. Consistently greater resistance to disruption in the component with the richer schedule occurred with the alternative stimulus disrupter but not with the other disrupters. These results suggest parameters that may be more (or less) effective if behavioral momentum inspired techniques are to be exploited in therapeutic environments.

Human risky choice: Delay sensitivity depends on reinforcer type.

The present study was designed to help bridge the methodological gap between human and nonhuman animal research in delay-based risky choice. In Part 1, 4 adult human subjects made repeated choices between variable-time and fixed-time schedules of 30-s video clips. Both alternatives had equal mean delays of 15 s, 30 s, or 60 s. Three of 4 subjects strongly preferred the variable-delay alternative across all conditions. In Part 2, these 3 subjects were then provided pairwise choices between 2 variable-time schedules with different delay distributions. Subjects generally preferred the variable-delay distributions with a higher probability of short-reinforcer delays, consistent with accounts based on nonlinear discounting of delayed reinforcement. There was only weak correspondence between experimental results and verbal reports. The overall pattern of results is inconsistent with prior risky choice research with human subjects but is consistent with prior results with nonhuman subjects, suggesting that procedural differences may be a critical factor determining risk-sensitivity across species.

A Methodology for Maintaining Low Levels of Attention-Maintained Problem Behaviors Following Variable-Time Schedule Thinning

Variable-time (VT) schedules of reinforcement deliver reinforcers contingent on the passage of a specified average amount of time. VT schedules that arrange for frequent reinforcer deliveries have been effective in producing immediate suppression of elevated levels of target behaviors. However, dense schedules may not be practical for caregivers and staff and a systematic methodology for thinning VT schedules is needed. This study employed VT schedules of attention to reduce attention-maintained problem behavior in a 14-year-old male. Following reduction in attention-maintained problem behaviors, the VT schedule was thinned. Results showed that a return to continuous attention following re-emergence of behaviors during schedule thinning resulted in maintenance of low levels of these behaviors and high levels of activity engagement.