Variable-interval 30-s Schedule Research Articles

Resurgence increases with repetition

Resurgence refers to a behavioral process in which a recent response is extinguished (or reinforcement conditions worsen) and a previously extinguished response recurs. In previous research, resurgence has been reliably produced when the resurgence procedure is repeated. Changes in the degree of increase of the resurging response across iterations of the procedure have been inconsistent, however, with some studies showing increases and some showing no changes or decreases in resurgence magnitude. The present study examined the nature of resurgence across repeated iterations of the conventional resurgence procedure by exposing four pigeons to the resurgence procedure six times in succession. In the first condition of the resurgence procedure, pecks on one key (e.g., the left) were reinforced under a variable-interval (VI) 30-s schedule. In the second condition pecks on that key were extinguished, and pecks on another key (e.g., the right) were reinforced under the same schedule. In the final condition there were no programmed consequences for either response. Resurgence was observed in 21 of 24 opportunities (87.5%). Iteration-over-iteration increases in resurgence were observed in 15 of 20 opportunities (75.0%), and this increase was found to be statistically significant. These findings demonstrate that, under certain conditions, resurgence generally increases in magnitude with repeated exposure to the procedures that generate it.

Resurgence during transitions from variable- to fixed-interval schedules

The effects of local periods of extinction on resurgence following transitions from variable-interval (VI) to fixed-interval (FI) schedules were studied using four pigeons exposed to a within-session resurgence procedure. Each session was divided into a Training (T) Alternative-Reinforcement (AR), and Resurgence Test (RT) phase. During the T phase, key pecking was reinforced under a VI 60-s schedule on one key. In the AR phase, responses reinforced in the T phase were extinguished, while responses to a different key were reinforced under a VI 90-s schedule. Next, responding to the same key that produced reinforcers in the AR phase was reinforced according to four different RT conditions: RT phase I (FI 90 s), RT phase II (FI 180 s), RT phase III (FI 45 s), or RT phase IV (extinction). The frequency of resurgence generally was an inverse function of the rate of reinforcement in the RT phase. Resurgence occurred less often when reinforcers were delivered under the FI 45-s schedule and more often under leaner schedules in the RT phase, peaking under extinction. The results show that resurgence may occur during local periods of extinction, with larger and more consistent effects occurring when the rate of reinforcement in the RT condition is leaner than it was during the preceding AR phase.

Does DRO type matter?: Cycle versus resetting contingencies in eliminating responding

Following lever-press training on variable-interval 30-s schedules, rats were exposed to three types of schedules designed to eliminate lever pressing. The first two were variations on what is called a differential-reinforcement-of-other-behavior (DRO, “zero rate”, or [target response] omission schedule) schedule. Under both variations, reinforcers were scheduled to occur in different conditions after either fixed or variable inter-reinforcer intervals (IRIs). Under one variation each lever press reset the time interval (i.e., “resetting DRO”) and under the other a reinforcer delivery scheduled at the end of an IRI was cancelled by the first response during the IRI (i.e., “cycle DRO”). In another condition reinforcers were delivered independently of responding after fixed or variable time periods. Each of the DRO procedures reduced response rates quickly and to near zero across ten sessions. The time schedules also reduced responding, albeit at a slower rate. The results extend the analogy of omission training to freeoperant avoidance to shock-deletion avoidance schedules.

An evaluation of resurgence in mice

Owing in part to the implications of resurgence for issues surrounding human health and adaptive problem solving, a substantial amount of laboratory research has been dedicated to understanding why this form of relapse occurs and what factors affect it. This research, however, has never been extended to mice, leaving unknown the specific experimental parameters that are appropriate for studying resurgence in this species. Two experiments were conducted in which mice were exposed to a three-phase resurgence preparation in a multiple-baseline-across-subjects design. In Phase 1, pressing a target lever produced food according to a variable-interval 15-s schedule. In Phase 2, target-lever pressing was extinguished, and nose poking produced reinforcement according to either a variable-interval 15-s (Experiment 1) or fixed-ratio 1 (Experiment 2) schedule. Finally, alternative reinforcement was suspended to test for resurgence. Lever pressing relapsed for one of four mice in Experiment 1, but all four mice demonstrated relapse in Experiment 2. Thus, relatively dense schedules of alternative reinforcement may be required to reliably study resurgence of previously reinforced responding in mice. These findings are instructive for researchers who are interested in studying resurgence of mice’s behavior and further demonstrate the cross-species generality of this form of relapse.

Resurgence and downshifts in alternative reinforcement rate.

Resurgence refers to an increase in a previously suppressed target behavior with a relative worsening of conditions for a more recently reinforced alternative behavior. This experiment examined the relation between resurgence and the magnitude of a reduction in the rate of reinforcement for the alternative behavior. Groups of both male and female rats initially pressed a target lever for food on a variable-interval (VI) 30-s schedule. In a second phase, responding to the target lever was extinguished for all groups and pressing an alternative lever was reinforced on a VI 10-s schedule. Next, the rate of reinforcement for alternative behavior was reduced differentially across groups by arranging extinction, VI 80-s, VI 40-s, VI 20-s, or continued VI 10-s reinforcement. Target responding increased as an exponential function of the magnitude of the reduction in alternative reinforcement rates. With the exception that males appeared to show higher rates of target responding in baseline and higher rates of alternative responding in other phases, the overall pattern of responding across phases was not meaningfully different between sexes. The pattern of both target and alternative response rates across sessions and phases was well described quantitatively by the Resurgence as Choice in Context model.

The effects of force requirements on microstructure: Subcriterion responses and the mismeasurement of bouts.

The effects of response force on microstructure were evaluated. A strain-gauge operandum permitted the manipulation of the force required to produce reinforcers (criterion responses) independently from the force defining response threshold. Thus, we could detect subcriterion forces that fell short of the force criterion. Eight rats earned food according to variable-interval (VI) 30- and 120-s schedules. The force requirements were set to 5.6 or 32.0 g; the response threshold was fixed at 5.6 g. Interresponse times were computed when subcriterion responses were both included and omitted from the analysis. Log-survivor functions of interresponse times showed that increasing force requirements elevated the mean between-bout interval of the VI 120-s schedule, but only if subcriterion behavior was excluded. Omitting subcriterion responses thus leads to overestimation of intervals separating response bouts. Increasing force requirements also increased the skewness of the between-bout distribution. A subsequent analysis found that subcriterion responses are most plentiful following reinforcer delivery, which helps to explain why their omission might inflate between-bout intervals, as this period is an important transition from reinforcer consumption to engagement in operant activity. The data suggest caution interpreting the effects of force on microstructure when subcriterion behavior is not or cannot be measured.

Repeated Operant Response Resurgence with a Peak-Interval Procedure

Resurgence is most often defined as the recurrence of previously reinforced behavior during extinction of another, more recently reinforced behavior. It provides a framework for deciphering the origin of operant responses, studying the effects of historical variables, and understanding socially relevant behaviors. The present manuscript proposes a novel procedure that incorporates the peak-interval procedure, typically used to examine temporal control, to examine resurgence repeatedly. Three experimentally naive Long Evans rats were trained using a concurrent variable-interval (VI) 30-s fixed-interval (FI) 30-s schedule, with each schedule programmed on a different lever. After stability was reached, responding under the VI 30-s schedule was extinguished for 15 sessions. A peak-interval procedure then was employed to examine resurgence of the previously VI responding. The peak-interval procedure consisted of FI 30-s trials intermixed with 300-s peak trials, each separated by 20-s blackouts. Resurgence of operant responding was obtained and examined within and across sessions.

Punishment and the potential for negative reinforcement with histamine injection.

The present study examined punishment of responding with histamine injection, and its potential to generate avoidance of punishment. Sprague-Dawley rats were trained under concurrent schedules in which responses on one lever (the punishment lever) produced food under a variable-interval schedule, and under some conditions intermittent injections of histamine, which suppressed behavior. Responses on a second (avoidance) lever prevented histamine injections scheduled on the punishment lever. After stabilization of punished responding, a variable-interval 15-s schedule of cancellation of histamine (avoidance) was added for responding on the second/avoidance lever, without subsequent acquisition of responding on that lever. Progressive decreases in the length of the punishment variable-interval schedule increased suppression on the punishment lever without increases in response rates on the avoidance lever. Exchanging contingencies on the levers ensured that response rates on the avoidance lever were sufficiently high to decrease the histamine injection frequency; nonetheless response rates on the avoidance lever decreased over subsequent sessions. Under no condition was responding maintained on the avoidance lever despite continued punishing effectiveness of histamine throughout. The present results suggest that avoidance conditioning is not a necessary condition for effective punishment, and confirm the importance of empirical rather than presumed categorization of behavioral effects of stimulus events.

The role of response force on the persistence and structure of behavior during extinction.

Behavior Momentum Theory has emerged as a prominent account of resistance to change in both basic and applied research. Although laboratory studies often define precise, repeatable responses, application research often deals with response classes that may vary widely along a number of dimensions. In general, Behavior Momentum Theory has not addressed how response dimensions impact resistance to change, providing an opportunity to expand the model in new directions. Four rats pressed a force transducer under a multiple variable interval (VI) 60-s VI 60-s schedule of reinforcement. In one component, responses satisfied the schedule only if the response force fell within a "low" force band requirement; responses in the other schedule were required to satisfy a "high" force band. Once responding stabilized, extinction was programmed for three sessions. Then, the procedures were replicated. The results showed that response force came under discriminative control, but force requirements had no impact on resistance to extinction. In a follow-up condition, the schedule was changed to a multiple VI 30-s VI 120-s schedule and the low-force band operated in both components. The results showed that behavior maintained by the VI 30-s schedule was generally more resistant to extinction. A secondary analysis showed that force distributions created under baseline maintained during extinction. Overall, the results suggest that differential response force requirements prevailing in steady state do not affect the course of extinction.

Multiple schedules, off-baseline reinforcement shifts, and resistance to extinction.

Resistance to extinction in a target multiple-schedule component varies inversely with the rate of reinforcement arranged in an alternative component during baseline. The present experiment asked whether changing the reinforcer rate in an alternative component would impact extinction of target component responding if those changes occurred in an off-baseline phase during which the target component was never experienced. Pigeons' key pecking was studied in three types of conditions, and each condition consisted of three phases. In Phase 1, pecking produced food in the target and alternative components of a multiple schedule according to variable-interval 60-s schedules. In Phase 2, the alternative-component stimulus was presented alone in a single schedule. Pecking during this phase produced the same reinforcer rate as in baseline in the Control condition, a higher rate of food (variable-interval 15 s) in the High-Rate condition, or was extinguished in the Extinction condition. Extinction of target- and alternative-component key pecking then was assessed in a multiple schedule during the final phase of each condition. Resistance to extinction of target-component key pecking was the same between the Control and High-Rate conditions but lower in the Extinction condition. These findings are discussed in terms of discrimination and generalization processes.

Stimuli previously associated with reinforcement mitigate resurgence

Resurgence refers to the recurrence of an extinguished target behavior following subsequent suspension of alternative reinforcement. Delivery of reinforcers during extinction of alternative behavior has been shown to mitigate resurgence. The present experiment aimed to determine whether delivering stimuli associated with reinforcers during resurgence testing similarly mitigates resurgence. Three groups of rats pressed target levers for food according to variable-interval 15-s schedules during Phase 1. In Phase 2, lever pressing was extinguished, and an alternative nose-poke response produced alternative reinforcement according to a variable-interval 15-s schedule. Food reinforcement was always associated with illumination of the food aperture and an audible click from the pellet dispenser during Phases 1 and 2. Phase 3 treatments differed between groups. For one group, nose poking continued to produce food and food-correlated stimuli. Both of these consequences were suspended for a second group. Finally, nose poking produced food-correlated stimuli but not food for a third group. Target-lever pressing resurged in the group that received no consequences and in the group that received only food-correlated stimuli for nose poking. Resurgence, however, was smaller for the group that received food-correlated stimuli than for the group that received no consequences for nose poking. Target-lever pressing did not increase between phases in the group that continued to receive food and associated stimuli. Thus, delivery of stimuli associated with food reinforcement after suspension of food reduced but did not eliminate resurgence of extinguished lever pressing. These findings contribute to potential methodologies for preventing relapse of extinguished problem behavior in clinical settings.

The effect of lever height on the microstructure of operant behavior

The effect of lever height on the temporal organization of reinforced lever pressing was examined. Lever pressing was reinforced on a variable-interval 30-s schedule in rats, with lever height manipulated across six successive conditions. Parameters of the organization of responses in bouts (bout length distribution, bout-initiation rate, within-bout rate, and sequential dependency) were estimated. These estimates revealed (1) a qualitative change in the distribution of IRTs and their sequential dependency when the lever was too high, (2) a mixture of geometrically-distributed bout lengths at all lever heights, and (3) longer bouts at lower and intermediate lever heights. In accordance with previous data, these findings suggest that lower and intermediate lever heights favored lever pressing with longer bout lengths, faster bout initiation, faster within-bout responding, and more sequentially dependent timing. These results underscore the disociability of motoric capacity in operant performance, and may reflect the influence of the body size on the temporal organization of the operant.

Progesterone attenuates impulsive action in a Go/No-Go task for sucrose pellets in female and male rats

Impulsivity, or a tendency to act without anticipation of future consequences, is associated with drug abuse. Impulsivity is typically separated into two main measures, impulsive action and impulsive choice. Given the association of impulsivity and drug abuse, treatments that reduce impulsivity have been proposed as an effective method for countering drug addiction. Progesterone has emerged as a promising treatment, as it is associated with decreased addiction-related behaviors and impulsive action. The goal of the present study was to determine the effects of progesterone (PRO) on impulsive action for food: a Go/No-Go task. Female and male rats responded for sucrose pellets during a Go component when lever pressing was reinforced on a variable-interval 30-s schedule. During the alternate No-Go component, withholding a lever press was reinforced on a differential reinforcement of other (DRO) behavior 30-s schedule, where a lever press reset the DRO timer. Impulsive action was operationally defined as the inability to withhold a response during the No-Go component (i.e. the number of DRO resets). Once Go/No-Go behavior was stable, responding between rats treated with PRO (0.5mg/kg) or vehicle was examined. Progesterone significantly decreased the total number of DRO resets in both males and females, but it did not affect VI responding for sucrose pellets. This suggests that PRO decreases motor impulsivity for sucrose pellets without affecting motivation for food. Thus, PRO may reduce motor impulsivity, a behavior underlying drug addiction.

Wheel-running reinforcement in free-feeding and food-deprived rats

Rats experiencing sessions of 30min free access to wheel running were assigned to ad-lib and food-deprived groups, and given additional sessions of free wheel activity. Subsequently, both ad-lib and deprived rats lever pressed for 60s of wheel running on fixed ratio (FR) 1, variable ratio (VR) 3, VR 5, and VR 10 schedules, and on a response-initiated variable interval (VI) 30s schedule. Finally, the ad-lib rats were switched to food deprivation and the food-deprived rats were switched to free food, as rats continued responding on the response-initiated VI 30-s schedule. Wheel running functioned as reinforcement for both ad-lib and food-deprived rats. Food-deprived rats, however, ran faster and had higher overall lever-pressing rates than free-feeding rats. On the VR schedules, wheel-running rates positively correlated with local and overall lever pressing rates for deprived, but not ad-lib rats. On the response-initiated VI 30s schedule, wheel-running rates and lever-pressing rates changed for ad-lib rats switched to food deprivation, but not for food-deprived rats switched to free-feeding. The overall pattern of results suggested different sources of control for wheel running: intrinsic motivation, contingencies of automatic reinforcement, and food-restricted wheel running. An implication is that generalizations about operant responding for wheel running in food-deprived rats may not extend to wheel running and operant responding of free-feeding animals.

Mechanisms of impulsive choice: III. The role of reward processes

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10s) and an LL reward (2 and 4 pellets, 30s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

Sensitivity to pre- and post-reinforcer delays in self-control choice

Rats were exposed to a concurrent-chains schedule in which reinforcer amounts, pre-reinforcer and post-reinforcer delays were varied between alternatives. Two independent variable-interval 60-s schedules were in effect during the initial links. Pre- and post-reinforcer delays were defined by fixed-time schedules. In one condition, rats chose between a larger reinforcer delayed by 0, 40, 80, or 120-s and an immediate smaller reinforcer. In the other condition, rats chose between an immediate larger reinforcer terminated with 0, 40, 80, or 120-s post-reinforcer delays and an immediate smaller reinforcer terminated with no post-reinforcer delay. In both conditions, choice was predicted from overall reinforcement density (i.e. reinforcer amount divided by total trial time). The results indicate that the preference for the larger reinforcer systematically decreased as both pre- and post-reinforcer delays increased. Although the effect of pre-reinforcer delay was greater than that of post-reinforcer delay, post-reinforcer delay itself influenced rats' choices in the present self-control situation. In conclusion, the present study demonstrated the limitation of local reinforcement density model of the matching law and all time components that constitute a choice alternative need to be weighted and incorporated into the general model of choice behavior.

Resurgence in Siamese fighting fish, Betta splendens

Resurgence of previously reinforced responding was investigated in male Siamese fighting fish (Betta splendens). Swimming through a ring produced 15-s mirror presentations according to, with different fish, either a fixed-ratio 1 or a variable-interval 60-s schedule of reinforcement. When responding was stable, a differential-reinforcement-of-other-behavior schedule was substituted for the mirror-presentation schedule. Following this, mirror presentations were discontinued (extinction). During this latter phase, there were transient increases in the ring-swim response relative to the frequency of such responding during the differential-reinforcement-of-other behavior schedule. Resurgence was similar for the fish exposed previously to the fixed-ratio or to the variable-interval schedule. These results extend to Siamese fighting fish a well-established behavioral phenomenon previously not observed in this species or with this response topography, and only rarely reported following the removal of a non-consumable reinforcer.

On the joint control of preference by time and reinforcer-ratio variation

Five pigeons were trained in a procedure in which, with a specified probability, food was either available on a fixed-interval schedule on the left key, or on a variable-interval schedule on the right key. In Phase 1, we arranged, with a probability of 0.5, either a left-key fixed-interval schedule or a right-key variable-interval 30s, and varied the value of the fixed-interval schedule from 5s to 50s across 5 conditions. In Phase 2, we arranged either a left-key fixed-interval 20-s schedule or a right-key variable-interval 30-s schedule, and varied the probability of the fixed-interval schedule from 0.05 to 1.0 across 8 conditions. Phase 3 always arranged a fixed-interval schedule on the left key, and its value was varied over the same range as in Phase 1. In Phase 1, overall preference was generally toward the variable-interval schedule, preference following reinforcers was initially toward the variable-interval schedule, and maximum preference for the fixed-interval schedule generally occurred close to the arranged fixed-interval time, becoming relatively constant thereafter. In Phase 2, overall left-key preference followed the probability of the fixed-interval schedule, and maximum fixed-interval choice again occurred close to the fixed-interval time, except when the fixed-interval probability was 0.1 or less. The pattern of choice following reinforcers was similar to that in Phase 1, but the peak fixed-interval choice became more peaked with higher probabilities of the fixed interval. Phase 3 produced typical fixed-interval schedule responding. The results are discussed in terms of reinforcement effects, timing in the context of alternative reinforcers, and generalized matching. These results can be described by a quantitative model in which reinforcer rates obtained at times since the last reinforcer are distributed across time according to a Gaussian distribution with constant coefficient of variation before the fixed-interval schedule time, changing to extended choice controlled by extended reinforcer ratios beyond the fixed-interval time. The same model provides a good description of response rates on single fixed-interval schedules.This article is part of a Special Issue entitled: SQAB 2012.

Reinforced Behavioral Variability in Humans

This study aimed to assess the role of reinforced behavioral variability in the learning of a 6-digit target sequence (211212) with 3 groups of human participants (n = 39). For the first group (Control), only the target sequence was reinforced. For the second group (Any), the target sequence was reinforced, and any sequence other than the target sequence could be reinforced on a variable interval 60-s schedule. For the third group (Variable), the target sequence was reinforced, and any sequence other than the target sequence could be reinforced on a variable interval 60-s schedule, if it met a variability criterion. The Control group produced the target sequence significantly more often than the Variable group by the end of the experimental sessions. These findings contradict previous studies with rats that have shown that reinforcement of behavioral variability facilitates the learning of difficult response sequences but are consistent with results from previous studies with humans. Potential reasons for this disparity are discussed.

Environmental rearing effects on impulsivity and reward sensitivity.

Previous research has indicated that rearing in an enriched environment may promote self-control in an impulsive choice task. To further assess the effects of rearing environment on impulsivity, 2 experiments examined locomotor activity, impulsive action, impulsive choice, and different aspects of reward sensitivity and discrimination. In Experiment 1, rats reared in isolated or enriched conditions were tested on an impulsive choice procedure with a smaller-sooner versus a larger-later reward, revealing that the isolated rats valued the smaller-sooner reward more than the enriched rats. A subsequent reward challenge was presented in which the delay to the 2 rewards was the same but the magnitude difference remained. The enriched rats did not choose the larger reward as often as the isolated rats, reflecting poorer reward discrimination. Impulsive action was assessed using a differential-reinforcement-of-low-rate task, which revealed deficits in the enriched rats. In Experiment 2, rats reared in isolated, standard, or enriched conditions were tested on reward contrast and reward magnitude sensitivity procedures. The rats were presented with 2 levers that delivered different magnitudes of food on variable interval 30-s schedules. Across all tests, the enriched and social rats displayed more generalized responding to the small-reward lever, but a similar response to the large-reward lever, compared with the isolated rats. This confirmed the results of Experiment 1, indicating poorer reward discrimination in the enriched condition compared with the isolated condition. The results suggest that enrichment may moderate reward generalization/discrimination processes through alterations in incentive motivational processes.