Variable-interval Component Research Articles

COMBINATIONS OF RESPONSE‐DEPENDENT AND RESPONSE‐INDEPENDENT SCHEDULE‐CORRELATED STIMULUS PRESENTATION IN AN OBSERVING PROCEDURE

Pigeons pecked a response key on a variable-interval (VI) schedule, in which responses produced food every 40 s, on average. These VI periods, or components, alternated in irregular fashion with extinction components in which food was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli (houselight flashes) correlated with the component schedule currently in effect. Across conditions within a phase, the dependency between observing and presentation of the stimuli was decreased systematically while the density of stimulus presentation was held constant. Across phases, the proportion of session time spent in the VI component was adjusted from 0.5 to 0.25, and then to 0.75. Results indicate that rate of observing decreased as the dependency between responses and stimulus presentations was decreased. Further, discriminative control by the schedule-correlated stimuli was systematically weakened as dependency was decreased. Increasing the proportion of session time spent in VI decreased the rate of observing. This effect was additive with the manipulation of the dependency between observing and presentation of the stimuli. Overall, these results show that conditioned reinforcers function similarly to unconditioned reinforcers with respect to response-consequence dependencies, and that stimulus control is enhanced under conditions in which the relevant stimuli are produced by an organism's behavior.

Manual and Ocular Observing Behavior in Human Subjects

The present experiment analyzed manual and ocular observing behavior during a Wyckoff observing-response procedure. Following visual discrimination training with a multiple variable-interval extinction schedule, four human subjects pressed a key to earn points under the corresponding mixed schedule. Subjects could emit a manual response, touching a spot on a computer touchscreen, that produced S+ and S- stimuli correlated with the variable-interval and extinction components, respectively, thus converting the mixed schedule into a multiple schedule. Eye-tracking apparatus measured ocular observing behavior in both multiple and mixed schedules. Under the multiple schedule, subjects tended to look at S+ for longer durations than S-. Under the mixed schedule, in contrast, most subjects manually produced S- for longer durations than S+. Comparisons of the ocular and manual observing responses indicate that direct measurements of eye movements reveal characteristics of observing behavior that do not derive from manual observing responses alone.

Bouts of responding: the relation between bout rate and the rate of variable-interval reinforcement.

By nose poking a lighted key, rats obtained food pellets on either a variable-interval schedule of reinforcement or a schedule that required an average of four additional responses after the end of tile variable-interval component (a tandem variable-interval variable-ratio 4 schedule). With both schedule types, the mean variable interval was varied between blocks of sessions from 16 min to 0.25 min. Total rate of key poking increased similarly as a function of the reinforcer rate for the two schedule types, but response rate was higher with than without the four-response requirement. Analysis of log survivor plots of interresponse times showed that key poking occurred in bouts. The rate of initiating bouts increased as a function of reinforcer rate but was either unaffected or was decreased by adding the four-response requirement. Within-bout response rate was insensitive to reinforcer rate and only inconsistently affected by the four-response requirement. For both kinds of schedule, the ratio of bout time to between-bout pause time was approximately a power function of reinforcer rate, with exponents above and below 1.0.

Stimulus control of cocaine self-administration.

Environmental stimuli that set the occasion wherein drugs are acquired can "trigger" drug-related behavior. Investigating the stimulus control of drug self-administration in laboratory animals should help us better understand this aspect of human drug abuse. Stimulus control of cocaine self-administration was generated here for the first time using multiple and chained schedules with short, frequently-alternating components--like those typically used to study food-maintained responding. The procedures and results are presented along with case histories to illustrate the strategies used to produce this stimulus control. All these multicomponent schedules contained variable-interval (VI) components as well as differential-reinforcement-of-other-behavior (DRO) or extinction components. Schedule parameters and unit dose were adjusted for each rat to produce stable, moderate rates in VI components, with minimal postreinforcement (infusion) pausing, and response cessation in extinction and DRO components. Whole-body drug levels on terminal baselines calculated retrospectively revealed that all rats maintained fairly stable drug levels (mean, 2.3 to 3.4 mg/kg) and molar rates of intake (approximately 6.0 mg/kg/hr). Within this range, no relation between local VI response rates and drug level was found. The stimulus control revealed in cumulative records was indistinguishable from that achieved with food under these schedules, suggesting that common mechanisms may underlie the control of cocaine- and food-maintained behavior.

BRIEF‐STIMULUS PRESENTATIONS ON MULTIFORM TANDEM SCHEDULES

Three experiments examined the influence of a brief stimulus (a light) on the behavior of food-deprived rats whose lever pressing on tandem schedules comprising components of different schedule types resulted in food presentation. In Experiment 1, either a tandem variable-ratio variable-interval or a tandem variable-interval variable-ratio schedule was used. The variable-interval requirement in the tandem variable-ratio variable-interval schedule was yoked to the time taken to complete the variable-ratio component in the tandem variable-interval variable-ratio schedule, and the length of the variable-interval component in the latter schedule was yoked to the variable-ratio component in the former schedule. If a brief stimulus occurred following completion of the first component, then behavior was differentiated in the two components; subjects responded more quickly in the variable-ratio than in the variable-interval component. If the stimulus was removed, then response rate was determined by the nature of the final component. Similar results were obtained in Experiments 2 and 3 with the use of a three-component tandem variable-ratio variable-interval variable-ratio schedule or tandem variable-interval variable-ratio variable-interval schedule. Thus, a brief stimulus that was not explicitly paired with reinforcement engendered behavior typical of the component schedule preceding its presentation.

RESPONSE‐RATE DIFFERENCES IN VARIABLE‐INTERVAL AND VARIABLE‐RATIO SCHEDULES: AN OLD PROBLEM REVISITED

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate-reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.

Differential Outcome Effect Does Not Control Performance in a Two-Choice Discrimination Task with Visual Stimuli

Three experiments examined the extent to which a differential outcome effect controlled performance in a two-choice instrumental discrimination. In Experiment 1, hungry rats responded on a concurrent variable interval (VI) 60-s extinction (ext) schedule; food reinforcement was presented for responses to the VI 60-s component after a 5-s delay. When a 2-s localized visual stimulus (illumination of the response lever) was presented after the to-be-reinforced response on the VI lever, improved discrimination performance (i.e., more responding to the VI than the ext lever) compared to an unsignaled control condition was noted. In Experiment 2, the same visual stimulus presented after responses to the extinction lever retarded emergence of the discrimination compared to an unsignaled condition (Phase 1). A similar retardation of responding to the correct lever was noted in Phase 2 when the stimulus was a 2-s overhead light flash. In Experiment 3, rats with a 2-s response-contingent overhead light flash associated with the VI component learned to discriminate the VI from the ext component more readily than did a group of rats with no stimulus. However, a group of rats with the overhead light presented after incorrect responses to the ext component demonstrated retarded discrimination learning. These results suggest that a differential outcome effect does not control performance in a delayed reward discrimination task when visual stimuli are employed.

PERFORMANCES ON RATIO AND INTERVAL SCHEDULES OF REINFORCEMENT: DATA AND THEORY

TWO DIFFERENCES BETWEEN RATIO AND INTERVAL PERFORMANCE ARE WELL KNOWN: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio "strain"). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby "yoking" or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.

Failure to find positive key-press contrast for milk reinforcers using a within-session procedure

Rats pressed keys for sweetened condensed milk delivered by multiple schedules. Component duration varied from five seconds to 16 minutes. Positive contrast did not occur for any duration. That is, the rate of responding emitted during a variable interval component of a multiple schedule was not greater when the other component was extinction than when it was an identical variable interval schedule. The function that related key pressing to component duration also differed from that reported by past studies of pigeons pecking keys. Finding such functional differences suggests that behavioral contrast may be mediated by different mechanisms for different responses. It also implies that neither differences in the rates of responding, nor differences in the parts of the body utilized by the response, produced past functional differences in the observation of contrast for different responses.

Positive behavioral contrast as a function of time-out duration when pigeons peck keys on a within-session procedure

Three pigeons pecked keys for food reinforcers delivered by multiple variable interval variable interval schedules in the first part of each session (baseline) and by multiple variable interval extinction schedules in the second part of each session (contrast). The variable interval schedules delivered reinforcers after an average of 4 min or 30 sec in different conditions. The duration of a time-out between the components varied in five steps from 5 to 120 sec. Positive contrast occurred for all time-out durations in both experiments. That is, the rate of responding emitted during the constant, variable interval component was greater during the contrast than during the baseline schedules. The size of contrast did not change systematically with changes in timeout duration. These results violate most theories of contrast. They are compatible with the idea that animals integrate reinforcers over intervals longer than 2 min.

CONDITIONED REINFORCEMENT VERSUS TIME TO REINFORCEMENT IN CHAIN SCHEDULES

Pigeons were trained on three-component chain schedules in which the initial component was either a fixed-interval or variable-interval schedule. The middle and terminal components were varied among fixed-interval fixed-interval, variable-interval variable-interval, and an interdependent variable-interval variable-interval schedule in which the sum of the durations of the two variable-interval components was always equal to the sum of the fixed-interval fixed-interval components. At issue was whether the response rate in the initial component was controlled by its time to primary reinforcement or by the temporal parameters of the stimulus correlated with the middle terminal link. The fixed-interval initial-link schedule maintained much lower response rates than the variable-interval initial-link schedule regardless of the schedules in the middle and terminal links. Nevertheless, the intervening schedules played some role: With fixed-interval schedules in the initial links, response rates were consistently highest with independent variable-interval schedules in the middle and terminal links and intermediate with the interdependent variable-interval schedules; these initial-link differences were predicted by the response rates in the middle link of the chain. With variable-interval schedules in the initial links, response rates were lowest with the fixed-interval fixed-interval schedules following the initial link and were not systematically different for the two types of variable-interval variable-interval schedules. The results suggest that time to reinforcement itself accounts for little if any variance in initial-link responding.

ON CONDITIONED REINFORCING EFFECTS OF NEGATIVE DISCRIMINATIVE STIMULI

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S-) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S- when extinction was in effect on the food key. Observing responses that produced only S- were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S- alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.

Failure to obtain behavioral contrast when components change after reinforcement

The rate of pecking under the variable-interval component of a multiple variable-interval extinction schedule was investigated as a function of whether the components were changed after reinforcement or were changed on the basis of elapsed time. Behavioral contrast—an increase in the response rate during the unchanged variable-interval component—was observed when the components were changed on the basis of time, but not when the components were changed after reinforcement. These results support the hypothesis that behavioral contrast is primarily a result of selective reinforcement of short interresponse times.

IS THERE A DECISIVE TEST BETWEEN MATCHING AND MAXIMIZING?

Reinforcers under typical concurrent variable-interval, variable-ratio schedules may be (a) earned and obtained during the variable-interval component, (b) earned and obtained during the variable-ratio component, or (c) earned during the variable-ratio component and obtained during the variable-interval component. Categories a and b, which have no bearing on matching versus maximizing accounts of choice, were set at zero. The rate of Category c reinforcers and the duration of a changeover delay were varied. Simple matching, which predicts exclusive choice of the variable-interval component, and strict maximizing of overall reinforcement rate, which predicts a bias towards the variable-ratio component, were both disconfirmed: Subjects spent approximately 25% of their time in the variable-ratio component, contrary to the matching prediction, but earned only about one third of the reinforcers predicted by strict maximizing. However, maximizing describes the findings functionally in terms of discounting of delayed reinforcers; matching may describe the data in terms of a restructuring of the alternatives. Matching and maximizing are not competing theories about the fundamental nature of choice, but compatible points of view that may reveal environmental function and behavioral structure.

Effects of lesions of the dorsal noradrenergic bundle on successive discrimination in the rat

Male Sprague-Dawley rats were trained to bar-press for food reward on a successive discrimination involving periods of reward on a variable-interval (VI) 18-s schedule interspersed with periods of extinction. The two components of the schedule were signaled by a steady or a flashing light, counterbalanced between VI and extinction components. Confirming previous findings, the discrimination was easier when the flashing light signaled VI and the steady light signaled extinction, than with the reverse allocation of stimuli. This pattern of results is consistent with a dynamogenic effect of flashing light relative to steady light, facilitating discrimination when the flashing light signals the occasion to respond but impairing discrimination when this stimulus signals the occasion to withhold responding. Given this interpretation of performance in the successive discrimination task, it may be used to test three different hypotheses of the functions of the dorsal noradrenergic bundle (DB): that this subserves learning, selective attention, or behavioral inhibition plus arousal. To examine these hypotheses sham-operated animals were compared to animals in which hippocampal noradrenaline levels had been reduced by 98% and hypothalamic levels by 48% after injection into the DB of the catecholamine-specific neurotoxin, 6-hydroxy-dopamine. The lesioned animals responded more slowly than controls in VI components when these were signaled by the flashing light, and more rapidly than controls in extinction components when these were signaled by the steady light. In consequence, the discrimination was impaired only in the condition (flashing light signaling VI, steady light signaling extinction) which controls found easier. These results are in conflict with predictions from the learning and attentional hypotheses of DB function. They are consistent, however, with a model that attributes behavioral inhibitory functions to the DB projection to the septohippocampal system, and arousing functions to the DB projection to the hypothalamus.

CS duration and reinforcement schedule effects on conditioned enhancement and positive conditioned suppression

Two groups of rats learned a discrimination in which responses were reinforced on a variableinterval (VI) schedule during one stimulus and extinguished during another. Eight conditioned stimuli (CSs), each followed by a food unconditioned stimulus (US), were presented during each session: four during the VI component and four during the extinction component. CS duration was 2 min for Group L and 30 sec for Group S. Subjects in both groups responded at higher rates during CSs presented in the extinction component. Group L subjects also responded at higher rates during VI-component CSs, but Group S subjects responded at lower rates during VI-component CSs. These data indicate that reinforcement schedules may affect rate changes during CSs only within a limited range of CS durations.

BEHAVIORAL CONTRAST IN COMPETITIVE AND NONCOMPETITIVE ENVIRONMENTS

Three experiments examined the effects of opportunities for an alternative response (drinking) on positive behavioral contrast of rats' food-reinforced bar pressing. In both Experiments 1 and 2 the baseline multiple variable-interval schedules were rich (variable interval 10-s), and contrast was examined both with and without a water bottle present. In Experiment 1, the rats were not water deprived. When one component of the multiple schedule was changed to extinction, the rate of bar pressing increased in the constant component (positive behavioral contrast). The magnitude of contrast was larger when the bottle was absent than when it was present, as predicted by the matching law. Drinking did not shift from the constant variable-interval component to the extinction component, as might have been expected from competition theory. In Experiment 2, the rats were water deprived. Contrast was larger when the bottle was present than when it was absent, and drinking did shift to the extinction component, as predicted by competition theory. In Experiment 3, water-deprived rats responded on leaner multiple variable-interval schedules (60-s) in the presence of a water bottle. When one component was changed to extinction, contrast did not occur, and drinking did not shift to the extinction component. The present results suggest that there are at least two different sources of behavioral contrast: "competitive" contrast, observed when an alternative response occurs with high probability, and "noncompetitive" contrast, observed when an alternative response occurs with low probability. The results, in conjunction with earlier studies, also suggest that the form of the alternative response and the rate of food reinforcement provided by the multiple schedule combine to determine the amount of contrast.

The Conditioned Reinforcing Strength of Redundant Stimuli

In a test of the uncertainty reduction and conditioned reinforcement hypotheses, the effects of three procedures on right keypecking (observing) behavior of pigeons were compared in Experiment 1. All three procedures incorporated a variable interval (VI) 15-s, extinction (EXT) 30-s schedule of reinforcement on a center, food, key. In one procedure, Baseline Observing, the center and right keys were illuminated with white light until a right keypeck occurred which changed the color of both keys to either red (during the variable interval component) or green (during the extinction component) for 3 s. Components alternated randomly after reinforcements. In another procedure, Observing: Alternating Components, an identical procedure was employed except that components alternated after reinforcements’. In the remaining procedure, Redundant S+ and S−: Correlated, the center key was red during the variable interval component or green during the extinction component. The right key was white until a right keypeck occurred, whereupon the white color was temporarily replaced (3 s) by the color present on the center key. All pigeons maintained high probabilities (near 1) of right keypecking during the Baseline Observing procedure, low probabilities (range = .0 to .36) of right keypecking during the Redundant S+ and S−: Correlated procedure, and intermediate probabilities of right keypecking during the Observing: Alternating Components procedure. In Experiment 2, a test of the delay-reduction hypothesis under the Redundant S+ and S−: Correlated procedure (from Experiment 1) was performed. In this case, a right keypeck produced the “redundant” stimulus change only during the last 5 s of any component; at other times the right key remained white. The results of both experiments support the view that, of several copresent stimuli, the stimulus that is both nonredundant and correlated with positive reinforcement is the stronger conditioned reinforcer.

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.

Modification of Component Preference on a Multiple Variable Interval Variable Time Schedule: Implications for Chronic Welfare Dependency

During choice testing, 20 rats obtained food pellets by leverpressing on a variable interval (VI) or obtained them freely on a variable time (VT) schedule. Alternation of schedule components occurred when a rat emitted a changeover response on one of the two response levers in the chamber. Rats which had 20 days access to the changeover lever before testing (Group C-C-C) showed a very strong preference for the VT component during the 10-day choice period. Rats which received 20 days of VI training before access to the changeover lever showed a strong preference for the VI component. Rats which received primarily free food for 10 days, were then given 10 days of VI training only, showed a preference for the VT component but this preference was significantly less than that of Group C-C-C.