Interreinforcement Times Research Articles

The effect of periodicity of reinforcement on food accumulation

Recent studies on food accumulation have shown that the rate of food-procuring responses increases by delaying the delivery of food (i.e., an increasing delay gradient). However, these studies have only delivered the food periodically, which could lead to the establishment of a behavior sequence between successive food deliveries. Therefore, the increasing delay gradient could be due to the establishment of different behavior sequences, composed of more food-procuring responses as the delay of reinforcement is lengthened. The present study investigated whether the increasing delay gradient occurs when the establishment of a behavior sequence is hindered by eliminating the periodicity of reinforcement from a situation of food accumulation. The periodicity of reinforcement was eliminated by using a variable delay in Experiment 1 and a variable inter-trial interval in Experiment 2. It was found that the increasing delay gradient occurred both with periodic and aperiodic reinforcement, which may imply that it does not depend on the establishment of a behavior sequence. In contrast, the increasing delay gradient was attributed to the temporal distribution of the food-procuring period within the inter-reinforcement time.

RESPONSE‐RATE DIFFERENCES IN VARIABLE‐INTERVAL AND VARIABLE‐RATIO SCHEDULES: AN OLD PROBLEM REVISITED

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate-reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.

INTERREINFORCEMENT TIME, WORK TIME, AND THE POSTREINFORCEMENT PAUSE

Six rats were trained with food deliveries contingent upon their pressing a lever and holding it down for fixed, cumulative durations. Hold requirements were varied from 7.5 seconds to 120 seconds. Lever holding was maintained reliably at hold requirements as long as 30 seconds to 105 seconds for different rats. At longer hold requirements, lever holding was erratic and tended to occur only early in sessions. At shorter and intermediate requirements, the patterns of lever holding resembled those of responding under fixed-ratio schedules for discrete responses, with breaks in responding immediately after reinforcement alternating with relatively continuous lever holding until the next reinforcement. At longer hold requirements, postpause lever holding frequently was interrupted with additional pauses. The duration of postreinforcement pauses increased linearly with the scheduled hold requirement. However, for five of six rats, the hold requirement, which represents the actual time spent lever holding per reinforcer, accounted for somewhat less variance in pause duration than did interreinforcement time.

A COMPARISON OF RATIO AND INTERVAL REINFORCEMENT SCHEDULES WITH COMPARABLE INTERREINFORCEMENT TIMES

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Matching and maximizing in concurrent schedules.

In a recent article, Staddon and Motheral derived a mathematical model of responding in concurrent variable-interval-variable-interval schedules. According to this model, maximization of overall reinforcement rate predicts the operant matching law. However, Staddon and Motheral's derivation ignores a fundamental aspect of the concurrent schedule contingency. There are two simultaneously available schedules. A reinforcer, therefore, can occur in two ways, (a) following two consecutive responses on the same schedule, and (b) following a switch between the two schedules. Staddon and Motheral's model does not distinguish between these two possibilities, and this omission leads to an incorrect estimate of the expected reinforcement rate. In addition, the exact density function for obtained interreinforcement times in a concurrent interval schedule shows that matching and maximizing are different.

Response suppression on DRL by rats with septal damage.

The effectiveness of the differential reinforcement for low rates of responding (DRL) contingency in suppressing response rates of septal rats was investigated by using a Multi-DRL-yoked-VI (variable interval) schedule of reinforcement. The yoking procedure equated the interreinforcement times on the two schedules. Each schedule was in effect for half of each session, and the change in schedule was signaled by the presence or absence of a cue light. Schedule order and DRL delay requirement were varied. For both normal and septal rats, the response rates were higher in the VI component than the DRL component; this effect demonstrates that the responding of septals as well as normals is suppressed by the differential reinforcement of a particular class of IRTs. A sharp difference in the level of responding occurred at the point of transition from one component of the multiple schedule to the other, which provides evidence of a discrimination between the two schedules for both normals and septals. The conclusion is that the responding of septals is suppressed by the DRL contingency and not controlled solely by the density and distribution of reinforcement.

YOKED VARIABLE‐RATIO AND VARIABLE‐INTERVAL RESPONDING IN PIGEONS1

Pigeons' key pecks were maintained by variable-ratio or variable-interval schedules of food reinforcement. For pairs of pigeons in one group, variable-ratio reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-interval schedule yoked to the interreinforcement times produced by the first pigeon. For pairs of pigeons in another group, variable-interval reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-ratio schedule yoked to the interreinforcement responses produced by the first pigeon. For each pair, the yoking procedure was maintained for four or five consecutive sessions of 50 reinforcements each. In more than three-quarters of the pairs, variable-ratio response rates were higher than variable-interval rates within two sessions; in all cases, the rate difference developed within four sessions.

The sensitivity of the pigeon’s keypeck to the differential reinforcement of long interresponse times

Keypecking of pigeons was studied under differential-reinforcement-of-low-rate (DRL) and variable-interval (VI) schedules in which the interreinforcement times on the two schedules were equated by a yoking procedure. Each schedule was available for half of every session and a change of schedule was signaled by a change of key color. The value of the DRL schedule was varied from .5 to 300 sec. Response rates were always higher in the VI schedule, but within sessions there was a sharp change in response rate coincident with the change in schedule only under lower schedule values. A group without prior training was tested with a 180-sec schedule value, and it, too, developed a higher response rate during the VI schedule, showing that the effect was not dependent on prior experience under low schedule values. In all conditions except the .5- and 1-sec values of the schedule, the mean proportion of responses emitted during the VI schedule was approximately .85 of the responses emitted during both schedules. The conclusion was that the requirement of a minimum interresponse time for reinforcement may work its effect by determining which responses may occur just prior to the reinforced response and thus receive delayed reinforcement.

CONJUNCTIVE SCHEDULES OF REINFORCEMENT: III. A FIXED‐INTERVAL ADJUSTING FIXED‐RATIO SCHEDULE1

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.

CONJUNCTIVE SCHEDULES OF REINFORCEMENT II: RESPONSE REQUIREMENTS AND STIMULUS EFFECTS1

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.

BEHAVIOR UNDER LARGE VALUES OF THE DIFFERENTIAL‐REINFORCEMENT‐OF‐LOW‐RATE SCHEDULE1

Pigeons pecked a key and rats pressed a lever for food reinforcement under large values of the differential-reinforcement-of-low-rate schedule. Each subject was tested under 10 different schedule values ranging from 1 to 45 min and was exposed to each schedule value at least twice. The mean interresponse time and mean interreinforcement time increased with the schedule value according to power functions. Response-probability functions were computed for schedule values below 20 min and showed an increase in response probability as a function of time since the last response in most cases. Mean responses per reinforcer increased as a function of schedule value for the rats, but decreased as a function of schedule value for the pigeons. The proportion of responses with interresponse times shorter than 1 sec were an increasing function of schedule value for the pigeons, but did not vary as a function of schedule value for the rats.

Classically conditioned limb reflexes reinforced by motor cortex stimulation

This study measures the latencies of, and the effect of the reduction of the interreinforcement time interval upon, conditional limb reflexes reinforced to an auditory conditional stimulus with movement evoking electrical stimulation of the motor-sensory cortex as the unconditional stimulus. Conditioned head movements were obtained in all, while limb movements developed in only 2, of 3 cats following 4.6 min reinforcement intervals. One-min reinforcement intervals did not abolish these limb responses which are here shown to have a latency significantly in excess of 1 sec. The conditional limb responses of this study are identified as a classical, as contrasted with the instrumental variant of other studies. The classical limb response is thought to develop if there is: (a) an absence of head movement induced extensor tonus; (b) a motor-sensory cortex stimulus adequate to evoke limb movement; and, (c) a period of conditional stimulus isolation ample to permit the observation of these long latency responses.

Response Independent Food as an Extinction Procedure for Responding on DRL Schedules

Following extensive training on a schedule which differentially reinforced low rates of responding (DRL), 3 rats were exposed to an extinction procedure under which delivery of food was no longer dependent upon responding. Each S’s schedule of non-response contingent food was based upon distributions of interrein-forcement times previously recorded for that S during conditioning on the Drl schedule. The procedure resulted in nearly immediate reduction of response rates and a shift in interresponse time distributions away from the previously reinforced interval. These data support the notion that a response-reinforcement contingency is vital in maintaining operant behavior and that nonresponse contingent delivery of food is an operationally logical and empirically effective extinction procedure.

ACCURACY OF PERFORMANCE ON A MATCHING‐TO‐SAMPLE PROCEDURE UNDER INTERVAL SCHEDULES1

Correct matches on a matching-to-sample procedure were reinforced under fixed-interval, chained fixed-interval, and fixed-interval schedules with exteroceptive stimulus changes correlated with time since the last reinforcer (an added clock). For all four pigeons, accuracy changed within the fixed-interval and fixed-interval schedules with added clock, decreasing from the beginning of the interval to some point in the middle. The performance then became increasingly more accurate until the end of the interval. Under the chained schedules, accuracy also changed within the components. During the initial component, accuracy decreased from the beginning of the fixed interval to some point in the middle or at the end. During the middle component, the performance usually remained at an intermediate level of accuracy. During the terminal component, the initially inaccurate performance became increasingly more accurate throughout the interval. Systematic relationships between response rate and per cent error showed that all four pigeons performed most accurately at high rates. The accuracy of the performance at low rates was also quite high. These relationships held for all three types of schedules through an eight-fold variation in scheduled interreinforcement time.

SOME EFFECTS OF INTERREINFORCEMENT TIME UPON CHOICE1

Pigeons' responses were reinforced on two identical and concurrently available chain variable-interval-schedules. Unlike the typical concurrent chains procedure, both links were operative throughout, thus producing three types of concurrency: (1) concurrent initial links; (2) concurrent initial and terminal links; (3) concurrent terminal links. Choice proportions in each of these three states suggested that the pigeons were sensitive to momentary likelihoods of reinforcement: choice proportions for a schedule were higher when the schedule had been operative for some time, resulting in a higher probability of reinforcement. The study also showed that the relative rates of responding did not match the relative rates of reinforcement in any of the three states of concurrency. Instead, the choice proportions in both the concurrent initial and in the concurrent terminal links were intermediate between the scheduled and the obtained relative rates of reinforcement, while the choice proportions for a terminal link concurrent with an initial link consistently overmatched the relative interreinforcement times (and were typically 1.00). These data indicate that an accurate characterization of choice may not be obtained by considering only the relative interreinforcement interval where one interreinforcement interval is segmented into a chain. Instead, the organism's choice for a schedule will be substantially lowered by the chaining operation.

STUDIES ON RESPONDING UNDER FIXED‐INTERVAL SCHEDULES OF REINFORCEMENT: THE EFFECTS ON THE PATTERN OF RESPONDING OF CHANGES IN REQUIREMENTS AT REINFORCEMENT1

In pigeons responding under a 180-sec fixed-interval schedule of reinforcement, the frequency distribution of the duration of the final interresponse time before the reinforcer was compared with the distribution of the preceding two interresponse times. The results confirmed qualitatively and quantitatively the expected preferential reinforcement of longer interreinforcement times under fixed-interval reinforcement. Requirements at reinforcement were then changed to eliminate the preferential reinforcement of longer interresponse times. Local patterns and mean rate of responding could change, without the characteristic fixed-interval pattern of increasing responding through the interval (scalloping) being much affected. It is concluded that this characteristic pattern of fixed-interval responding does not depend crucially on effects of the reinforcer at the moment of reinforcement, but rather to effects extending over much longer periods of time than just the last interresponse time.

SEPARATING THE EFFECTS OF INTERREINFORCEMENT TIME AND NUMBER OF INTERREINFORCEMENT RESPONSES1

The relative importance of interreinforcement time and interreinforcement responses was evaluated by varying each independently. To do this, a blackout was presented after each nonreinforced response under both fixed-ratio and fixed-interval schedules of reinforcement. Manipulating the blackout duration under the fixed-ratio schedule caused interreinforcement time to vary without affecting the number of interreinforcement responses. Pigeons' post-reinforcement and post-blackout response latencies were found to increase linearly with interreinforcement time. Under the fixed-interval schedule, the same blackout manipulations changed the number of interreinforcement responses without affecting interreinforcement time. Post-reinforcement and post-blackout response latencies under this condition were approximately constant. These results suggest that responding is controlled by interreinforcement time and is not influenced by the number of responses emitted between reinforcements.

Control of schedule-induced polydipsia: type, size, and spacing of meals.

Rats were given daily 1-min variable-interval sessions for several types of food delivered in various amounts per reinforcement and the concurrent, schedule-induced polydipsia was measured. Dry, solid food was neither a necessary nor sufficient condition for the development of polydipsia. Small portions of liquid Standard Monkey Diet produced polydipsia, but 45-mg dextrose or sucrose pellets did not. Within the range studied, smaller portions of both solid and liquid foods produced more drinking than larger portions per reinforcement. Two-min variable-interval sessions produced a greater polydipsic response than 1-min variable-interval, even though the number of 45-mg Noyes pellets allowed per session was held constant. Polydipsia was greatly attenuated on these schedules when the number of pellets remained constant, but were delivered two at a time. Within the ranges studied, the concurrent polydipsic response was increased by decreasing the rate of food acquisition, either by using smaller portions of food per reinforcement or by increasing the interreinforcement time.

Subject control of reinforcement density on an interval schedule.

Rats were trained on schedules in which the rate of responding in one inter-reinforcement time (ISRT) determined the minimum duration of the following ISRT. The durations of the four ISRTs which were used were related in a simple arithmetic ratio; T, 2T, 3T, and 4T sec. For one group high rates in the current ISRT shortened the following ISRT from the basic value of 4T sec. A high response rate lengthened the next ISRT from the basic value of T sec. in the other group. Ss in each group were able to approximate the response rate which would result in maximum reinforcement density.