Interreinforcement Interval Research Articles

Psychological distance to reward: Effects of S+ duration and the delay reduction it signals

A concurrent-chains procedure was used to examine choice between segmented (two-component chained schedules) and unsegmented schedules (simple schedules) in terminal links with equal inter-reinforcement intervals. Previous studies using this kind of experimental procedure showed preference for unsegmented schedules for both pigeons and humans. In this study, two changes in the experimental procedure were made relative to the usual experimental procedure reported in the literature. The first was that in the segmented schedule the second terminal-link stimulus appeared close to reinforcement presentation and the second was that the change in stimulus was brief, allowing the first stimulus to reappear and to be contiguous with reinforcement. With these changes, preference occurred for the segmented schedule. This result is consistent with principles of conditioned reinforcement, including delay-reduction theory.

Efecto de tiempo relativo en demoras de reforzamiento señaladas y no señaladas

One purpose of this study was to determine if by correlating each component of a multiple schedule with delays-of-reinforcement of either 0, 2, 4, or 8 s, a within-session delay-of-reinforcement gradient could be obtained. Reinforcement was delivered on each component using a tandem schedule composed by a random interval that added to each nominal delay yielded a constant interreinforcement interval (IRI) of either 32 or 128 s. Three pigeons were assigned to each IRI. Response rates in each component of the multiple schedules decreased gradually as the reinforcement delay was lengthened. A second purpose was to determine the point on a continuum defined by the probability of signaling the delay period p(signal) at which frequency-of-reinforcement effects become relative-time effects in a reinforcement-delay procedure. The p(signal) was either 0.00, 0.33, 0,66 or 1.00. When p(signal) was 0.00, global response rates for any given delay were higher with the 32 s than with the 128 s IRI, showing a frequency-of-reinforcement effect. When p(signal) was increased from 0.33 to 1.00 global response rates for any given delay were higher with the 128 s than the 32 s IRI, showing a relative-time effect.

Efectos de diferentes probabilidades de reforzamiento sobre la precisión de palomas en tareas de igualación a la muestra: alargando la duración del intervalo entre reforzadores

Using a matching to sample task pigeons were exposed to different durations of interreinforcer interval (19, 38, 76 and 152 s) according to different probability of reinforcement values (1.0, 0.5, 0.25, 0.12). The speed of acquisition was quicker and the discrimination index was higher for the pigeons exposed to a longer interreinforcer interval. It is identified to the interreinforcer interval like a more general variable that controls the accuracy levels and the acquisition speed of conditional discriminations.

Variables temporales como determinantes de la pausa posreforzamiento

Steady-state performance under both, Fixed-ratio and Fixed-interval reinforcement schedules generate a similar response pattern: a period with few or no responses, called postreinforcement pause (PRP), proportional to the interreinforcement interval (IRI). Because of this relationship, the IRI was considered as the determinant of the PRP, but the possibility exists that work time or delay to reinforcer are also involved. However, these variables are usually correlated, thus difficulting the evaluation of their relative contribution on the determination of the pause. To partially solve this problem, in the present experiment the contribution of the IRI and the reinforcer delay on the size of the pause was evaluated with a yoked procedure using Chained FR1-FTx and Conjunctive FR1-FTx reinforcement schedules. Results indicated that both IRI and delay to reinforcer were good predictors of the PRP value in steady state, and that the PRP in the Conjunctive schedule was temporally determined.

TOLERANCE TO EFFECTS OF COCAINE ON BEHAVIOR UNDER A RESPONSE‐INITIATED FIXED‐INTERVAL SCHEDULE

Tolerance to effects of cocaine can be modulated by schedules of reinforcement. With multiple ratio schedules, research has shown an inverse relationship between ratio requirement and amount of tolerance that resulted from daily administration of the drug. In contrast, tolerance to the effects of cocaine on behavior under multiple interval schedules generally has developed regardless of interval value. Under interval schedules reinforcement depends on the animal making one response following a time interval. Thus, as time to respond increases, the time to reinforcement decreases. On the other hand, fixed ratio schedules require a specified number of responses to be made prior to reinforcement. Therefore, delaying the initiation of responding does not coincide with a significant decrease in the time to reinforcement. In the current experiment, 6 pigeons were trained to respond under a three-component multiple schedule, with a different tandem fixed-ratio 1 fixed-interval schedule in each component. The multiple schedule required one response, which was followed by one of three fixed-interval values (5, 15, or 60 s). Thus, the multiple schedule was interval-like because after the fixed-ratio 1, only one more response was required for reinforcement, but it was also ratio-like because the length of the pause at the beginning of each interreinforcer interval affected the time until the next reinforcer. Acute administration of cocaine generally resulted in dose-dependent decreases in responding. Chronic (i.e., daily) administration of a rate-decreasing dose resulted in tolerance patterns similar to those usually obtained with multiple ratio schedules. That is, the magnitude of tolerance was related inversely to schedule size. These results suggest that delay to reinforcement from the initial response may play a role in the development of schedule-parameter-related tolerance.

Choice and the Initial Delay to a Reinforcer

Pigeons were trained in two experiments that used the concurrent-chains procedure. These experiments sought to identify the variables controlling the preference of pigeons for a constant duration over a variable duration of exposure to an aperiodic, time-based, terminal-link schedule. The results indicated that two variables correlated with the constant-duration terminal link combined to control preference: (a) a shorter initial delay to a reinforcer; and (b) the probabilistic occurrence of multiple reinforcers. Grace and Nevin (2000) trained pigeons on a concurrent-chains procedure with equal variable-interval (VI) schedules in the initial links and equal VI schedules in the terminal links. The terminal links differed in that one ended after a single reinforcer, which they called “variable-duration” terminal link, whereas the other ended after a fixed period of exposure equal to the average interreinforcement interval (IRI) of the schedule, which they called “constantduration” terminal link. As Grace and Nevin identified, and as discussed at some length below, an important feature of the constant-duration terminal link is that it probabilistically yielded 0, 1, or multiple reinforcers per entry, although it provided the same average rate of reinforcement overall as the variable-duration terminal link. Grace and Nevin (2000) found that three of four pigeons clearly preferred the constant-duration terminal link. In their words, the data of a fourth pigeon “demonstrated a consistent right-key bias” (p. 178), and the present conclusion is that its data are more difficult to interpret. In any case, an important question is what variables caused the preference. Ordinarily, one would have expected the pigeons to be indifferent, since the schedules in effect during the alternatives were identical, and each alternative yielded the same overall rate of reinforcement. Grace and Nevin (2000) initially pondered the role of multiple reinforcers in the constant-duration terminal link, because research has shown that subjects may well prefer a choice alternative associated with multiple reinforcers rather than a single reinforcer per terminal-link entry (e.g., Fantino & Herrnstein, 1968; Mazur, 1986; McDiarmid & Rilling, 1965; Moore, 1979; Poniewaz, 1984; Shull, Mellon, & Sharp, 1990; Shull, Spear, & Bryson, 1981). In particular, Grace and Nevin discussed their findings from the view Correspondence concerning this article may be addressed to Dr. J. Moore, Dept. of Psychology,

The matching law and effects of reinforcer rate and magnitude on choice in transition

Four pigeons responded in a concurrent-schedule procedure in which reinforcer rates and magnitudes changed unpredictably across sessions according to independent random series. Programmed relative reinforcement rates and magnitudes were always either 2:1 or 1:2. Pigeons' response allocation tended to stabilize within sessions and multiple regression analyses showed that it was determined by rates and magnitudes from the current session. Sensitivity coefficients were positive and statistically significant for current-session reinforcement and magnitude ratios. Although there were individual differences in sensitivity to rate and magnitude, their interaction was not significant across subjects. Rate and magnitude both controlled responding in single sessions and individual interreinforcer intervals. Analyses of responding within sessions showed that preference was more extreme when the richer rate and larger magnitude were associated with the same alternative than when they were associated with different alternatives. Overall, results support the concatenated generalized matching law's assumptions of additivity and independence as applied to choice in transition.

Schedule discrimination in a mixed schedule: Implications for models of the variable-ratio, variable-interval rate difference

In Experiment 1, each of three humans knowledgeable about operant schedules used mouse clicks to respond to a "work key" presented on a monitor. On a random half of the presentations, work-key responses that completed a variable ratio (VR) 12 produced a tone. After five tones, the work key was replaced by two report keys. Pressing the right or left report key, respectively, added or subtracted yen50 from a counter and produced the work key. On the other half of the presentations, a variable interval (VI) associated with the work key was defined so its interreinforcer intervals approximated the time it took to complete the variable ratio. After five tone-producing completions of this schedule, the report keys were presented. Left or right report-key presses, respectively, added or subtracted yen50 from the counter. Subjects achieved high yen totals. In Experiment 2, the procedure was changed by requiring an interresponse time after completion of the variable interval that approximated the duration of the reinforced interresponse time on the variable ratio. Prior to beginning, subjects were shown how a sequence of response bouts and pauses could be used to predict schedule type. Subjects again achieved high levels of accuracy. These results show humans can discriminate ratio from interval schedules even when those schedules provide the same rate of reinforcement and reinforced interresponse times.

Choice Behavior in Pigeons Maintained With Probabilistic Schedules of Reinforcement

Pigeons were trained in three experiments with a two-key, concurrent-chains choice procedure. The initial links were equal variable-interval schedules, and the terminal links were randomtime schedules with equal average interreinforcement intervals. Across the three experiments, the pigeons either stayed in a terminal link until a reinforcer was delivered or were returned to the initial links if a reinforcer was not delivered after the first unit time interval. The results showed that choice responding in concurrent chains is a complex function of (a) local factors, such as the immediacy of reinforcement in a terminal link, and (b) overall factors, such as the overall interreinforcement interval as calculated over both initial and terminal links of the chain. However, the local factors are generally stronger when the two factors are directly opposed.

RESPONSE INDUCTION DURING THE ACQUISITION AND MAINTENANCE OF LEVER PRESSING WITH DELAYED REINFORCEMENT

The acquisition of lever pressing by rats and the occurrence of unreinforced presses at a location different from that of the reinforced response were studied using different delays of reinforcement. An experimental chamber containing seven identical adjoining levers was used. Only presses on the central (operative) lever produced food pellets. Groups of 3 rats were exposed to one of seven different tandem random-interval (RI) fixed-time (FT) schedules. The average RI duration was the complement of the FT duration such that their sum yielded a nominal 32-s interreinforcement interval on average. Response rate on the operative lever decreased as the FT value was lengthened. The spatial distribution of responses on the seven levers converged on the operative lever when the FT was 0 or 2 s and spread across the seven levers as the FT value was lengthened to 16 or 32 s. Presses on the seven levers were infrequent during the FT schedule. Both operative- and inoperative-lever pressing intertwined in repetitive patterns that were consistent within subjects but differed between subjects. These findings suggest that reinforcer delay determined the response-induction gradient.

Schedule‐induced electrodermal responding in children

The current studies were designed to determine whether different intermittent schedules of reinforcement would have a differential effect on the skin conductance responses (SCRs) of children in the 8-12-year-old age range. Results of two experiments indicated that the amplitude of children's SCRs immediately following the occurrence of reinforcement were a function of the schedule of reinforcement, with larger SCRs associated with longer inter-reinforcement intervals. The findings are consistent with the research literature suggesting that long inter-reinforcement intervals can be evocative of aversively motivated emotions, as well as with research on the effects of interstimulus intervals on SCRs. Possible implications for these findings are discussed.

RESISTANCE TO EXTINCTION FOLLOWING VARIABLE‐INTERVAL REINFORCEMENT: REINFORCER RATE AND AMOUNT

Rats obtained food-pellet reinforcers by nose poking a lighted key. Experiment 1 examined resistance to extinction following single-schedule training with different variable-interval schedules, ranging from a mean interval of 16 min to 0.25 min. That is, for each schedule, the rats received 20 consecutive daily baseline sessions and then a session of extinction (i.e., no reinforcers). Resistance to extinction (decline in response rate relative to baseline) was negatively related to the rate of reinforcers obtained during baseline, a relation analogous to the partial-reinforcement-extinction effect. A positive relation between these variables emerged, however, when the unit of extinction was taken as the mean interreinforcer interval that had been in effect during training (i.e., as an omitted reinforcer during extinction). In a second experiment, rats received blocks of training sessions, all with the same variable-interval schedule but with a reinforcer of four pellets for some blocks and one pellet for others. Resistance to extinction was greater following training with the larger (four pellets) than with the smaller (one pellet) reinforcer. Taken together, these results support the principle that greater reinforcement during training (e.g., higher rate or larger amount) engenders greater resistance to extinction even when the different conditions of reinforcement are varied between blocks of sessions.

EFFECTS OF REINFORCEMENT HISTORY ON RESPONSE RATE AND RESPONSE PATTERN IN PERIODIC REINFORCEMENT

Several researchers have suggested that conditioning history may have long-term effects on fixed-interval performances of rats. To test this idea and to identify possible factors involved in temporal control development, groups of rats initially were exposed to different reinforcement schedules: continuous, fixed-time, and random-interval. Afterwards, half of the rats in each group were studied on a fixed-interval 30-s schedule of reinforcement and the other half on a fixed-interval 90-s schedule of reinforcement. No evidence of long-term effects attributable to conditioning history on either response output or response patterning was found; history effects were transitory. Different tendencies in trajectory across sessions were observed for measures of early and late responding within the interreinforcer interval, suggesting that temporal control is the result of two separate processes: one involved in response output and the other in time allocation of responding and not responding.

Variation, repetition, and choice.

Experiment 1 investigated the controlling properties of variability contingencies on choice between repeated and variable responding. Pigeons were exposed to concurrent-chains schedules with two alternatives. In the REPEAT alternative, reinforcers in the terminal link depended on a single sequence of four responses. In the VARY alternative, a response sequence in the terminal link was reinforced only if it differed from the n previous sequences (lag criterion). The REPEAT contingency generated low, constant levels of sequence variation whereas the VARY contingency produced levels of sequence variation that increased with the lag criterion. Preference for the REPEAT alternative tended to increase directly with the degree of variation required for reinforcement. Experiment 2 examined the potential confounding effects in Experiment 1 of immediacy of reinforcement by yoking the interreinforcer intervals in the REPEAT alternative to those in the VARY alternative. Again, preference for REPEAT was a function of the lag criterion. Choice between varying and repeating behavior is discussed with respect to obtained behavioral variability, probability of reinforcement, delay of reinforcement, and switching within a sequence.

EFFECTS OF COCAINE ON PERFORMANCE UNDER FIXED‐INTERVAL SCHEDULES WITH A SMALL TANDEM RATIO REQUIREMENT

Daily administration of cocaine often results in the development of tolerance to its effects on responding maintained by fixed-ratio schedules. Such effects have been observed to be greater when the ratio value is small, whereas less or no tolerance has been observed at large ratio values. Similar schedule-parameter-dependent tolerance, however, has not been observed with fixed-interval schedules arranging comparable interreinforcement intervals. This experiment examined the possibility that differences in rate and temporal patterning between the two types of schedule are responsible for the differences in observed patterns of tolerance. Five pigeons were trained to key peck on a three-component multiple (tandem fixed-interval fixed-ratio) schedule. The interval values were 10, 30, and 120 s; the tandem ratio was held constant at five responses. Performance appeared more like that observed under fixed-ratio schedules than fixed-interval schedules. Effects of various doses of cocaine given weekly were then determined for each pigeon. A dose that reduced responding was administered prior to each session for 50 days. A reassessment of effects of the range of doses revealed tolerance. The degree of tolerance was similar across components of the multiple schedule. Next, the saline vehicle was administered prior to each session for 50 days to assess the persistence of tolerance. Tolerance diminished in all subjects. Overall, the results suggested that schedule-parameter-dependent tolerance does not depend on the temporal pattern of responding engendered by fixed-ratio schedules.

The effect of rate of reinforcement and time in session on preference for variability.

Pigeons pecked keys on concurrent-chains schedules that provided a variable interval 30-sec schedule in the initial link. One terminal link provided reinforcers in a fixed manner; the other provided reinforcers in a variable manner with the same arithmetic mean as the fixed alternative. In Experiment 1, the terminal links provided fixed and variable interval schedules. In Experiment 2, the terminal links provided reinforcers after a fixed or a variable delay following the response that produced them. In Experiment 3, the terminal links provided reinforcers that were fixed or variable in size. Rate of reinforcement was varied by changing the scheduled interreinforcer interval in the terminal link from 5 to 225 sec. The subjects usually preferred the variable option in Experiments 1 and 2 but differed in preference in Experiment 3. The preference for variability was usually stronger for lower (longer terminal links) than for higher (shorter terminal links) rates of reinforcement. Preference did not change systematically with time in the session. Some aspects of these results are inconsistent with explanations for the preference for variability in terms of scaling factors, scalar expectancy theory, risk-sensitive models of optimal foraging theory, and habituation to the reinforcer. Initial-link response rates also changed within sessions when the schedules provided high, but not low, rates of reinforcement. Within-session changes in responding were similar for the two initial links. These similarities imply that habituation to the reinforcer is represented differently in theories of choice than are other variables related to reinforcement.

Concurrent schedules: reinforcer magnitude effects.

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key procedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each condition remained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed. At a molar level of analysis, preference was described well by a version of the generalized matching law, consistent with previous reports. More local analyses showed that recently obtained reinforcers had small measurable effects on current preference, with the most recently obtained reinforcer having a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses, and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choice have both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitude is exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcer magnitude is varied are likely to be due to equal frequencies of different sized preference pulses, whereas higher sensitivities when reinforcer rates are varied might result from changes in the frequencies of different sized preference pulses.

Effects of differences in interreinforcer intervals between past and current schedules on fixed-interval responding.

Undergraduates were exposed to a mixed fixed-ratio differential-reinforcement-of-low-rate schedule. Values of the schedule components were adjusted so that interreinforcer intervals in one component were longer than those in another component. Following this, a mixed fixed-interval 5-s fixed-interval 20-s schedule (Experiment 1) or six fixed-interval schedules in which the values ranged from 5 to 40 s (Experiment 2) were in effect. In both experiments, response rates under the fixed-interval schedules were higher when the interreinforcer intervals approximated those produced under the fixed-ratio schedule, whereas the rates were lower when the interreinforcer intervals approximated those produced under the different-reinforcement-of-low-rate schedule. The present results demonstrate that the effects of behavioral history were under control of the interreinforcer intervals as discriminative stimuli.

A tuned-trace theory of interval-timing dynamics.

Animals on interval schedules of reinforcement can rapidly adjust a temporal dependent variable, such as wait time, to changes in the prevailing interreinforcement interval. We describe data on the effects of impulse, step, sine-cyclic, and variable-interval schedules and show that they can be explained by a tuned-trace timing model with a one-back threshold-setting rule. The model can also explain steady-state timing properties such as proportional and Weber law timing and the effects of reinforcement magnitude. The model assumes that food reinforcers and other time markers have a decaying effect (trace) with properties that can be derived from the rate-sensitive property of habituation (the multiple-time-scale model). In timing experiments, response threshold is determined by the trace value at the time of the most recent reinforcement. The model provides a partial account for the learning of multiple intervals, but does not account for scalloping and other postpause features of responding on interval schedules and has some problems with square-wave schedules.

Rapid timing of transitions in inter-reinforcement interval duration in infants

Infants trained on FI schedules are able to adjust their post-reinforcement pause, but many sessions at a given parameter value are necessary before the FI cumulative record appears stable. We analysed transitions from one FI value to another from our earlier experiments. Infants were exposed to FI schedules from 10 to 80 s. Under these schedules, they showed long post-reinforcement pauses with most intervals involving the emission of a single, reinforced response. The passage from one FI value to another was unpredictable for infants. We observed rapid timing effects during transitions (passage from one FI value to another, from FI 10 s to FI 20 s, etc.). Wait times increased during the first session of the new FI value, and accurate adjustments of wait times quickly appeared. Dynamic effects observed were similar to those obtained from animals in experiments on rapid-timing effects.