Reinforced Interresponse Times Research Articles

Tactical contingencies in the experimental analysis of reinforcement and operant classes

In his "Tactics of Scientific Research" (1960), his work on avoidance, his discovery of equivalence classes and his cautions on applications of coercion, Murray Sidman created high standards for behavior analytic research. I illustrate his influence in the context of three examples he might have characterized as pilot studies. Each examined trial N+1 response probabilities depending on whether trial N responding had produced a reinforcer. Differentially reinforced interresponse times, keys pecked in arbitrary matching, and two-key response sequences provided no robust evidence that reinforcing some response property on trial N raises the probability of responding with that property on trial N+1. These negative findings shed light on the nature of operant classes and on the relation of reinforcers to the responses that produced them. Through selection, reinforcers create operant classes and engender variations of the responses within those classes; operant classes are held together by common contingencies. Sidman extended our understanding of operant classes by expanding them to include equivalence relations.

Temporal regulation of children with autism spectrum disorder exposed to a differential-reinforcement-of low-rates schedule.

This study investigated temporal adjustment of children with autism spectrum disorder under a differential-reinforcement-of-low-rates (DRL) schedule. Sixteen participants, aged 3.2 to 7 years, were exposed to two conditions, DRL 5 s and DRL 20 s. Children participated in 7 sessions in each condition, except for 1 participant who attained the adjustment criteria in the DRL 5-s schedule. Temporal adjustment was measured with the proportion of reinforced interresponse times (IRTs) and the mean IRT. The operant response was a press on a touch screen and the reinforcers were cartoons. IQ and receptive language were measured prior to the DRL sessions. Results showed that the mean proportion of reinforced IRTs was slightly higher in the DRL 5-s schedule. The mean IRT was above the IRT requirement in both conditions. However, substantial individual variability was observed. Children with higher IQ and receptive language scores presented a greater proportion of reinforced IRTs in both conditions. Moreover, participants who adjusted their responses to the DRL 5-s schedule were more likely to adjust responding to the DRL 20-s schedule. This suggests that some children might be more sensitive to reinforcement contingencies than others. This study points at future research in the field of timing in children.

A re-examination of responding on ratio and regulated-probability interval schedules

The higher response rates observed on ratio than on matched interval reward schedules has been attributed to the differential reinforcement of longer inter-response times (IRTs) on the interval contingency. Some data, however, seem to contradict this hypothesis, showing that the difference is still observed when the role of IRT reinforcement is neutralized by using a regulated-probability interval schedule (RPI). Given the mixed evidence for these predictions, we re-examined this hypothesis by training three groups of rats to lever press under ratio, interval and RPI schedules across two phases while matching reward rates within triads. At the end of the first phase, the master ratio and RPI groups responded at similar rates. In the second phase, an interval group yoked to the same master ratio group of the first phase responded at a lower rate than the RPI group. Post-hoc analysis showed comparable reward rates for master and yoked schedules. The experienced response-outcome rate correlations were likewise similar and approached zero as training progressed. We discuss these results in terms of a contemporary dual-system model of instrumental conditioning.

Rats show molar sensitivity to different aspects of random-interval-with-linear-feedback-functions and random-ratio schedules.

Three experiments examined the impact of various aspects of reinforcement contingencies on responding maintained by free-operant schedules by food-deprived rats. Experiment 1 demonstrated that random interval (RI) and random-interval-with-positive-response-reinforcer-feedback (RI+) schedules maintained similar rates of responding at a variety of reinforcer frequencies. Experiment 2 demonstrated that a random ratio (RR) schedule maintained higher rates than RI or RI+ schedules, except at high rates of reinforcement, where response rates were similar on all schedules. Experiment 3 again demonstrated that RR schedules produced higher response rates than either RI or RI+ schedules, but modification of the RI+ schedule to prevent ratio strain enhanced response rates relative to an RI schedule. Together these results reveal a pattern of interacting factors in schedule controlled behavior: at high rates of reinforcement, this factor overrides the impact of other controlling factors, but as reinforcement rate decreases the joint impact of interresponse times reinforcement, response-reinforcer feedback functions, and ratio strain are observed.

Discrimination of variable schedules is controlled by interresponse times proximal to reinforcement.

In Experiment 1, food-deprived rats responded to one of two schedules that were, with equal probability, associated with a sample lever. One schedule was always variable ratio, while the other schedule, depending on the trial within a session, was: (a) a variable-interval schedule; (b) a tandem variable-interval, differential-reinforcement-of-low-rate schedule; or (c) a tandem variable-interval, differential-reinforcement-of-high-rate schedule. Completion of a sample-lever schedule, which took approximately the same time regardless of schedule, presented two comparison levers, one associated with each sample-lever schedule. Pressing the comparison lever associated with the schedule just presented produced food, while pressing the other produced a blackout. Conditional-discrimination accuracy was related to the size of the difference in reinforced interresponse times and those that preceded it (predecessor interresponse times) between the variable-ratio and other comparison schedules. In Experiment 2, control by predecessor interresponse times was accentuated by requiring rats to discriminate between a variable-ratio schedule and a tandem schedule that required emission of a sequence of a long, then a short interresponse time in the tandem's terminal schedule. These discrimination data are compatible with the copyist model from Tanno and Silberberg (2012) in which response rates are determined by the succession of interresponse times between reinforcers weighted so that each interresponse time's role in rate determination diminishes exponentially as a function of its distance from reinforcement.

SINGLE‐SAMPLE DISCRIMINATION OF DIFFERENT SCHEDULES' REINFORCED INTERRESPONSE TIMES

Food-deprived rats in Experiment 1 responded to one of two tandem schedules that were, with equal probability, associated with a sample lever. The tandem schedules' initial links were different random-interval schedules. Their values were adjusted to approximate equality in time to completing each tandem schedule's response requirements. The tandem schedules differed in their terminal links: One reinforced short interresponse times; the other reinforced long ones. Tandem-schedule completion presented two comparison levers, one of which was associated with each tandem schedule. Pressing the lever associated with the sample-lever tandem schedule produced a food pellet. Pressing the other produced a blackout. The difference between terminal-link reinforced interresponse times varied across 10-trial blocks within a session. Conditional-discrimination accuracy increased with the size of the temporal difference between terminal-link reinforced interresponse times. In Experiment 2, one tandem schedule was replaced by a random ratio, while the comparison schedule was either a tandem schedule that only reinforced long interresponse times or a random-interval schedule. Again, conditional-discrimination accuracy increased with the temporal difference between the two schedules' reinforced interresponse times. Most rats mastered the discrimination between random ratio and random interval, showing that the interresponse times reinforced by these schedules can serve to discriminate between these schedules.

ON THE PRIMACY OF MOLECULAR PROCESSES IN DETERMINING RESPONSE RATES UNDER VARIABLE‐RATIO AND VARIABLE‐INTERVAL SCHEDULES

This study focused on variables that may account for response-rate differences under variable-ratio (VR) and variable-interval (VI) schedules of reinforcement. Four rats were exposed to VR, VI, tandem VI differential-reinforcement-of-high-rate, regulated-probability-interval, and negative-feedback schedules of reinforcement that provided the same rate of reinforcement. Response rates were higher under the VR schedule than the VI schedule, and the rates on all other schedules approximated those under the VR schedule. The median reinforced interresponse time (IRT) under the VI schedule was longer than for the other schedules. Thus, differences in reinforced IRTs correlated with differences in response rate, an outcome suggestive of the molecular control of response rate. This conclusion was complemented by the additional finding that the differences in molar reinforcement-feedback functions had little discernible impact on responding.

Schedule discrimination in a mixed schedule: Implications for models of the variable-ratio, variable-interval rate difference

In Experiment 1, each of three humans knowledgeable about operant schedules used mouse clicks to respond to a "work key" presented on a monitor. On a random half of the presentations, work-key responses that completed a variable ratio (VR) 12 produced a tone. After five tones, the work key was replaced by two report keys. Pressing the right or left report key, respectively, added or subtracted yen50 from a counter and produced the work key. On the other half of the presentations, a variable interval (VI) associated with the work key was defined so its interreinforcer intervals approximated the time it took to complete the variable ratio. After five tone-producing completions of this schedule, the report keys were presented. Left or right report-key presses, respectively, added or subtracted yen50 from the counter. Subjects achieved high yen totals. In Experiment 2, the procedure was changed by requiring an interresponse time after completion of the variable interval that approximated the duration of the reinforced interresponse time on the variable ratio. Prior to beginning, subjects were shown how a sequence of response bouts and pauses could be used to predict schedule type. Subjects again achieved high levels of accuracy. These results show humans can discriminate ratio from interval schedules even when those schedules provide the same rate of reinforcement and reinforced interresponse times.

Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement.

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule.

The operant reserve: A computer simulation in (accelerated) real time

In Skinner's Reflex Reserve theory, reinforced responses added to a reserve depleted by responding. It could not handle the finding that partial reinforcement generated more responding than continuous reinforcement, but it would have worked if its growth had depended not just on the last response but also on earlier responses preceding a reinforcer, each weighted by delay. In that case, partial reinforcement generates steady states in which reserve decrements produced by responding balance increments produced when reinforcers follow responding. A computer simulation arranged schedules for responses produced with probabilities proportional to reserve size. Each response subtracted a fixed amount from the reserve and added an amount weighted by the reciprocal of the time to the next reinforcer. Simulated cumulative records and quantitative data for extinction, random-ratio, random-interval, and other schedules were consistent with those of real performances, including some effects of history. The model also simulated rapid performance transitions with changed contingencies that did not depend on molar variables or on differential reinforcement of inter-response times. The simulation can be extended to inhomogeneous contingencies by way of continua of reserves arrayed along response and time dimensions, and to concurrent performances and stimulus control by way of different reserves created for different response classes.

Rats' performance on variable-interval schedules with a linear feedback loop between response rate and reinforcement rate.

Three experiments investigated whether rats are sensitive to the molar properties of a variable-interval (VI) schedule with a positive relation between response rate and reinforcement rate (i.e., a VI+ schedule). In Experiment 1, rats responded faster on a variable ratio (VR) schedule than on a VI+ schedule with an equivalent feedback function. Reinforced interresponse times (IRTs) were shorter on the VR as compared to the VI+ schedule. In Experiments 2 and 3, there was no systematic difference in response rates maintained by a VI+ schedule and a VI schedule yoked in terms of reinforcement rate. This was found both when the yoking procedure was between-subject (Experiment 2) and within-subject (Experiment 3). Mean reinforced IRTs were similar on both the VI+ and yoked VI schedules, but these values were more variable on the VI+ schedule. These results provided no evidence that rats are sensitive to the feedback function relating response rate to reinforcement rate on a VI+ schedule.

MOLAR AND MOLECULAR CONTROL IN VARIABLE‐INTERVAL AND VARIABLE‐RATIO SCHEDULES

Response rates are typically higher under variable‐ratio than under variable‐interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable‐interval‐with‐added‐linear‐feedback schedule is a variable‐interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable‐ratio 15, 30, and 60 food reinforcement schedules, variable‐interval 15‐, 30‐, and 60‐s food reinforcement schedules, and two versions of variable‐interval‐with‐added‐linear‐feedback 15‐, 30‐, and 60‐s food reinforcement schedules. Response rates on the variable‐interval‐with‐added‐linear‐feedback schedule were similar to those on the variable‐interval schedule; all three schedules led to lower response rates than those on the variable‐ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable‐interval‐with‐added‐linear‐feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable‐interval schedules in rats are lower than those on comparable variable‐ratio schedules, primarily because the former schedules reinforce long interresponse times.

RESPONSE‐RATE DIFFERENCES IN VARIABLE‐INTERVAL AND VARIABLE‐RATIO SCHEDULES: AN OLD PROBLEM REVISITED

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate-reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.

Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the str...

Constraints on the process of interresponse-time reinforcement as the explanation of variable-interval performance

Computer modelling was used to investigate the adequacy of the principle of reinforcement of single interresponse-times (IRTs) as an explanation of variable-interval performance. Variations in three components of models employing IRT reinforcement were simulated; these were (i) rules for selecting previously-reinforced IRTs from the IRT memory as “candidates” for output (acceptance rules), (ii) processes involving the transformation of previously-reinforced IRTs to generate behavioural variability, and (iii) the role of the minimum IRT value permitted by the models. In general, simulation output resembled experimental data only when (i) acceptance rules discriminated strongly against long, and in favour of short, IRTs; and (ii) when IRTs were transformed by distributions in which standard deviation was a large fraction (50–125%) of the mean, thus generating substantial IRT variability. Simulation results were analyzed by means of Herrnstein's hyperbolic equation, and in general, it was found that the output of IRT reinforcement models was insensitive to the rate of reinforcement received, as well as conforming less well to Herrnstein's equation than did experimental data. Overall, therefore, it was concluded that models of the reinforcement of single IRTs could provide an adequate simulation of VI performance only when the processes within them operated within narrow constraints. Even then, models involving the reinforcement of single IRTs could adequately simulate only those VI data in which response rate was insensitive to reinforcement rate.

Preference for starting and finishing behavior patterns.

Pigeon's key pecking was reinforced with food in two experiments in which the correspondence between preference for starting one of two reinforced behavior patterns and the likelihood of finishing it subsequently was examined. Reinforcers were scheduled according to concurrent schedules for two classes of interresponse times, modified such that reinforcers followed a center-key peck terminating either a shorter interresponse time started by a left-key peck or a longer interresponse time started by a right-key peck. In Experiment 1, the times when reinforcers potentially were available were not discriminated, whereas in Experiment 2 they were. Absolute reinforced pattern durations were varied. The relative frequency of starting a particular pattern was highly correlated with relative frequency of that completed pattern in both experiments. Other relations between starting and finishing a pattern depended on whether reinforced interresponse times were discriminated. For instance, preference for starting a pattern sometimes correlated negatively with the likelihood of subsequently completing it. The present experiments are described as capturing part of the ordinary language meaning of "intention," according to which an organism's behavior at one moment sets the occasion for an observer to say that the organism "intends" in the future to engage in one behavior rather than another.

Influence of interresponse time reinforcement on signalled-reward effect.

Trois experiences etudient chez le rat les conditions de production d'une augmentation ou d'une diminution du taux de reponses de pression de levier par la presentation d'un signal de renforcement

Influence of interresponse time reinforcement on signalled-reward effect.

Influence of interresponse time reinforcement on signalled-reward effect.

MAXIMIZING PRESENT VALUE: A MODEL TO EXPLAIN WHY MODERATE RESPONSE RATES OBTAIN ON VARIABLE‐INTERVAL SCHEDULES

In Phases 1 and 3, two Japanese monkeys responded on a multiple variable-ratio 80 variable-interval X schedule, where the value of X was adjusted to ensure equal between-schedule reinforcement rates. Components strictly alternated following the delivery of a food pellet, and each session ended following 50 components. Phase 2 differed from the others only in that the 50 pellets previously earned during the session were delivered together at session's end. Variable-ratio response rates did not decrease across phases, but variable-interval response rates decreased substantially during the Phase 2 procedure. This rate decrease was attributed to the food-at-session's-end manipulation removing the greater immediacy of reinforcement provided by short interresponse times relative to long interresponse times. Without this time preference for short interresponse times, the variable-interval interresponse-time reinforcement feedback function largely controlled response emission, dictating a response-rate reduction. This result was explained in terms of the economic notion of "maximizing present value."

Interresponse-time reinforcement and behavior under aperiodic reinforcement schedules: A case study using computer modeling.

Interresponse-time reinforcement and behavior under aperiodic reinforcement schedules: A case study using computer modeling.