Relative Reinforcement Rate Research Articles

DETERMINANTS OF CONTRAST IN THE SIGNAL‐KEY PROCEDURE: EVIDENCE AGAINST ADDITIVITY THEORY

Two experiments are reported that challenge the interpretation of previous results with the signal-key procedure, in which the discriminative stimuli are located on a response key different from the key associated with the operant response requirement. Experiment 1 replicated the procedure of Keller (1974), and found that contrast effects on the operant key occurred reliably for only one of four subjects. High rates to the signal key initially occurred for only one subject, but modifications of the procedure produced substantial rates to the signal key for all subjects. In all cases, however, signal-key behavior was greatly reduced by the addition of a changeover delay which prevented reinforcement within 2 seconds of the last peck to the signal key, suggesting that signal-key pecking was maintained primarily by adventitious reinforcement. Experiment 2 modified the signal-key procedure by using three response keys, so that the discriminative stimuli on the signal key controlled different responses during all phases of training. With this modification, reliable contrast effects on the operant key occurred for all subjects, suggesting that the failure to find contrast in previous studies has been due to the confounding of changes in the discrimination requirements with changes in relative rate of reinforcement. The results challenge the additivity theory of contrast, and suggest that "elicited" behavior plays a minor role, if any, in the determination of contrast effects in multiple schedules.

INDEPENDENCE OF SENSITIVITY TO RELATIVE REINFORCEMENT RATE AND DISCRIMINABILITY IN SIGNAL DETECTION

Six pigeons were trained to detect differences between two white stimuli, S1 and S2, differing in duration and arranged probabilistically on the center key of a three-key chamber. Detection performance was measured at two levels of discriminability. At one level, S1 was five seconds and S2 was thirty seconds. At the other level, S1 was twenty seconds and S2 was thirty seconds. The procedure was a standard signal-detection yes-no design in which stimulus-presentation probability was varied from .1 to .9 at both discriminability levels. On completion of the center-key stimulus, a peck on the center key darkened the center-key light and turned on the two red side keys. A left-key response was "correct" on S1 trials, and a right-key response was "correct" on S2 trials. Correct responses produced food reinforcement on a variable-ratio 1.3 schedule. Incorrect responses produced three second blackout. Discriminability was higher for the five-second versus thirty-second conditions than for the twenty-second versus thirty-second conditions, but there were no differences in sensitivity of behavior to reinforcement variation for the two stimulus pairs. Response bias was a function of the relative reinforcement rate for correct choice responses.

Choice responding following multiple schedule training

Pigeons’ choice responding on 10-sec interpolated probes was studied after baseline training on multiple variable-interval variable-interval schedules of food reinforcement. Unreinforced choice following training with three different relative reinforcement rates (Experiment 1), with a 3-ply multiple schedule (Experiment 2), and with three different relative reinforcement durations (Experiment 3) was examined. Least squares lines were fit to choice relative response rate and schedule relative response rate as functions of training relative reinforcement rate; choice slope was significantly greater than schedule slope in all three experiments. This result is counter to the prediction of Herrnstein’s (1970) theory that these slopes should not differ. Luce’s (1959) theory also failed to account for the data. It was concluded that choice responding was controlled by both approach to the stimulus associated with the smaller mean interreinforcer interval or the longer duration, and avoidance of the other stimulus.

Temporal discrimination and a free-operant psychophysical procedure.

Pigeons were presented a series of keylight time periods (separated by blackouts) during which two response keys were lit, one by blue light and the other either by orange or green. Blue-key responses changed the color on the other key. Orange-key responses sometimes produced food during the first half of a time period; green-key responses sometimes produced food during the second half. In three experiments, the probability of a green-key response increased as a function of elapsed time. Experiment 1 compared performance when the duration of the keylight periods was varied across a wide range. Discrimination of performance was similar across the range of durations. Experiment 2 varied both relative reinforcement rate and the local reinforcement rate for orange-key and green-key responses. These manipulations produced changes in response bias but not discrimination sensitivity. Experiment 3 varied the local temporal placement of reinforcers within time periods and demonstrated that choice behavior was affected by differential reinforcement at different points during the time periods. The results were consistent with previous research on duration discrimination that used psychophysical trials procedures.

Choice in Concurrent Schedules with a Fixed-Duration Alternative

The experiment was based on the hypothesis that changeover behavior is maintained by the relatively immediate consequences of the changeover response. Time allocation and relative response rates were measured in a situation where the changeover response resulted in a fixed duration stay in one of two random-interval schedules. Under these conditions, time distributions tended to match relative rates of reinforcement only if the duration of this imposed stay was approximately equal to the in-terchangeover time that was emitted when the opportunity to change over was always available. Interchangeover times were insensitive to the duration of the imposed stay in a schedule. Relative response rates did not always match relative rates of reinforcement, nor did they always conform to time allocation. The results are discussed in terms of local control of responding.

Human Concurrent Performances: Variation of Reinforcer Magnitude and Rate of Reinforcement

Humans participated in experiments using concurrent variable-interval variable-interval schedules. Reinforcer magnitude or rate of reinforcement availability was varied. Relative rate of responding was insensitive to changes in relative reinforcer magnitude. When relative rate of reinforcement was altered, relative rate of responding increased in the component with the shorter average interreinforcement interval. Participants were also exposed to concurrent variable-interval extinction schedules to test for sensitivity to schedule parameters. All participants continued responding in the variable-interval component while responding in the extinction component diminished. The current study is one of a limited number of experiments that examine responding by humans in procedures designed to approximate those used in animal studies. The findings are consistent with those of some animal studies in which responding under concurrent variable-interval variable-interval schedules was examined. In particular, the data indicate that frequency of reinforcement affects response distribution in humans more than does magnitude of reinforcement. Thus frequent delivery of smaller reinforcers may be more effective in modifying human behavior than infrequent delivery of larger reinforcers.

CHOICE: EFFECTS OF CHANGEOVER SCHEDULES ON CONCURRENT PERFORMANCE

The components of concurrent schedules were separated temporally by placing interval schedules on the changeover key. The rates of responding on both the main and changeover keys were examined as a function of the reinforcement rates. In the first experiment, the sensitivity of main-key performance to changing reinforcement rates was inversely related to the temporal separation of components, and changeover performance was monotonically related to the ratio of the reinforcement rates. In the second experiment, when the ratio of the reinforcement rates was scheduled to remain constant while the frequency of reinforcement was varied, changeover performance did not remain constant. A "sampling" interpretation of changeover responding was proposed and subsequently tested in a third experiment where extinction was always scheduled in one component and the frequency of reinforcement was varied in the second component. It was concluded that changeover performance can be interpreted using molar measures of reinforcement and that animals sample activities available to them at rates which are controlled by relative reinforcement rates.

Effects of a Limited Hold on Changeovers Maintained by Concurrent Interval Schedules of Reinforcement

In order to investigate the effect of reinforcers occurring immediately after a changeover on the changeover behavior of pigeons, limited holds of various durations were placed on one of two concurrently available random- interval schedules. It was found that the limited hold generally had the effect of reducing the relative number of responses and the relative amount of time allocated to the schedule with the limited hold, and that this effect was primarily the result of an increased latency of changeover into the limited hold schedule. This tendency could be augmented by increasing the duration of a changeover delay from 2 sec. to 8 sec. Despite the fluctuations in behavior induced by these manipulations, response and time matching to relative rates of reinforcement prevailed. Implications for various hypotheses regarding the fundamental nature of the behavior maintained by concurrent interval schedules of reinforcement were discussed.

Escape from Noise by Rats and Undermatching

Responses of four albino rats on two levers were followed by escape from 111 dB white noise according to pairs of concurrent variable-interval schedules. Stable, moderate responding was maintained. There was substantial undermatching of relative response rate to relative reinforcement rate. Several alternative explanations for this deviation from matching are suggested.

RESPONSE BIAS AND THE DISCRIMINATION OF STIMULUS DURATION1

Pigeons discriminated stimulus duration in a psychophysical choice situation. Following presentation of any duration from a set of short duration (11 to 15 sec), responses on a red key were reinforced intermittently. Following presentation of any duration from a set of long durations (16 to 22 sec), responses on a green key were reinforced intermittently. Relative reinforcement rates were manipulated for choice responses across conditions. As relative reinforcement rates were varied, psychometric functions showed shifts in green-key responses at all durations. A signal-detection analysis showed that sensitivity remained roughly constant across conditions while response bias changed as a function of changes in relative reinforcement rate. Relative error rates tended to match relative reinforcement rates.

MATCHING WITH A TRIO OF CONCURRENT VARIABLE‐INTERVAL SCHEDULES OF REINFORCEMENT

A trio of concurrent variable-interval schedules of reinforcement was arranged according to a changeover-key procedure, including a changeover delay of 1.5 sec. The three schedules provided a combined maximum reinforcement rate of 45 reinforcements per hour. With that restriction, the nine experimental conditions included several combinations of variable-interval schedules, sometimes including extinction. The pigeons matched relative response rate and relative time to relative reinforcement rate. Relative time appeared to match some-what better than relative response rate. Performance adjusted rapidly from one experimental condition to the next, whether the change involved two or all three schedules of the concurrent trio.

CONCURRENT SECOND‐ORDER SCHEDULES OF REINFORCEMENT1

Responses on one key (the main key) of a two-key chamber produced food according to a second-order variable-interval schedule with fixed-interval schedule components. A response on a second key (the changeover key) alternated colors on the main key and provided a second independent second-order variable-interval schedule with fixed-interval components. The fixed-interval component on one variable-interval schedule was held constant at 8 sec, while the fixed interval on the other variable-interval schedule was varied from 0 to 32 sec. Under some conditions, a brief stimulus terminated each fixed interval and generated fixed-interval patterns; in other conditions, the brief stimulus was omitted. Relative response rate and relative time deviated substantially from scheduled relative reinforcement rate and, to a lesser extent, from obtained relative reinforcement rate under both brief-stimulus and no-stimulus conditions. Matching was observed with equal components on both schedules; with unequal components, increasingly greater proportions of time and responses than the matching relation would predict were spent on the variable-interval schedule containing the shorter component. Preference for the shorter fixed interval was typically more extreme under brief-stimulus than under no-stimulus schedules. The results limit the extension of the matching relation typically observed under simple concurrent variable-interval schedules to concurrent second-order variable-interval schedules.

SHORT‐COMPONENT MULTIPLE SCHEDULES: EFFECTS OF RELATIVE REINFORCEMENT DURATION1

Pigeons were exposed to multiple variable-interval 2-min variable-interval 2-min schedules of food presentation in which relative duration of food presentation was manipulated. When components alternated every 5 sec and were scheduled on separate response keys, relative response rates closely matched relative reinforcement duration in three of four pigeons. On the other hand, relative response rates were insensitive to relative reinforcement duration when components scheduled on a single response key alternated every 5 sec, and when components scheduled on separate response keys alternated every 2 min. Thus, both rapid alternation and spatial separation of components were necessary to produce approximate matching of relative responding to relative reinforcement duration. This finding contrasts with previous findings that only rapid component alternation is necessary for matching when relative rate of reinforcement is manipulated.

Concurrent schedule responding as a function of body weight

Five pigeons pecked for food reinforcement on several concurrent schedules. Their body weights were varied from 80% to 110% of their free-feeding weights. A number of predictions of the equations proposed by Herrnstein (1970) were tested. As predicted, the relative rate of responding equalled the relative rate of reinforcement for all subjects, on all schedules, at all body weights. And, as predicted, the overall rates of responding on the components of a concurrent schedule were slower than the local rates of responding on the components of an identical multiple schedule. Contrary to prediction, the total rate of responding generated by the concurrent schedules did not increase with increases in the total rate of reinforcement they provided. And, contrary to prediction, the k parameter did not remain constant, and the R0 parameter did not increase with increases in body weight. It was concluded that Herrnstein’s matching law and his interpretation of the m parameter are correct but that the interpretations of k and R0 require further investigation.

MATCHING AND CONTRAST ON SEVERAL CONCURRENT TREADLE‐PRESS SCHEDULES

Four White King pigeons pressed treadles for food reinforcement on several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding in one of the two component schedules was held constant at 30 reinforcers per hour. The rate of reinforcement available for responding in the other was varied from 120 to 60 to 15, and then to 30 reinforcers per hour. The relative rate of responding in each component schedule equalled the relative rate of reinforcement that the component provided. And, behavioral contrast, defined as an inverse relationship between the rate of responding in the constant component and the rate of reinforcement obtained by responding in the other component, occurred for all schedules.

Perseveration in Concurrent Performances by the Developmentally Retarded

Perseveration in choice behavior of retardates was defined as failure to match absolute and relative response rates to changes in relative reinforcement rates with concurrent schedules of reinforcement. 4 experiments showed that concurrent performances of retardates varied systematically as a function of type of schedule, schedule value, duration, and symmetry of changeover delays. Relative but not absolute response rates were proportional to relative reinforcement rates with long changeover delays. Increasing the changeover delay punished changeovers and reduced total reinforcement rates. Contrary to prediction, asymmetrical changeover delays disrupted rather than improved matching. The retardate’s response is similar to that of Weiner’s “one-behavior organism” even in concurrent situations by persisting in maladaptive response patterns until net gains or losses of reinforcement are made contingent upon changes in response patterns.

Reinforcement probability and concurrent operants

When reinforcements for concurrent operants are programmed by independent variable-interval schedules of reinforcement, the relative response rate obtained for each operant is found to approximate the relative reinforcement rate for that operant. This relationship, called matching, is a necessary consequence of a maximizing principle which states that the operant emitted is the one having the momentarily greater reinforcement probability. Matching, however, can come about in other ways as well. To test the maximizing principle rats were trained to respond on either one of two levers. Whether a leverpress was reinforced was governed by a random process having a constant reinforcement probability which was not necessarily the same for the two levers. It was found that, after a few sessions’ exposure, responding occurred nearly exclusively to the lever having the greater reinforcement probability as predicted by the maximizing principle.

EFFECTS OF REINFORCEMENT RATE AND REINFORCER MAGNITUDE ON CHOICE BEHAVIOR OF HUMANS1

Two experiments with human subjects investigated the effects of rate of reinforcement and reinforcer magnitude upon choice. In Experiment 1, each of five subjects responded on four concurrent variable-interval schedules. In contrast to previous studies using non-human organisms, relative response rate did not closely match relative rate of reinforcement. Discrepancies ranged from 0.03 to 0.43 (mean equal to 0.19). Similar discrepancies were found between relative amount of time spent responding on each schedule and the corresponding relative rates of reinforcement. In Experiment 2, in which reinforcer magnitude was varied for each of five subjects, similar discrepancies ranging from 0.05 to 0.50 (mean equal to 0.21), were found between relative response rate and relative proportion of reinforcers received. In both experiments, changeover rates were lower on the long-interval concurrent schedules than on the short-interval ones. The results suggest that simple application of previous generalizations regarding the effects of reinforcement rate and reinforcer magnitude on choice for variable-interval schedules does not accurately describe human behavior in a simple laboratory situation.

SIGNAL DETECTION METHODS FOR MEASUREMENT OF UTILITY IN ANIMALS1

Analytic methods of signal detection theory were employed to assess the utility of reinforcers. Four pigeons were trained to detect the presence or absence of a stimulus by pecking one of two side keys in a trial-by-trial choice paradigm. The relative rate of positive reinforcement for correct choices was varied to offset the biasing effects of electric shock for incorrect right side-key choices. The effects of relative rate of reinforcement on bias were similar at all shock intensities even though the subjects' sensitivity changed during the course of the experiment. The relative rate of reinforcement required to produce equal bias was calculated and plotted against shock intensity to generate utility functions. The relative rate of reinforcement necessary to offset the bias induced by shock was an increasing function of shock intensity.

ON THE FORM OF THE RELATION BETWEEN RESPONSE RATES IN A MULTIPLE SCHEDULE1

Three pigeons received training on multiple variable-interval schedules with brief alternating components, concurrently with a fixed-interval schedule of food reinforcement on a second key. Fixed-interval performance exhibited typical increases in rate within the interval, and was independent of multiple-schedule responding. Responding on the multiple-schedule key decreased as a function of proximity to reinforcement on the fixed-interval key. The overall relative rate of responding in one component of the multiple schedule roughly matched the overall relative rate of reinforcement. Within the fixed interval, response rate during one multiple-schedule component was a monotonic, negatively accelerated function of response rate during the other component. To a first approximation, the data were described by a power function, where the exponent depended on the relative rate of reinforcement obtained in the two components. The relative rate of responding in one component of the multiple schedule increased as a function of proximity to fixed-interval reinforcement, and often exceeded the overall obtained relative rate of reinforcement. The form of the function relating response rates is discussed in relation to findings on rate-dependent effects of drugs, chaining, and the relation between response rate and reinforcement rate in single-schedule conditions.