Rate Of Primary Reinforcement Research Articles

MATCHING AND CONDITIONED REINFORCEMENT RATE

Attempts to examine the effects of variations in relative conditioned reinforcement rate on choice have been confounded by changes in rates of primary reinforcement or changes in the value of the conditioned reinforcer. To avoid these problems, this experiment used concurrent observing responses to examine sensitivity of choice to relative conditioned reinforcement rate. In the absence of observing responses, unsignaled periods of food delivery on a variable-interval 90-s schedule alternated with extinction on a center key (i.e., a mixed schedule was in effect). Two concurrently available observing responses produced 15-s access to a stimulus differentially associated with the schedule of food delivery (S+). The relative rate of S+ deliveries arranged by independent variable-interval schedules for the two observing responses varied across conditions. The relation between the ratio of observing responses and the ratio of S+ deliveries was well described by the generalized matching law, despite the absence of changes in the rate of food delivery. In addition, the value of the S+ deliveries likely remained constant across conditions because the ratio of S+ to mixed schedule food deliveries remained constant. Assuming that S+ deliveries serve as conditioned reinforcers, these findings are consistent with the functional similarity between primary and conditioned reinforcers suggested by general choice theories based on the concatenated matching law (e.g., contextual choice and hyperbolic value-added models). These findings are inconsistent with delay reduction theory, which has no terms for the effects of rate of conditioned reinforcement in the absence of changes in rate of primary reinforcement.

RATE OF CONDITIONED REINFORCEMENT AFFECTS OBSERVING RATE BUT NOT RESISTANCE TO CHANGE

The effects of rate of conditioned reinforcement on the resistance to change of operant behavior have not been examined. In addition, the effects of rate of conditioned reinforcement on the rate of observing have not been adequately examined. In two experiments, a multiple schedule of observing-response procedures was used to examine the effects of rate of conditioned reinforcement on observing rates and resistance to change. In a rich component, observing responses produced a higher frequency of stimuli correlated with alternating periods of random-interval schedule primary reinforcement or extinction. In a lean component, observing responses produced similar schedule-correlated stimuli but at a lower frequency. The rate of primary reinforcement in both components was the same. In Experiment 1, a 4:1 ratio of stimulus production was arranged by the rich and lean components. In Experiment 2, the ratio of stimulus production rates was increased to 6:1. In both experiments, observing rates were higher in the rich component than in the lean component. Disruptions in observing produced by presession feeding, extinction of observing responses, and response-independent food deliveries during intercomponent intervals usually were similar in the rich and lean components. When differences in resistance to change did occur, observing tended to be more resistant to change in the lean component. If resistance to change is accepted as a more appropriate measure of response strength than absolute response rates, then the present results provide no evidence that higher rates of stimuli generally considered to function as conditioned reinforcers engender greater response strength.

Delay reduction theory: Choice, value, and conditioned reinforcement.

Numerous theories have been proposed to explain how organisms choose between different alternatives. Two areas of research in particular have serious implications for the validity of these choice theories: one, is the investigation of whether difference or ratio controls choice, and another is the investigation into the role that temporal context exerts on conditioned reinforcement value. Recent developments in both areas of research, as well as the implications for various choice theories, are reviewed. Findings are generally consistent with delay reduction theory: however, further research is clearly needed. According to delay reduction theory (DRT), the conditioned reinforcing strength of a stimulus may be predicted by calculating the reduction in the duration of time to primary reinforcement signaled by that stimulus (Fantino, 1969; 1981; Squires & Fantino, 1971). The simplest form of DRT (from Fantino, 1981) states that: Reinforcement strength of stimulus A = f (T-[t.sub.A]/T] (1) where [t.sub.A] is the time between the appearance of stimulus A and primary reinforcement, and where T is the overall time between reinforcer presentations. Thus, a stimulus that signals a greater improvement in time to reinforcement will be a more effective conditioned reinforcer than a stimulus that signals less of an improvement. DRT has been extended, with some success, to self-control (Ito & Asaki, 1982; Navarick & Fantino, 1976), elicited responding (Fantino, 1982), three-alternative choice (Fantino & Dunn, 1983), observing (Case & Fantino, 1981; Fantino & Case, 1983), operant analogues to foraging (Abarca & Fantino, 1982; Fantino & Arbaca, 1985), percent reinforcement (Spetch & Dunn, 1987), and the serial-position effect in short-term memory (Wixted, 1989). Perhaps the most common procedure for assessing choice is the concurrent-chains procedure, as depicted in Figure 1, where responding to concurrent initial-link schedules (the choice phase) is reinforced by the presentation of two mutually exclusive terminal-link schedules (the outcome phase), completion of which leads to primary reinforcement. Following primary reinforcement, the initial links are reinstated and a new trial begins. Preference, and presumably conditioned reinforcement value, of the terminal links is measured by response allocation during the initial links. [FIGURE 1 OMITTED] Figure 1. Diagram of the concurrent-chains procedure. Panels A and B are the sequence of events when responses on the left and right are reinforced, respectively. Circles represent the response keys with the schedule requirement indicated within. The initial links are normally equal VI schedules with the same color, while the terminal links are on different schedules, each having a unique color. Responses to the terminal links are reinforced with food according to some schedule. Equation 1 represents the essence, or kernel, of DRT. Applying DRT to the concurrent-chains procedure however, requires that the delay reduction signaled by each alternative be weighted by the rate of primary reinforcement for that alternative. Thus, the Squires and Fantino (1971) form of DRT is written such that: [B.sub.L]/[B.sub.L] + [B.sub.R] = [r.sub.L] (T - [t.sub.L])/[r.sub.L] (T - [t.sub.L] + [r.sub.R] (T - [t.sub.R]) for [t.sub.L] and [t.sub.R] = 1, for [t.sub.L] , = 0, for [t.sub.L] where [B.sub.L] and [B.sub.R] are the choice responses on the left and right initial links, respectively; [r.sub.L] and [r.sub.R] are the overall rates of primary reinforcement on the left and right keys, respectively; T is the average overall time to primary reinforcement measured from the onset of the choice phase; and [t.sub.L] and [t.sub.R] are the average delays during the terminal links on the left and right keys, respectively. …

CHOICE AND CONDITIONED REINFORCEMENT

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.

Choice and the Conditioned Reinforcing Strength of Informative Stimuli

In two experiments using the concurrent chains procedure, pigeons were presented with a choice between alternatives in which the prevailing stimuli were correlated or uncorrelated with differing conditions of reinforcement. In both experiments, the overall rate of primary reinforcement afforded by the alternative with the correlated stimuli was lower than that afforded by the alternative with the uncorrelated stimulus. The difference between rates of primary reinforcement was achieved through selective manipulation of the probability of the reinforcement (Experiment 1) and duration of the interreinforcement interval (Experiment 2) in the alternative with the correlated stimuli. The results indicated that the alternative with the correlated stimuli maintained at least as much choice responding as did the alternative with the uncorrelated stimulus, despite the lower overall rate of primary reinforcement. The results further suggested that the conditioned reinforcement provided by the stimulus positively correlated with the availability of reinforcement compensated for the lower overall rate of primary reinforcement from that alternative.

The “Gambler Effect”: Choice and the Probability of Reinforcement of Small Response Outputs

Several types of concurrent-chains studies in which preference was found disproportionate to the rate of primary reinforcement were discussed. The disproportionality was attributed to the unusual effect on preference of the smallest interreinforcement interval components of schedules of reinforcement. The present study investigated an aspect of this factor in the control of preference. Concurrently available fixed-ratio 10 schedules were chained to a fixed-ratio 30 schedule on one key and a multiple fixed-ratio 5 fixed-ratio 80 on the alternative key. In the course of the study the relative frequency of the smallest interreinforcement interval (FR 5) in the multiple schedule option was reduced from 0.50 to 0. Pigeons could maximize reinforcement by always choosing the FR 30 schedule or choosing the multiple schedule and “gambling” that the FR 5 would be produced. Three pigeons chose the multiple schedule exclusively with one exception until the FR 5 was removed entirely from the multiple schedule. At this point, all birds chose the fixed schedule exclusively. Preference defined as rate of initial links responding was, in most instances, inversely related to rate of primary reinforcement.

Effects of percentage reinforcement on choice in a concurrent chain schedule

The effects of random reinforcement omissions (percentage reinforcement) on choice behavior was studied with three pigeons trained on a two-link concurrent chain schedule. A terminal-link VI schedule on one key always produced primary reinforcement; a terminal-link VI-percentage reinforcement schedule on the other key produced with equal probability either reinforcement or blackout. Relative response rates in the choice links matched the relative rates of reinforcement associated with the terminal-link schedules when these were VI 30 sec vs VI 30 sec (50%); VI 1 min vs VI 30 sec (50%); and VI 30 sec vs VI 30 sec (baseline). These results suggest that percentage reinforcement affects choice solely through its effect on rate of primary reinforcement.