Delay Of Reinforcement Research Articles

Fixed and variable ratios and delays: Further tests of an equivalence rule.

A discrete-trial procedure was used to measure pigeons' choices between fixed and variable ratio schedules and between fixed and variable delays before reinforcement. A peck at a green key produced a reinforcement schedule that was constant within a condition but varied across conditions. A peck at a red key produced a ratio schedule (or, in other conditions, a simple delay) whose size was increased or decreased many times a session, depending on the subject's previous choices. The purpose of these adjustments was to estimate an indifference point--a ratio size (or delay duration) at which the subject chose each key about equally often. The results were used to test a simple "equivalence rule" for choices between fixed and variable schedules (Mazur, 1984). This rule, which was applied without using free parameters, predicted the major trends in the obtained indifference points from both ratio and delay conditions. However, some small but consistent deviations from the predictions were apparent. Better predictions were generated with a more complex equation, which included parameters reflecting the subjects' sensitivities to delay of reinforcement and to events of different probabilities. It was concluded that a successful equivalence rule must include parameters that can be adjusted to describe the effects of delay and probability in a given experimental setting. Once these parameters are estimated, however, choices involving both fixed and variable delays and fixed and variable ratios can be accurately predicted with the same equation.

Reinforcement delay of one second severely impairs acquisition of brain self-stimulation

The effect of delayed reinforcement on the acquisition of lateral hypothalamic self-stimulation was investigated. Brain stimulation reinforcement minimizes cues associated with reinforcement delivery (secondary reinforcement) and, by eliminating consummatory responses, permits precise temporal control of the interval between the operant response and reinforcement. Different groups were trained in daily 1-h sessions for brain stimulation reinforcement at one of 4 delay intervals (1, 2, 3 or 6 s). Responses made during the delay interval were not reinforced and reset the delay timer. Control groups (IMMEDIATE) were reinforced immediately, but were required to space responses--according to a delayed reinforcement of low rates (DRL) schedule--for an interval corresponding to one of the delay of reinforcement intervals. The DRL schedule equalized opportunities for reinforcement and non-reinforcement. At all intervals, rats trained with delayed reinforcement had significantly lower bar-press rates than controls trained with immediate reinforcement under DRL. When reinforcement schedules were switched (DELAY groups now get IMMEDIATE and vice versa), response rates rapidly shifted to levels appropriate to the new schedule. The pre-switch results indicate that delays even as short as 1 s markedly impede the acquisition of self-stimulation behavior. The post-switch results suggest that delay of reinforcement, like stimulation intensity, may determine the strength of hypothalamic reinforcement and hence final levels of performance.

Resistance to variable-time schedules produced by spaced-response reinforcement schedules

Two experiments are reported in which rats were exposed to either a tandem fixed-ratio 1 differential-reinforcement-of-low-rate 10 sec schedule (tand FR1 DRL 10) (Experiment 1) or to a DRL 10 sec schedule (Experiment 2) prior to being exposed to a variable-time (VT) schedule. Response decrement was not universally found during the VT phase, one rat emmitting an increase in response rate relative to baseline (Experiment 1), while another showed neither an increase nor decrease during VT relative to baseline (Experiment2). The VT schedule induced less efficient responding in both experiments. Interresponse time (IRT) distributions (Experiment 2) indicated that the VT changed the pattern of responding. These results, when combined with others from our laboratory, indicate that although unsignalled variable delay of reinforcement may be a sufficient condition for producing resistance to response-independent reinforcement, it is not a necessary condition. It was concluded that a response competition model of the kind proposed by Henton and Iversen (1978) might have explanatory merit in this kind of experimental situation.

Probability and delay of reinforcement as factors in discrete-trial choice.

Pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck on the red key always produced a delay of 5 s and then a possible reinforcer. The probability of reinforcement for responding on this key varied from .05 to 1.0 in different conditions. A response on the green key produced a delay of adjustable duration and then a possible reinforcer, with the probability of reinforcement ranging from .25 to 1.0 in different conditions. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices. The purpose of these adjustments was to estimate an indifference point, or a delay that resulted in a subject's choosing each alternative about equally often. In conditions where the probability of reinforcement was five times higher on the green key, the green-key delay averaged about 12 s at the indifference point. In conditions where the probability of reinforcement was twice as high on the green key, the green-key delay at the indifference point was about 8 s with high probabilities and about 6 s with low probabilities. An analysis based on these results and those from studies on delay of reinforcement suggests that pigeons' choices are relatively insensitive to variations in the probability of reinforcement between .2 and 1.0, but quite sensitive to variations in probability between .2 and 0.

The effect of food deprivation on self-control

Two experiments examined the effect of food deprivation on choice in a discrete-trials self-control paradigm, choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer. In Experiment 1, four pigeons were each deprived to 65%, 80%, and 90% of their free-feeding weights, and the delay to the smaller reinforcer was varied. Deprivation level did not affect choice, but the rate of ineffective key pecks made during the reinforcer delays increased as deprivation increased. In Experiment 2, four pigeons were exposed to conditions in which they were fed up to their 80% free-feeding weights following experimental sessions, and in which they were given no postsession feedings. Both the pigeons' weights and their latencies to insert their heads into the food hopper when food was available were lower when the pigeons were not fed following experimental sessions. Choice showed no change. Deprivation level affects response rate and eating behavior in these procedures, but not choice.

Utilization of odor cues as a function of reward-magnitude contrast and delay of reinforcement.

An experiment manipulating both reward-magnitude contrast and delay of reinforcement is reported. The results indicated that odor cues are produced under conditions of delayed large-reward versus nonreward contrast but not under large- versus small-reward contrast. Additional magnitude and delay manipulations that resulted in the elimination of odor-based double-alternation patterning in the straight runway are presented.

Influences of delay and rate of reinforcement on discrete-trial choice.

An adjusting procedure was used to measure pigeons' preferences among alternatives that differed in the duration of a delay before reinforcement and of an intertrial interval (ITI) after reinforcement. In most conditions, a peck at a red key led to a fixed delay, followed by reinforcement, a fixed ITI, and then the beginning of the next trial. A peck at a green key led to an adjustable delay, reinforcement, and then the next trial began without an ITI. The purpose of the adjusting delay was to estimate an indifference point, or a delay that made a subject approximately indifferent between the two alternatives. As the ITI for the red key increased from 0 s to 60 s, the green-key delay at the indifference point increased systematically but only slightly. The fact that there was some increase showed that pigeons' choices were controlled by more than simply the delay to the next reinforcer. One interpretation of these results is that besides delay of reinforcement, rate of reinforcement also influenced choice. However, an analysis that ignored reinforcement rate, but considered the delays between a choice response and the reinforcers on subsequent trials, was able to account for most of the obtained increases in green-key delays. It was concluded that in this type of discrete-trial situation, rate of reinforcement exerts little control over choice behavior, and perhaps none at all.

Influences of delay and rate of reinforcement on discrete-trial choice.

Influences of delay and rate of reinforcement on discrete-trial choice.

A test of the predicted relation describing the change in response rate with the number of intervals on an FI schedule

Two experiments, one with rats and the other with pigeons, supported the prediction that the logarithm of the rate of responding in the last fifth of a fixed interval is linearly related to the logarithm of the number of intervals to which the organism is exposed. This prediction was derived with the use of logical positivism from Hull’s law on delay of reinforcement. The support of this relation strengthens the possible role that deterministic equations can play in learning theory.

A correction to Hull’s law on delay of reinforcement and its extension to fixed-interval operant schedules

Hull’s law on delay of reinforcement was corrected for internal consistency by testing the law against the data from which it was derived. Using these same data, the law was extended to be a function of trials. The resulting equation was extended further by mathematically transforming the latency of responding into the rate of responding, and equating the delay of reinforcement with the fixed interval of an operant schedule. The prediction of the transformed equation was that the logarithm of the rate of responding of an organism is a linear function of the logarithm of the number of trials or reinforcements, with the slope of this linear function being dependent upon the numerical value of the fixed interval.

Delay versus probability of reinforcement in discrimination learning.

Delay versus probability of reinforcement in discrimination learning.

Tests of an equivalence rule for fixed and variable reinforcer delays.

Tests of an equivalence rule for fixed and variable reinforcer delays.

EFFECTS ON PREFERENCE OF REINFORCEMENT DELAY, NUMBER OF REINFORCERS, AND TERMINAL‐LINK DURATION

Three experiments used concurrent-chains procedures to examine the effects of reinforcement delay, number of reinforcers, and terminal-link duration on preference. In Condition 30 of Experiment 1, food was delivered after 30 seconds in each 150-second terminal link, with four additional food deliveries occurring at 30-second intervals in one of the links. In Condition 5, food was delivered after 5 seconds in each 25-second terminal link, and the four additional reinforcers were delivered at 5-second intervals. Preferences for the multiple-food chain were greater in Condition 30. In Experiment 2, the terminal link(s) providing only one reinforcer terminated immediately after delivery of the reinforcer. Preferences for the multiple-food chain were smaller than in Experiment 1. In Condition 5 of Experiment 3, food was delivered after 5, 75, 100, 125, and 150 seconds in one 150-second link and after 5 seconds in the other. Condition 50 differed only in that the first (or only) reinforcer in each link was delivered after 50 seconds instead of after 5 seconds. Preferences for the multiple-food chain were greater in Condition 50. Results of Experiments 1 and 2 do not correspond to results obtained by Moore (1979).

Self-control and choice in humans: Effects of video game playing as a positive reinforcer

Participants chose between reinforcement schedules differing in delay of reinforcement (interval between a choice response and onset of a video game) and/or amount of reinforcement (duration of access to the game). Experiment 1 showed that immediate reinforcement was preferred to delayed reinforcement with amount of reinforcement held constant, and a large reinforcer was preferred to a small reinforcer when both were obtainable immediately. Imposing a delay before the large reinforcer produced a preference for the immediate, small reinforcer in 40% of participants. This suggested a limited degree of “impulsivity.” In Experiment 2, unequal delays were extended by equal intervals, the amounts being kept equal. Preference for the shorter delay decreased, an effect that presumably makes possible the “preference reversal” phenomenon in studies of self-control. Overall, the results demonstrate that video game playing can produce useful, systematic data when used as a positive reinforcer for choice behavior in humans.

Self‐Control and Responding during Reinforcement Delay

Annals of the New York Academy of SciencesVolume 423, Issue 1 p. 618-621 Self-Control and Responding during Reinforcement Delay TELMO E. PEÑA-CORREAL, TELMO E. PEÑA-CORREAL Department of Psychology, State University of New York at Stony Brook, Stony Brook, New York 11794Search for more papers by this authorA. W. LOGUE, A. W. LOGUE Department of Psychology, State University of New York at Stony Brook, Stony Brook, New York 11794Search for more papers by this author TELMO E. PEÑA-CORREAL, TELMO E. PEÑA-CORREAL Department of Psychology, State University of New York at Stony Brook, Stony Brook, New York 11794Search for more papers by this authorA. W. LOGUE, A. W. LOGUE Department of Psychology, State University of New York at Stony Brook, Stony Brook, New York 11794Search for more papers by this author First published: May 1984 https://doi.org/10.1111/j.1749-6632.1984.tb23473.xCitations: 1AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Citing Literature Volume423, Issue1Timing and Time PerceptionMay 1984Pages 618-621 RelatedInformation

Effects of task-irrelevant cues and reinforcement delay on choice-escape learning following inescapable shock: Evidence for a deficit in selective attention.

Effects of task-irrelevant cues and reinforcement delay on choice-escape learning following inescapable shock: Evidence for a deficit in selective attention.

CHOICE IN A SELF‐CONTROL PARADIGM: QUANTIFICATION OF EXPERIENCE‐BASED DIFFERENCES

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.

Effects of task-irrelevant cues and reinforcement delay on choice-escape learning following inescapable shock: Evidence for a deficit in selective attention.

Prior exposure to inescapable shock has been reported to interfere with choice-escape learning, but several investigators have failed to obtain this effect. A series of five experiments examined the conditions under which choice-escape learning in an automated Y-maze is impaired by pretreatment with inescapable shock. Inescapably shocked rats made more errors and responded more slowly than did controls only when shock termination was delayed and task-irrelevant cues were present during choice-escape training. These findings are discussed in terms of information processing and neurochemical consequences of exposure to inescapable shock.

Delay versus probability of reinforcement in discrimination learning.

Delay versus probability of reinforcement in discrimination learning.

Tests of an equivalence rule for fixed and variable reinforcer delays.

Tests of an equivalence rule for fixed and variable reinforcer delays.