Delay Of Reward Research Articles

Regarding Roberts's reported failure to obtain visual discrimination learning with delayed reward

Roberts's (1976) results should not be considered a failure to replicate the experiment by Lett (1974) demonstrating visual discrimination learning with a 1-min delay of reward. Although performance in Roberts's experiment was poor, the rats subjected to Lett's exact procedure showed learning by Roberts's own criterion of a significant-blocks effect. Moreover, by another criterion, the rats subjected by Roberts to three variants of Lett's procedure also exhibited learning. Thus, Roberts's results are consistent with Lett's.

Still no evidence for visual discrimination learning: A reply to Lett

Neither of the points made by Lett (1977) regarding the data from my experiments ( Roberts, 1976) can be taken as evidence of visual discrimination learning with a 1-min delay of reward. If it is concluded that the Lett Group of my second experiment improved over blocks of trials but the other three groups tested in my experiments did not, this can lead only to the conclusion that rats in my Lett Group and in Lett's original experiment ( Lett, 1974) improved on the basis of odor cues and not associations between chamber brightness and reward. The significantly higher frequency of initial choices of the black chamber found for black-rewarded animals over white-rewarded animals is shown to result from differential rates of stimulus satiation for animals whose initial chamber preferences agree or disagree with their designated S+.

Positive Contrast following a Shift from Partial Punishment to Continuous Reinforcement

The primary emphasis in the area of incentive contrast has centered around the effects of shifts in reward magnitude. However, with increasing evidence of the broad nature of contrast phenomena, investigators have explored the effect under other experimental contingencies. One procedure of potential theoretical interest deals with shifts in percentage or frequency of reward. Lehr (1974) and Shanab, Birnbaum, and Cavallaro (1974) have recently found positive contrast following shifts from partial reinforcement to continuous reinforcement schedules. The purpose of the present study was to expand the scope of research on percentage shift to include an examination of preshift incentive level by introducing punishment as a partial determinant of incentive. Twenty-four albino rats, 100 days old, were initially given 40 trials of continuous reinforcement (10 .045-gm. pellets on each trial) in a straight, 1.2-m., wooden alley (4 trials/day). All subjects received a constant 20-sec. delay of reward throughout the experiment to hold down running speed and allow for the emergence of positive contrast. The alley was 10 cm wide, 13.75 cm high, and covered by a hinged Plexiglas top. During a subsequent 40-trial preshift-phase 3 conditions (8 subjects/group) experienced different reward schedules. As a control, continuous reinforced subjects received 10 .045-gm. Noyes pellets per trial throughout the experiment (4 trials/day). Subjects partially reinforced received 2 reinforced and 2 nonreinforced trials each day and subjects partially punished received 2 rewarded and 2 punished trials each day (punishment was superimposed on nonreinforcement). During this phase, the alley was modified such that bIack and white vertical stripes were placed over the grey interior and a grid floor was installed in the delay and goal sections of the alley. A Lehigh Valley, Model 1531 shock unit was employed to administer a brief (.5-sec. duration) electric shock at 2.5 mA. Following preshift training, in order to avoid the possible contaminating influence of conditioned fear, the alley was reconverted to grey interior and wooden floor and all subjects shifted to delayed continuous reinforcement (10 pellets on each trial). This postshift phase lasted 32 trials (4 trials/day). A 3 groups X 8 blocks of 4 trials repeated-measures analysis of variance was performed on total speeds during postshift. The finding of greatest interest was a significant interaction of groups X blocks (K,,I,, = 1.82, p < .05). Newman-Keuls post hoc procedure indicated that a significant positive contrast effect emerged on Blodc 4; the condition of partial punishment was superior to those of partial and continuous reinforcement, which did not differ (p < .05). Since parcial punishment would have been expected to have had a lower preshift-incentive level than partial reinforcement, the present experiment supporcs the position that in studies of percentage shift, preshift-incentive level determines the magnitude of contrast in postshift.

Preference for food and water in rats as a function of delay of reward

In Experiment I, rats which were both hungry and thirsty were given a choice between a food reward and a water reward. The animals preferred food to water when the reward was delivered immediately, but preferred water to food when a 30-sec delay was imposed in the goalbox before the reward was received. Experiment II replicated the results of the first experiment and showed, in addition, that when the delay was imposed in a separate delay chamber devoid of differential goalbox cues, subjects preferred food to water, similar to the immediate group. The results were discussed in terms of an incentive value process and a competing response hypothesis.

Failure to replicate visual discrimination learning with a 1-min delay of reward

Two experiments were carried out in an attempt to replicate Lett's (1974) demonstration of visual discrimination learning with a 1-min delay of reward. In Experiment 1, controls for olfactory cues were introduced by testing animals in different orders from day to day and by reversing the positions of the discrimination chambers within days; also, secondary reinforcement effects were precluded by the use of opaque rather than transparent doors to the discrimination chambers. No evidence of learning was found either in a group of rats returned to the home cage during the delay or in a group left in the apparatus during the delay. It was hypothesized that the failure to find improvement in Experiment 1 might have resulted from either the controls for odor cues or from a lack of memory retrieval cues provided by the apparatus. In Experiment 2, both of these hypotheses were tested and neither was supported. The Lett group in Experiment 2 was run according to the exact procedures used by Lett and still no learning was demonstrated. It was concluded that further demonstrations of visual discrimination learning with a 1-min delay of reward will be necessary for the acceptance of the validity of this phenomenon.

Delay of gratification in children: The effects of training under fixed, decreasing and increasing delay of reward

The effect of fixed, gradually decreasing, or increasing delay of reward in discimination learning on later delay of gratification was investigated. In discrimination training, employing a correction procedure, a candy reward was delivered either after 0, 10, 20, 40 or 60 sec fixed delay; or after 60 sec in the first block of trials and decreased in successive block; or reward was immediate in the first block of trials and delay was gradually increased to 60 sec. In the delay of gratification tests, subjects could press a button immediately to receive a small reward (one candy or a cheap toy) or delay pressing and receive an increasingly larger reward (more candy or a better toy). Learning was not significantly affected by either fixed or decreasing delays. Increasing delays resulted in faster learning than decreasing delays. The increasing delay group demonstrated superior delay of gratification on both tests. Fixed delay groups did not differ significantly among themselves, nor from the decreasing delay group. The effectiveness of exposure to increasing delays in facilitating delay of gratification was interpreted as due to either the acquisition of coping responses or the extinction of frustration.

Expectation of rewards and emotional behavior in monkeys

From previous theoretical proposals about emotional processes, it was hypothesized that the lack of, or delay of, expected reinforcement for a previously rewarded ongoing plan of action would be associated with emotional behavior. This hypothesis was confirmed by altering the reward contingencies during the differential reinforcement of low rates (DRL) schedule. The DRL requires that the animal wait for a prescribed time interval before responding in order to be reinforced; premature responses are non-rewarded and postpone the availability of a reward until the next prescribed time interval has elapsed. Using each monkey as his own control, two experimental approaches were employed; a mid-point change in the length of the DRL interval or availability of rewards, and discrete manipulations of reward contingencies (no reward or extra rewards) for approriately timed delayed responses after a train of prior consecutive rewards had been established. For the mid-point change experiment, lengthening the DRL interval was associated with sustained emotional behavior, such as bursts of responding and vigorous activity. For the discrete reward manipulations, the lack of expected reinforcement for delayed timing responses that ordinarily would have been rewarded was associated with emotional behavior. The DRL schedule is useful for studying emotional behavior associated with the lack of or delay of rewards. This approach may complement experimental paradigms for studying emotion in animals that employ the use of noxious or painful stimuli. The lack of expected rewards may provide a relevant model for investigating certain types of emotions, such as frustration and anger.

“Thinking ahead” in rat discrimination learning

If rats chose S+ in a brightness discrimination in a T-maze, they experienced, on that run and over four forced runs to S+ which followed, a pattern of reinforcement in which quantity of reward in the goalbox increased from 0 to 14 food pellets, decreased from 14 to 0 food pellets, or varied randomly. If the rats erred and chose S−, reinforcement was withheld, and they were forced a second time to 0 reward in the S− goalbox. The results indicate that rats readily learn the brightness discrimination under these conditions, the animals exposed to the sequentially increasing pattern learning somewhat slower than the others. This was true in spite of substantial delay of reward. Theoretical accounts based on perseverative inhibitory or facilitatory tendencies, or upon frustration, fail to describe the data accurately. Instead, analysis of the data shows that the animals were accurately anticipating the quantity of reward to be obtained on each run, running fast for large quantities and slowly for small.

The effects of delay of reward on negative contrast effects associated with reductions in reward magnitude

The results of three runway experiments in which rats received a reduction in the magnitude of delayed vs. immediate reward are reported. The results indicated that the negative contrast associated with a reduction in reward amount is more pronounced when all rewards are delayed. In the past three decades. considerable theoreticai attention has focused on the behavioral effects of changes in magnitude of reward in instrumental conditioning. Until quite recently. it appeared that \\hile decreases in reward magnitude produced contrast effects. comparable increases in reward did not (d. Black. 1968: Dunham. 1968). Thus. rats shifted from large to small reward typically show negative contrast effects (NCEs). depressed perfor­ mancc to small as compared with that of subjects trained and maintained on small reward. \\hile subjects shifted from small to large reward do not display posItI\e contrast effects (PCEs). facilitation of performance to large reward as compared \\ith that of subjects maintained on large re\\arci. Recent studies. howe\er. ha\'e clearly shown that PCEs attend increases in reward amount if all re\\ards are delayed follo\\'ing the instrumental response (Mellgren. 1972). While it is clear that reward delay interacts with increases in reward magnitude such that PCEs are larger and more reliable under delayed as compared \\ith immediate reward conditions. the theoretical basis of this interactive remains largely unexplored. Most frequently. the interactive has been attributed to the operation of a physiological or psychological effect on the performance of unshifted large reward (contro\) subjects (cf. BO\\er. 1961: Mellgren. 1972; Shanab & Cm·allaro. 1975). According to the ceiling hypothesis, PCEs are not observed under conditions of immediate reward because control subjects have achieved optimal response le\'els: lowering the performance level of control su bjects. through delay of reinforcement. provides a context in which the facilitative effects of increases in re\\ard magnitude may be seen. It should be clear that such a hypothesis does not attempt to

Extinction persistence in the rat following brief training with constant or partial delay of reward

Forty-eight rats were trained in a runway under three acquisition schedules: constant immediate reward, partial delay of reward, and constantly delayed reward. Delay occurred in the goalbox for a duration of 20 sec. Acquisition trials were given at one trial a day for 10 days. The constant delay group ran somewhat slower in acquisition than the other two groups. During extinction there were two treatment levels; constant confinement and no confinement. Confinement occurred in the goalbox for 20 sec, and one extinction trial was given each day for 24 days. The animals which were confined during extinction ran significantly slower than the nonconfined animals. The extinction effects of the acquisition schedules included a modest partial delay extinction effect but no evidence that constant delay caused an increase in persistence.

Positive discrimination contrast with delay of reward or low drive

In the present study, six groups of 10 albino rats each were given differential conditioning training in parallel alleys differing only in brightness. The results showed a positive S+ contrast effect when 20-sec delay of reinforcement or lowered drive (12-h deprivation) were used to lower responding, but a negative S+ contrast effect when subjects received both immediate reinforcement and 23-h deprivation. No present theory can adequately explain both effects.

Contrast effects as a function of shifts in delay of water reward

Following stable performance in the acquisition (preshift) phase, subjects receiving either 0, 8, or 16 sec delay of water reward were subdivided and either continued on the preshift delay, or shifted to the other preshift delay intervals. The postshift performance was examined for evidence of positive and negative contrast effects. It was found that delay produced significant differences in acquisition such that, the shorter the delay interval, the faster the running speed. This effect was also found in the second phase in which the same relationship held between length of delay and speed, regardless of earlier acquisition experience. Although a graphical, albeit statistically nonsignificant, PCE was observed in all sections of the runway, no NCE was found. The results were discussed in terms of several theories of behavior.

Transfer between downshift in reward magnitude and continuous delay of reward

Two separate experiments were conducted to investigate transfer of persistence between delay and a downshift in reward magnitude. In the first experiment, experimental rats were initially downshifted in reward magnitude and later tested for persistence to continuous delay of large reward. It was found that these rats were more persistent to the effects of delay than control rats which did not receive prior experience with a downshift in reward magnitude. In the second experiment, experimental rats were first trained to receive large reward under delayed conditions and then tested for persistence to a downshift in reward magnitude. Compared to control rats which received no prior experience with delay, the experimental rats showed a significantly smaller negative contrast effect. The results were interpreted as supporting Amsel's theory of persistence as well as Capaldi's recent interpretation of contrast effects.

Positive contrast obtained in rats following a shift in schedule, delay, and magnitude of reward

Sixty-four rats were used in a 2 by 2 by 2 factorial design (Phase 1) which combined two levels of delay schedule (partial vs. continuous), two levels of magnitude of reward (1 vs. 22 pellets), and two levels of delay of reward (10 vs. 30 sec). On each trial in Phase 2, each subject received 22 pellets after a 10-sec delay interval. A significant positive contrast effect was obtained when all three variables of magnitude, delay, and schedule were upshifted. Similarly, positive contrast was obtained based on a shift in delay and schedule, magnitude and schedule, and magnitude and delay. Moreover, positive contrast was obtained based on either an upward shift in schedule or magnitude. A simple decrease in delay of reward did not produce positive contrast. The results were viewed as favoring relative rather than absolute theories of reinforcement. However, a modified sequential (capaldian) interpretation of positive contrast was also offered.

Long delay learning in the T-maze

Rats were trained to go to one side of a T-maze with delays of reward lasting 1, 20, or 60 min in Expt 1 and 1 or 60 min in Expt 2. Mediation by secondary reward was prevented by administering the same delay treatment regardless of whether the response was correct or incorrect: after a response, the rat was removed from the choice alley and placed in its home cage to spend the delay. Feedback for the response was given in the startbox after the delay interval ended. The rats learned and there were no significant differences in performance among groups trained with different delays. These results had been expected on the basis of Revusky's (1971) hypothesis that removal of the rat from the learning situation to spend the delay elsewhere facilitates long delay learning by reducing associative interference. In Expt 3, this notion was tested explicitly by varying the amount of a 2-min delay to be spent in the experimental situation. Different groups of rats were left in the choice alley after the response for 0, 15, or 60 sec; then the rats were removed to spend the remainder of the 2-min delay in the home cage As predicted, the level of performance decreased as the length of time in the choice alley was increased.

Resistance to Extinction of the Partial Delay of Reward Concerning the Few-trial PRE

The partial reinforcement effect (PRE) which is obtained after a small number of training trials can't be easily explained by the foregoing theories, since in these the crucial mechanisms of the PRE presuppose rather extensive trials. Thus a PRE-mechanism that is independent of the number of trials shoud be hypothesized. Capaldi interpreted the PRE in terms of the generalization of conditioned stimulus-aftereffect. The present study was conducted to determine whether this aftereffect-mechanism is independent of the number of training trials, employing a situation in which reward was partially delayed.In Exp. I, extended training (48 trials) was used with 4 groups of rats;continuous reinforcement (CR), partial reinforcement (PR), 5 sec. partial delay of reinforcement (PD5), and 40 sec. partial delay of reinforcement (PD40). In Exp. II, the same design was used, but limited training (5 trials) was given. In each experiment, the subjects received a training and an extinction in a straight alley.In two experiment, the PRE was obtained and the resistance to extinction increased as a function of the delay time of reinforcement (Fig. 1 and Fig. 3). The results of Exp. II show that CAPALDI'S prediction concerning the extinction process following the partial delay of reward is applicable even to the condition of fewtrial training, and combined with those of Exp. I, suggest that the generalization of stimulus-aftereffect was independent of the number of training trials. The applicability of CAPALDI'S stimulus-aftereffect hypothesis to the few-trial PRE was proposed.

Partial reward and positive contrast effects

In two experiments, groups of rats were given 50% or 100% of either a large (eight pellets) or small (one pellet) reward. All rats were then given 100% large reward. A 20-sec delay of reward was used throughout. Positive contrast effects were obtained in both experiments and schedule of reward did not interact with this effect.

Visual discrimination learning with a 1-min delay of reward

Eight rats were successfully trained in a black-white discrimination with a 1-min delay of reward. The procedure was unusual in that the rat spent the delay outside the apparatus in its home cage. Immediately after the rat responded, whether correctly or incorrectly, it was removed from the choice compartment and placed in its home cage. When the delay ended, it was returned to the startbox. If the preceding response had been correct, the rat received a reward of sugar water; otherwise, it was allowed to make another choice response. Mediation by external cues was excluded because there was no difference in the way the rats were treated after a correct or an incorrect response until the delay interval ended. Mediation by proprioceptive stimuli was excluded because position was an irrelevant cue.

Simultaneous and successive contrast with delay of reward

In Experiment I, rats showed no evidence of a successive negative contrast effect (NCE) when shifted from immediate reward to a 15-sec delay of reward. Experiment II provided a direct comparison of NCE in both successive and simultaneous paradigms. As in Experiment I, there was no evidence of a successive NCE, but a reliable simultaneous NCE was observed. These results add support for the view that simultaneous and successive NCEs are not due to a single, common process.

Transfer between nonreward and delay of reward following minimal acquisition training

Four groups of rats were given six acquisition trials under continuous reward, continuous delay of reward, partial reward, or partial delay of reward, following which all Ss received continuous delay. It was found that the partial reward and the partial delay of reward groups were significantly more persistent during the shift phase than the continuous reward group. No differences were found over trials between either the two partial groups or between each one and the continuous delay group.