Interval Schedule Of Reinforcement Research Articles

The effect of second-half reinforcer type on responding for sucrose in the first half of the session

The present study investigated whether the sucrose-reinforced lever pressing of rats in the first half of a 50-min session would be sensitive to upcoming food-pellet reinforcement in the second half. In Experiment 1, the type of reinforcer in the first half of the session was always liquid sucrose and type of reinforcer in the second half (liquid sucrose or food pellets) varied across conditions. Sucrose concentration varied across groups (1, 5, or 25%). Results showed that rates and patterns of responding for 1%, and sometimes for 5%, sucrose reinforcers in the first half of the session were higher and steeper, respectively, when food-pellet, rather than sucrose, reinforcement occurred in the second half. Responding for 25% sucrose was not similarly affected. Experiment 2 replicated the results of Experiment 1 using a within-subjects design. Although the present results represent induction (i.e. the opposite of contrast), they are consistent with some results on consummatory contrast. They also further demonstrate that responding on interval schedules of reinforcement can be altered prospectively. By doing so, however, they pose potential problems for current theories for why operant response rates change within the session.

Serial pattern learning of temporal intervals

The problem was to determine how rats adjust the times of their lever responses to repeating sequences of interfood intervals. In Experiment 1, rats were trained on an interval schedule of reinforcement with a 12-element Fleshler-Hoffman series with a mean of 60 sec; the order was as follows: ascending, random with repetition, random with replacement, random without replacement. In Experiment 2, rats were trained with a 10-element ascending or descending series (from 20 to 29 sec), and in a ramp procedure in which these intervals increased and then decreased repeatedly. In the ascending, descending, and ramp conditions (but not in the random conditions), postreinforcement pause (PRP) was a function of the interval. PRP was most highly correlated with an interval later in the series. Theories of conditioning and timing based on the averaging of past experience must be modified to account for such anticipatory behavior.

Intermittent rewards raise self-stimulation thresholds

Rats implanted with lateral hypothalamic electrodes pressed a lever to obtain 0.5 s bursts of pulses under the four combinations of fixed or variable, interval or ratio, schedules of reinforcement. Along with continuous reinforcement schedules, intervals of 1, 2, 5, and 10 s, or ratios of 2, 5, and 10 responses per stimulation were used in sessions wherein the frequency thresholds, defined as pulse frequencies that would support half-maximum rates of response, were estimated. Thresholds rose systematically under both ratio and interval schedules of reinforcement, and there was no difference between fixed and variable variations. When normalized data were plotted as a function of reinforcement density, thresholds from both interval and ratio schedules followed the same pattern, suggesting that the common factor was time between rewards. An increase in the current with a corresponding decrease in pulse frequency increased the rate at which time between rewards augmented thresholds.

Prolonged Behavioral Effects of in Utero Exposure to Lead or Methyl Mercury: Reduced Sensitivity to Changes in Reinforcement Contingencies during Behavioral Transitions and in Steady State

Postnatal exposure to lead or methyl mercury results in mental retardation, learning deficits, and other neurobehavioral effects in humans, and adverse consequences of prenatal exposure have been clearly documented with methyl mercury. To examine the developmental neurotoxicity of these metals, especially lead, concurrent schedules of food reinforcement were used to identify learning deficits in squirrel monkeys exposed during gestation to either methyl mercury or lead. Pregnant squirrel monkeys were administered methyl mercury (0.7 to 0.9 ppm in maternal blood) or lead (21 to 79 μg/dl in maternal blood) during the last half to two-thirds of gestation. At about 5-6 years of age, offspring were trained to lever press under concurrent schedules of reinforcement in which separate random interval reinforcement schedules operated independently on two levers. Reinforcement densities were varied such that 20 to 90% of the reinforcers were programmed to derive from the left lever (i.e., one lever was "richer" than the other). At steady state, the behavior of the controls was sensitive to reinforcement density and showed little lever bias, but the behavior of monkeys exposed to more than 40 μg/dl of lead and to methyl mercury was less sensitive to reinforcement rates and heavily biased. When relative reinforcement density on a lever changed, the unexposed animals′ response rates gradually shifted to the newly rich lever. The behavior of monkeys exposed to methyl mercury or more than 40 μg/dl of lead changed slowly, not at all, or in the wrong direction. Steady-state behavior of monkeys exposed to less than 40 μg/dl resembled controls, but acquisition progressed more slowly and required 2-4 times as many reinforcers to complete. These effects suggest a behavioral mechanism-insensitivity to changing reinforcement contingencies-by which learning deficits and behavioral changes associated with these metals might be related to toxicant exposure. Since maternal blood levels corresponded to those that could be experienced in occupational settings, the present data raise the possibility of fetal hazards associated with maternal lead exposures at levels tolerated in humans in occupational settings.

Describing Work Environments as Reinforcer Systems: Reinforcement Schedules versus Reinforcer Classes

Ten judges (representing two levels of expertise) rated 183 occupational titles (representing every major DOT category) on three rating dimensions: predictable versus nonpredictable (to infer the variable vs fixed reinforcement schedule), self versus nonself (to infer the ratio vs interval reinforcement schedule), and social versus nonsocial (to partition the nonself dimension further). These ratings were compared with scores on the same three dimensions based on Dictionary of Occupational Titles (DOT) ratings of interest and temperament requirements for the 183 occupations. Finally, groupings of occupations based on inferred reinforcement schedule were compared with occupational groupings based on independently obtained ratings of reinforcer classes (Occupational Reinforcer Patterns). Results showed that occupations can be characterized reliably in terms of ratings on the three rating dimensions. Such inferred reinforcement schedule ratings are consistent with inferred reinforcement schedules estimated from DOT ratings of interest and temperament requirements. Reinforcement schedule inferred from these ratings is related to, but different from, reinforcer class, as represented by Occupational Reinforcer Patterns.

Further characterization of adjunctive behavior generated by schedules of cocaine self-administration.

Rhesus monkeys (n = 5), surgically implanted with double-lumen catheters, were allowed to self-administer cocaine (0.1 or 0.3mg/kg/injection, i.v.) on one lever (COC lever) under several fixed-interval schedules of reinforcement. Responding on a second lever (SAL lever) delivered saline (i.v.) under a fixed-ratio 1 schedule. Responding on both levers was a bitonic function of interval value and cocaine dose. A variety of experimental conditions were examined to determine whether SAL lever responding could be considered to be adjunctive in nature. SAL lever responding did not change when saline injections were discontinued, suggesting that SAL lever responding was not maintained by interoceptive stimuli associated with the injection. Discontinuation of various exteroceptive stimulus changes that had occurred as a consequence of SAL lever responding also did not affect the frequency of SAL lever responding. However, when there was no stimulus change following a SAL response, response rates on that lever decreased by approximately 40-60% indicating that stimulus change played some role in the maintenance of the behavior. The introduction of change-over-delays (2-16 s) between responding on the SAL and COC levers had little or no effect on responding on the SAL lever, suggesting that SAL lever responding was not maintained by adventitious reinforcement by cocaine injections. SAL lever responding also occurred in these same monkeys when cocaine was available under fixed-time or variable-interval schedules of reinforcement. These results confirm that presentation of cocaine under interval schedules of reinforcement can generate substantial amounts of behavior (pressing the SAL lever) that is not necessary for obtaining the drug. Further, the results strongly suggest that the behavior can be classified as an adjunctive behavior that is similar to adjunctive behaviors generated by the intermittent presentation of other positive reinforcers.

A matching law analysis of the effects of dopamine receptor antagonists

Herrnstein's matching equation was used to analyze drug effects on performance in random interval reinforcement schedules. Pimozide caused effects compatible with both motor and motivational impairments, in a 5-component multiple schedule, a 3-schedule 3-day cycle (ALT-3), and a 2-schedule 2-day cycle (ALT-2). However, at low doses, both sulpiride and SCH-23390, tested in the ALT-3 and ALT-2 procedures, caused effects compatible with selective motivational impairments. In experiments using the non-multiple schedules, motivational effects increased during the course of the experimental session, under all three drugs. The interpretation of "motor" and "motivational" deficits in the ALT-2 procedure was validated by experiments in which the response-force and deprivation level were systematically varied. The results support the view that dopamine may be involved in the maintenance of rewarded behaviour, but not differentially implicate the D1 or the D2 receptor subtype.

Choice with Concurrent Interval Schedules of Reinforcement: The Effects of Different Timing Procedures

Using a Findley changeover procedure, two groups of rats were exposed to various concurrent variable-time (VT) schedules of milk presentation. The rules governing the operation of the timers for the stimulus events differed for the two groups with resulting differences in the overall degree of changeover responding but not with respect to time allocation measures, which in both cases consisted of undermatching. When an interrupted VT reinforcement timer was reset upon re-entry of that component, the rats tended to emit changeover responses almost exclusively to the more favorable component. When the VT reinforcement timers were left running regardless of which components the rats selected, changeover responses were significantly more frequent and equally distributed among both the more and less favorable components. In comparing the present results with those from a similar procedure in which shock was used (Deluty & Church, 1978), it is apparent that what is basic to choice in concurrent interval schedules is not matching or some mathematical variation of it. Instead, it is a tendency to optimize on the basis of the favorability of the stimulus events and the contingencies regarding how those events are timed and administered.

Schedules of reinforcement as regulators of dispersion patterns

An eight-rat eight-station operant conditioning arena was used to study the spatial structure and temporal stability of foraging dispersion patterns. Food was obtained by bar pressing as the population was exposed to an ascending series of the fixed and variable aspects of ratio and interval schedules of reinforcement. Dispersion patterns, defined by the number of rats simultaneously foraging at each of the eight stations, and the temporal changes in these patterns, were the dependent variables. Both variables exhibited a unique relationship to each schedule type and value. The absence of such relationships when either food supply or response costs were examined suggests that these factors were not the determinants of spatio-temporal structure. An account is provided of how schedules may interact with behavioral foraging chains to explain dispersion patterns.

A comparison of the effects of amphetamine, apomorphine and white noise on response switching in the rat

The effects of d-amphetamine, apomorphine and white noise on response switching in the rat were examined using a schedule of reinforcement which resulted in the subjects displaying a range of different probabilities of switching. The procedure was analogous to the use of a fixed interval schedule of reinforcement for examining the rate-dependent effects of drugs. d-Amphetamine (0.4-4.0 mg/kg) increased response switching in a manner dependent both upon the dose of drug and the baseline probability of switching. Apomorphine (0.01-0.3 mg/kg) increased switching in a manner which depended upon dose but which was independent of the baseline probability of switching. Neither drug increased response rate, although both drugs reduced response rate at the highest doses. In contrast, continuous white noise (85-105 dB) increased response rate without affecting switching. The results indicate that different activating stimuli may have qualitatively different effects on behaviour.

RESPONSE—REINFORCER DEPENDENCY LOCATION IN INTERVAL SCHEDULES OF REINFORCEMENT

In five experiments we studied the effects on pigeons' key pecking of the location of four or more successive response-dependent reinforcers imbedded in a schedule arranging otherwise response-independent reinforcers. In Experiment 1, high local response rates early in the session were extended farther into the session as the number of response-dependent reinforcers at the beginning of the session increased. A block of four successive response-dependent reinforcers then was scheduled at the beginning, middle, or end of the session (Experiment 2) resulting in higher local response rates at those times in the session when the response-dependent reinforcers were arranged. When placed in random locations in successive sessions (Experiment 3), uniform local rates occurred throughout the session. In Experiments 1, 2, and 3, delivery of the remaining response-independent reinforcers was precluded until the response-dependent reinforcers were collected. Experiment 4 was similar to Experiments 1 and 2, except that all response-independent reinforcers occurred irrespective of whether the response-dependent reinforcers had been collected. This yielded results similar to those obtained in the first two experiments. In Experiment 5, responding early in the session had no consequence other than allowing access to the schedule of response-independent food delivery. As in the first experiment, local rates generally were higher early in the session. The results indicate that the location of response-reinforcer dependencies precisely control behavior and that such effects often are not captured by descriptions of behavior in terms of overall response rates.

Variability of force and interresponse time under random interval reinforcement schedules

Response variability was examined under five random interval schedules with minimum mean interreinforcement intervals of 15, 30, 60, 120, and 240 sec. Distributions of response force-time integrals showed no systematic variability changes with increases in reinforcement intermittency. Corresponding distributions of interresponse times revealed increases in variability accompanying increases in reinforcement intermittency. The relationship of variability to these two measures, and the potential effect of behavior outside the experimentally measured response class, on recorded variability within the class, are considered.

Responding on a multiple schedule of reinforcement as a function of different septal lesions in the rat

The response rates of rats with either control operations or lateral, medial, or complete septal lesions were compared on a multiple fixed-interval (FI)/random-interval (RI) 40-sec schedule of reinforcement in which the rat was required to respond on a different lever for each alternating schedule component. Rats with lateral septal lesions responded at higher rates during the first 8-sec period of the FI component and had poorer response patterning than rats with control operations on the FI 40-sec reinforcement schedule. On the other hand, rats with medial septal lesions responded at higher rates than controls during the RI 40-sec component. Rats with complete septal lesions switched levers more than did controls before the ongoing schedule was completed. The results suggest that the lateral and medial septum have different functional roles mediating responding for food on interval schedules of reinforcement.

Differential conditionability: Reinforcing grooming in golden hamsters

Differential conditionability is the empirical finding that not all responses are equally amenable to the same conditioning paradigm. One phenomenon associated with the conditioning of grooming behavior (a difficult-to-condition response) is a decrease in its average duration when followed by food reinforcement. The first experiment investigated this phenomenon by reinforcing golden hamsters (Mesocricetus auratus) with food for grooming or open rearing (a readily conditionable response) under three duration-dependent reinforcement schedules. The obtained data showed that different densities of food delivery had no differential effects on the average durations of grooming responses, indicating that the decreases were not the result of reinforcement-produced interruption. In the second experiment, golden hamsters were reinforced with food for grooming or for open rearing or received free food, under three interval reinforcement schedules. This experiment demonstrated that decreases in the average duration of grooming are independent of grooming behavior’s resistance to conditioning. Furthermore, although duration-dependent reinforcement schedules are largely ineffective in conditioning grooming behavior, interval schedules are shown to be quite effective in increasing rates of grooming.

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pulse-frequency analysis of septal and hypothalamic reinforcement effects

Electrical stimulation of the septal or the lateral hypothalamic region was used to reinforce responding of rats. Holding constant the number, duration, and intensity of the pulses in each stimulation train, the pulse frequency was varied. Septal self-stimulation was found to be relatively invariant when every response was reinforced but was sensitive to frequency manipulations when variable interval reinforcement schedules were used. Septal self-stimulators responded most rapidly for low pulse frequencies (50/sec), while hypothalamic rats responded most rapidly for higher frequencies (250–400/sec). The frequency functions are not well accounted for by refractory period data, but do seem to represent fundamental differences in the neural substrates of reinforcement for these loci.

Effect of Time-Slowing Suggestions on Rate of Emission of an Operant Response

Five suggestible female subjects acquired an operant response conditioned on a variable interval schedule of reinforcement. Each subject was then hypnotized and performed the response during 6-min. alternating baseline and experimental phases in an Abab withdrawal design. Experimental phases were preceded by administration of time-slowing suggestions. Return to baseline condition was accomplished via presentation of a time-normalization suggestion. Four of the subjects demonstrated noticeable decreases in operant response rate during time-slowed phases as compared to the preceding baseline phases. Subjects decreased their response rates despite a perceived response cost in doing so. Subjective reports indicated that all subjects experienced slowed time passage during experimental phases although subjective experiences varied greatly between individuals. An attempt is made to explain the differences between subjects in their overt and covert responses to time-slowing suggestions.

Mutual antagonism of behavioral effects of TRH and thiobarbiturate on an operant task in rhesus monkeys

Similar excitatory or depressant response rate dependent effects on monkeys responding on a variable interval reinforcement schedule were observed following intravenous administration of either thyrotropin releasing hormone (TRH) or thiobarbiturate. However, these agents were mutally antagonistic when given together even though the response rate altering effects of each agent were in the same direction. These findings establish an additional behavioral effect of exogenously administered TRH in primates and suggest that barbiturates might alter behavior in part through an interaction with brain TRH receptive mechanisms.

Effects of a Limited Hold on Changeovers Maintained by Concurrent Interval Schedules of Reinforcement

In order to investigate the effect of reinforcers occurring immediately after a changeover on the changeover behavior of pigeons, limited holds of various durations were placed on one of two concurrently available random- interval schedules. It was found that the limited hold generally had the effect of reducing the relative number of responses and the relative amount of time allocated to the schedule with the limited hold, and that this effect was primarily the result of an increased latency of changeover into the limited hold schedule. This tendency could be augmented by increasing the duration of a changeover delay from 2 sec. to 8 sec. Despite the fluctuations in behavior induced by these manipulations, response and time matching to relative rates of reinforcement prevailed. Implications for various hypotheses regarding the fundamental nature of the behavior maintained by concurrent interval schedules of reinforcement were discussed.

Response-Produced Tone and Shock as Added Stimuli in an Interval Schedule of Reinforcement

Summary Three pigeons were used as Ss in a study designed to examine the effects of single and multiple 6-second response-produced-brief-stimulus (RPBS) periods by programming the RPBS period as an added stimulus in an interval schedule of reinforcement. A tone and electric-shock were employed as the brief stimuli in order to compare differences in discriminative control acquired by an auditory and cutaneous stimulus. The tone and electricshock stimuli identified separate components of a two-ply multiple schedule. Response patterns and rates were found to vary according to the placement of the brief stimulus period in the middle, at the end, or at both locations in the schedule of reinforcement. Responses engendered by the tone and electric-shock stimuli were shown to be very similar to each other and the patterns observed with the use of visual stimuli.