Variable-interval Schedules Of Reinforcement Research Articles

A comparison of peripheral and central effects of CCK8 on water-reinforced operant responding

Systemic administration of cholecystokinin-octapeptide (CCK8) suppresses operant responding in water deprived rats, but it is unclear wether this effect is centrally or peripherally mediated. Rats were trained to press a lever for water in an operant conditioning chamber under a variable-interval schedule of reinforcement. After response rate stabilized injections were administered, and response suppression was measured by comparing injection response rate to baseline rate. Intracerebroventricular injections of CCK8 reduced lever pressing at relatively high doses (20 and 50 μg/rat). But in a direct comparison, the same dose of CCK8 (20 μg/rat) given intraperitoneally reduced responding significantly more than when given into the lateral ventricle. The suppressive effects of CCK8 (30 and 300 μg/kg i.p.) were significantly reduced by complete abdominal vagotomy. The effects of CCK8 (30 μg/kg i.p.) were blocked by pretreatment with the specific competitive CCK8 antagonist dibutyryl cyclic GMP (70 and 140 mg/kg i.p.), but not by the acetylcholine antagonist atropine (0.1 to 10 mg/kg i.p.). These data suggest that suppression by CCK8 of operant lever pressing in water-deprived rats is primarily mediated by vagal afferent fibers.

Cholinergic mechanisms and working memory in the rat: Effects of scopolamine on performance in two aversively motivated spatial tasks

Medium spiny neurons (MSNs) constitute 95% of neurons in the dorsal striatum subdivided into direct (striatonigral) and indirect (striatopallidal) pathways. Whereas D1 and D2 receptors and several neuropeptides, including dynorphin and enkephalin, are differentially expressed in these neurons, 5-hydroxytryptamine 6 receptors (5-HT6) are expressed in both pathways. Previous results demonstrate that concurrent 5-HT6 receptor overexpression in MSNs of both pathways in the dorsomedial striatum (DMS) interferes with instrumental learning and that 5-HT6 overexpression in the dorsolateral striatum (DLS) relieves rats from inflexible habitual behaviors. We hypothesized that 5-HT6 receptor-mediated co-activation of both pathways interferes with the differential activation/inhibition of direct/indirect pathways by dopamine. To test this idea, we cloned novel viral vectors to selectively overexpress 5-HT6 receptors in direct or indirect pathway MSNs to deconstruct their role in modulating instrumental learning and habitual responding. We found that increasing 5-HT6 receptor expression in either direct or indirect pathway MSNs of the posterior DMS selectively enhanced or impaired initial acquisition of a discrete instrumental learning task respectively, though all rats were ultimately able to learn the task. In a separate set of experiments, 5-HT6 receptor overexpression in indirect pathway MSNs of the DLS facilitated behavioral flexibility in rats overtrained on a repetitive pressing task using a variable interval schedule of reinforcement, during an omission contingency training session and subsequent probe testing. Together these findings further the notion that 5-HT6 signaling causes balanced activation of opposing MSN pathways by serotonin in sub-regions of the dorsal striatum allowing for more reflective modalities of behavior.

Sensitivity to reinforcement in successive and delayed discriminations

Pigeons' responses were recorded in successive 15-s subintervals of 60-s components of several multiple variable-interval schedules of food reinforcement. In the standard multiple schedule or successive discrimination, discriminative stimuli were present throughout each component. In the delayed discrimination or memory procedure, red or green stimuli were present in the first 15 s of components and were followed by a white stimulus for the remainder of both components. Ratios of responses in the first 15 s of the two components, where discriminative stimuli were present, were sensitive to ratios of reinforcers obtained in the two components, to the same extent in both multiple and memory procedures. In both procedures, sensitivity to reinforcement decreased systematically over component subintervals, but to a greater extent in the memory procedure where discriminative stimuli were absent. The reduction in sensitivity with time since presentation of prior discriminative stimuli in the memory procedure was therefore influenced by two main factors: delayed stimulus control by the discriminative stimuli presented earlier in the component, and a decrease in sensitivity to reinforcement with increasing time since component alternation.

Concurrent Reinforcement of Topographically Different Responses and the Matching Relation

Three rats responded under concurrent variable-interval schedules of food reinforcement. One schedule component provided reinforcement for the manipulation of an omnidirectional rod; the other schedule component provided reinforcement for pressing a lever. The same force was required to execute each response. Reinforcement rates provided by the schedules were varied over experimental conditions. Allocation of both responses and time between schedules undermatched the ratios of reinforcements obtained from the schedules for each rat. All rats displayed bias in favor of the lever with both response and time allocation, but no consistent difference in the magnitude of bias with response-based versus time-based measures of choice was observed. Previous research on choice between topographically different responses revealed substantially greater bias in response-based measures of choice than in time-based measures. However, response topography has been confounded with response force in previous research. Findings of the present study, in which force requirements for the two responses were equal, indicate that differences in response topography do not necessarily produce differences between response-based and time-based measures of bias. Rather, these findings suggest that differences in measures of bias may result from different force requirements of the responses.

A note on the measurement of stimulus discriminability in conditional discriminations

In a conditional discrimination, pigeons’ responses on a right key were reinforced in the presence of a 0° line orientation, and responses on a left key were reinforced in the presence of any of five other orientations, 15° through 75°. Variable-interval schedules of reinforcement for right and left responses were changed over five experimental conditions. Values for a measure of discriminability of 0° from other orientations increased as orientation difference increased. Sensitivity of the choice between right and left in the different orientations, to reinforcers for right and left responses, changed with orientation difference. The effect of reinforcers in one component of a conditional discrimination on responding in another may therefore be modulated by the discriminability of the stimuli signaling the components.

The effect of drugs on the acquisition of stimulus control in a conditioned suppression procedure.

Rats were trained to press a lever under a variable-interval (VI) schedule of water reinforcement. After stable responding had developed, a 4.5-KHz tone (CS) was conditioned classically to a 2.5-mA electric shock (US) in groups of animals which had been given various psychoactive drugs or saline. Twenty-four hours later, a stimulus generalization test was conducted in the absence of drug; during this session, tones that varied in frequency around 4.5 KHz were presented while the animals were responding under the VI schedule. In animals conditioned under saline, all tones (non-differentially) suppressed responding which, however, recovered gradually over time. This suppressive effect was eliminated by lysergic acid diethylamide (LSD; 0.2 and 0.32 mg/kg), cocaine (20 mg/kg), diazepam (2.5 mg/kg), lisuride (0.08 mg/kg), mescaline (20 mg/kg) and 5-methoxy-N,N-dimethyltryptamine (4 mg/kg), and was attenuated by amphetamine (4 mg/kg), pentobarbital (15 mg/kg) and morphine (4 mg/kg). Atropine (10 mg/kg), scopolamine (1 mg/kg), clonazepam (0.5 mg/kg), and chlorpromazine (4 mg/kg) did not alter the suppressive effect of the tone. The serotonin antagonist BC-105 (6 mg/kg) reversed the effect of 0.2 mg/kg of LSD. These results suggest (1) that drug-induced stimuli may "overshadow" other (e.g., external) stimuli during classical conditioning and, (2) that drugs might affect behavior by altering processes (stimulus control or others) that do not simultaneously involve response or motor control.

The effect of pentobarbitone on variable-interval performance: analysis in terms of Herrnstein's equation.

Six rats were exposed to a series of variable-interval schedules of reinforcement, each schedule specifying a different reinforcement frequency. The effects of two doses of pentobarbitone (24.16 and 40.27 mumol/kg) upon performance maintained under each schedule were examined. The data were analyzed in terms of the goodness of fit of Herrnstein's (1970) equation. Under control conditions (no injection or injection of vehicle alone) response rate was an increasing negatively accelerated function of reinforcement frequency, the data obtained from each rat conforming closely to Herrnstein's equation. In the presence of pentobarbitone there was a reduction in the value of Rmax (the theoretical maximum response rate) in all six rats. The effect of pentobarbitone upon the value of KH (the reinforcement frequency needed to obtain the half-maximal response rate) differed between animals, the value being increased in some rats and decreased in others. This was shown to reflect a systematic relationship between the baseline value of KH and the change in KH produced by pentobarbitone, high baseline values being associated with reductions, and low baseline values with elevations, in the presence of the drug. The data were also analyzed using conventional rate-dependency analysis. In some animals there was a tendency for lower response rates to be suppressed to a greater degree than higher rates, while the reverse was true in other animals. In general, rate-dependency functions provided poor descriptions of the data obtained from individual subjects.

Clozapine blocks disruptive and discriminative stimulus effects of quipazine

Clozapine was tested in two serotonin-dependent behavioral measures. One group of rats was trained to discriminate the serotonin agonist, quipazine, from saline in a two-lever operant choice task. Pretreatment with clozapine completely blocked the discrimination of quipazine. Another group of rats was trained to bar press for milk on a variable interval schedule of reinforcement. Quipazine decreased the response rate in these animals and pretreatment with clozapine completely reversed this effect. Thus, clozapine acted as a serotonin antagonist in both measures of serotonin function.

The effects of the β-carboline FG 7142, on intracranial self-stimulation in the rat

The effects of FG 7142 were examined, alone and in combination with chlordiazepoxide, on self-stimulation of the mid-lateral hypothalamus. Rewarding stimuli were delivered according to a 10-sec variable-interval schedule of reinforcement. FG 7142 (1-20 mg/kg) produced a dose-related depression in responding, and chlordiazepoxide (5 mg/kg) enhanced it. When these two drugs were given together, response rates did not differ significantly from control rates.

STIMULUS AND REINFORCER RELATIVITY IN MULTIPLE SCHEDULES: LOCAL AND DIMENSIONAL EFFECTS ON SENSITIVITY TO REINFORCEMENT

Pigeons' responses in two successive components of multiple schedules were reinforced according to variable-interval schedules of reinforcement that varied over five different conditions. Within each session of all conditions, line orientations of 0 degrees , 30 degrees , or 45 degrees in Component 1 alternated with orientations of 45 degrees , 60 degrees , or 90 degrees in Component 2. Response rates were recorded in three successive subintervals of each component. Ratios were taken between the response rate in each Component 1 line orientation and the response rate in each Component 2 orientation. These ratios were found to be power functions of the corresponding ratios of obtained reinforcement rates. Sensitivity of response ratios to changes in reinforcer ratios, given by the value of the exponent of the power function, increased systematically with increasing disparity between the dimensional values of orientation stimuli. In addition, sensitivity decreased systematically over successive subintervals of components, that is, with increasing time since component alternation. Dimensional and local (subinterval) effects interacted in that sensitivity increased with stimulus disparity to a far greater extent in the first subinterval than later in components. The data could be described by a combination of rectangular hyperbolae which attributed the interaction between local and dimensional effects to limits set by local effects on the extent that stimulus differences could affect sensitivity.

Post training lysine-vasopressin injections increase conditioned suppression in rats

Male rats were trained to lever press for food reward on a variable interval schedule of reinforcement. When stable response rates had been achieved they were trained to avoid footshock (UCS) in a two way shuttle box using a compound conditioning stimulus (CS) of tone and light. Having reached the learning criterion of ten consecutive avoidance responses they were returned to their home cage and injected with saline or lysine vasopressin (LVP 1 μg/rat/SC) after 30 min. Twenty four hours later, the suppressive effect of the avoidance CS on the appetitive baseline was tested. Rats which had been injected with LVP after avoidance training showed significantly more suppression of the operant response than saline controls. The results are discussed in terms of the behavioural substrate underlying the long term effects of vasopressin on behaviour.

UNDERMATCHING AND OVERMATCHING AS DEVIATIONS FROM THE MATCHING LAW

A model of performance under concurrent variable-interval reinforcement schedules that takes as its starting point the hypothetical "burst" structure of operant responding is presented. Undermatching and overmatching are derived from two separate, and opposing, tendencies. The first is a tendency to allocate a certain proportion of response bursts randomly to a response alternative without regard for the rate of reinforcement it provides, others being allocated according to the simple matching law. This produces undermatching. The second is a tendency to prolong response bursts that have a high probability of initiation relative to those for which initiation probability is lower. This process produces overmatching. A model embodying both tendencies predicts (1) that undermatching will be more common than overmatching, (2) that overmatching, when it occurs, will tend to be of limited extent. Both predictions are consistent with available data. The model thus accounts for undermatching and overmatching deviations from the matching law in terms of additional processes added on to behavior allocation obeying the simple matching relation. Such a model thus enables processes that have been hypothesized to underlie matching, such as some type of reinforcement rate or probability optimization, to remain as explanatory mechanisms even though the simple matching law may not generally be obeyed.

A time-series analysis of changeover performance on concurrent variable-interval schedules

Pigeons responded to concurrent variable-interval variable-interval schedules of food reinforcement. Interchangeover times and reinforcements were recorded in intact time series during acquisition following several schedule-density manipulations. Probability of a changeover, while constant as a function of time since the previous changeover, was found to be a nonmonotonically increasing function of time since reinforcement. Autocorrelation analyses of intact time series revealed the presence of statistically significant sequential dependencies in the time series during acquisition, but not during asymptotic performance. Lag transformations of actual interchangeover times revealed inhomogeneities in the time series not shown in the autocorrelations or in the overall distribution of interchangeover times.

Noncontingent and contingent no-choice intervals and concurrent performance

Pigeons responded to changeover-key concurrent variable-interval variable-interval reinforcement schedules while there were intervals during which the changeover key was inoperative (no-choice intervals). In Experiment 1, a multiple schedule on the changeover key signaled choice and no-choice intervals. All subjects showed near-perfect discrimination during initial discrimination training and rapid reacquisition of discrimination following contingency reversals. In Experiment 2, the onset of no-choice intervals was unsignaled and contingent on interchangeover time. The temporal distribution of changeover-key responses conformed to the temporal distribution of choice intervals. The results of both experiments suggest that changeover responding is modifiable as a function of its immediate consequences. The results of Experiment 2, in particular, suggest that time or some correlate of time since the last changeover response can determine subsequent changeover behavior.

FREE‐OPERANT FORGETTING: DELAYED STIMULUS CONTROL OF MULTIPLE‐SCHEDULE PERFORMANCE

Pigeons' responses were reinforced in two components of several multiple variable-interval variable-interval schedules of food reinforcement. In one component, the key was illuminated green for 15 seconds and white for 45 seconds. In the other component, the key was illuminated red for 15 seconds and white for 45 seconds. Values for the exponent of the power functions relating response ratios to reinforcement ratios were higher in the presence of the discriminative stimuli (green or red) than in the presence of white. Sensitivity of response ratios to changes in reinforcement ratios provided an index of the extent to which responding was under delayed stimulus control by prior discriminative stimuli.

The influence of positive and negative reinforcement on selective attention in the rat

In Experiments 1 and 2 rats were trained under two multiple schedules of reinforcement. In one, bar pressing during a tone-light compound stimulus was reinforced under a variable-interval food reinforcement schedule. In the other multiple schedule, bar pressing avoided grid shock on a free-operant schedule. In both multiple schedules, a discrimination was maintained by an extinction schedule that was operative during the absence of the tone-light compound. In Experiments 1 and 2 the intensity of the tone-light compound was manipulated over three levels. Subsequent extinction tests revealed that light was attended to, almost exclusively of the tone, when food reinforcement had maintained bar pressing. On the other hand, the tone gained considerable attentional control under the shock avoidance schedule. This stimulus-reinforcer interaction was maintained for all three levels of the compound intensity. In Experiment 3 it was investigated whether this interaction was associative by presenting shock during the absence of the tone-light compound when food reinforcement maintained responding, and food during the absence of the compound when shock avoidance maintained responding. Since both food and shock were presented during a single session for both schedules, nonassociative effects of the reinforcing stimuli were equivalent across the schedules. Nevertheless, the stimulus-reinforcer interaction was maintained, indicating that the interaction was an associative effect.

Effects of a brief stimulus accompanying reinforcement on instrumental responding in pigeons

The responding of pigeons on a variable interval schedule of reinforcement was investigated in four experiments. In some conditions in each experiment reinforced keypecks were accompanied by a brief (0.5-sec) flash of the houselight. This procedure resulted in a low rate of response in comparison with that found in conditions when response-contingent light flashes occurred uncorrelated with reinforcement (Experiments 1 and 2) or when no light flash was presented (Experiment 3). Experiment 4 allowed a comparison between the effects of a signal accompanying the reinforced response and one accompanying the delivery of “free” food. Signaling the delivery of earned food produced a lower rate of response than did signaling the delivery of free food. The role of stimulus-reinforcer and response-reinforcer associations in producing these effects is discussed.

Performance of Humans in Variable-Interval Avoidance Schedules Programmed Singly, and Concurrently with Variable-Interval Schedules of Positive Reinforcement

Performance maintained under single variable-interval avoidance schedules, single variable-interval schedules of positive reinforcement, and concurrent schedules consisting of a variable-interval avoidance component and a variable-interval positive reinforcement component, was studied in three human subjects, using points exchangeable for money as the reinforcer. Response rate in the single variable-interval avoidance schedules was an increasing function of the frequency of monetary loss avoidance. Response rate in the single variable-interval positive reinforcement schedules was an increasing function of the frequency of obtained monetary reinforcement. In the concurrent avoidance/reinforcement schedules, the rate of responding in the avoidance component increased, and the rate of responding in the positive reinforcement schedule decreased (with one exception) as a function of the frequency of loss avoidance in the avoidance component. The logarithms of the ratios of the response rates in the two components, and the logarithms of the ratios of the times spent in the two components, were linearly related to the logarithms of the ratios of the frequency of loss avoidance in the avoidance component to the frequency of reinforcement in the positive reinforcement component. All three subjects exhibited marked undermatching of response rate ratios to reinforcement frequency ratios. The results are discussed in the context of Herrnstein's quantitative model of operant performance.

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons were investigated in a series of three experiments. Experiment 1 compared the effects of a fixed, variable, or variable signaled changeover delay on interchangeover times and responding during and after the changeover delay. The duration of the changeover delays was systematically varied in Experiment 2, and the relative reinforcement frequencies were manipulated in Experiment 3. Interchangeover times were found to be shorter when changeover delays of variable duration were compared with those of fixed duration. Changeover delays of fixed duration produced higher response rates during the changeover delay than after the changeover delay had elapsed; changeover delays of variable duration produced such differences to a lesser extent. It was concluded that the changeover delay in concurrent variable-interval schedules of reinforcement functionally acts as a delay period to the next opportunity for reinforcement, possibly serving as a conditioned reinforcer for the behavior preceding it (the interchangeover time) and as a discriminative stimulus for the behavior in its presence (response rates during the delay).

Choice and the conditional probability of alternation: Some new data

Three pigeons were exposed to two independent concurrent variable-interval schedules of reinforcement, with a changeover key procedure. Relative reinforcement rate and the fixed-ratio changeover requirement were varied. The effects of these manipulations were assessed with respect to the conditional probability of alternation as a function of successive responses on the same schedule. Previous experiments employing this measure have found essentially unchanging probabilities of alternation. However, we found the probability of alternation to be an increasing function of successive choices of the same schedule, especially with the schedule providing the higher relative reinforcement rate. These findings were discussed with respect to the pro­ cedural variables we employed.