Variable-interval Performance Research Articles

About bouts: A heterogeneous tandem schedule of reinforcement reveals dissociable components of operant behavior in Fischer rats.

According to the biexponential refractory model (BERM) of variable-interval (VI) performance, operant behavior is organized in bouts, described by 3 dissociable components: between-bout interresponse times (IRTs), within-bout IRTs, and bout lengths. Research has shown that between-bout IRTs are sensitive to changes in rate of reinforcement and reinforcer efficacy, the length of some bouts is selectively sensitive to changes in response-reinforcer contingencies, and within-bout IRTs are relatively insensitive to both manipulations. BERM assumes that within- and between-bout IRTs are exponentially distributed, and bout lengths are described by a mixture of negative binomial and geometric distributions. To assess BERM assumptions and the interpretation of associated findings, Fischer 344/DuCrl rats were trained on a heterogeneous tandem VI fixed-ratio (FR) schedule of reinforcement intended to dissociate the components of operant behavior. Initial (VI) and terminal (FR) links were programmed on separate levers; no stimulus signaled the completion of the initial link. FR requirement, VI requirement, and deprivation level were varied. Typical performance consisted of single responses on the VI lever separated by response runs on the FR lever. It was hypothesized that (a) the interval between the end of each FR run and the first subsequent VI response (FR-VI IRTs) would constitute between-bout IRTs, and would be sensitive to changes in VI requirement and deprivation level, (b) FR runs would constitute response bouts, so the length of a fraction of them would be selectively sensitive to changes in FR requirement, and (c) intervals between consecutive FR responses (FR-FR IRTs) would constitute within-bout IRTs, and would be relatively robust to all manipulations. Findings were consistent with these expectations. The underlying distributions of FR-FR IRTs, FR-VI IRTs, and FR run lengths, however, were inconsistent with BERM assumptions. These data support the distinct components of operant performance, but challenge the simple processes assumed to underlie their generation. (PsycINFO Database Record

Pre-asymptotic response rates as a function of the delay-of-reinforcement gradient summation for Catania’s Operant Reserve: A reply to Berg & McDowell (2011)

Catania’s Operant Reserve (COR; Catania, 2005) is a computational model of operant behaviour. In COR, responding depletes the reserve while reinforcement replenishes the reserve. The replenishment to the reserve depends on the location of responses within the most recent inter-reinforcement interval. The rule that maps replenishment to the responses within an inter-reinforcement interval is given by a delay-of-reinforcement gradient (DOR). Previous research (Berg and McDowell, 2011) found that non-linear DORs produce sigmoidal response rates on single variable-interval schedules while a linear DOR produces hyperbolic response rates. Berg and McDowell took these sigmoidal response rates as evidence against the viability of COR for modelling variable-interval performance. However, Berg and McDowell did not consider the effect of the slope of each DOR. The present conjecture is that the response rates from COR can be made hyperbolic by manipulating the area under the DOR. Our results show that the manipulation of the area under the DOR allows COR to produce hyperbolic response rates regardless of the parametric form of the DOR. Hence, if COR is to be used to model single variable-interval performance, the reinforcement effect over time should be taken into consideration through the manipulation of the area under the DOR.

Within-session patterns in variable-interval schedule performance: Variation with deprivation level

Previous research has shown that patterns in operant responding may change within the course of individual experimental sessions. The proper interpretation of such changes is controversial. At least one source of this controversy may lie in unstated experimental practices across laboratories, as published reports often have failed to detail important particulars of deprivation operations. The present study was aimed at gathering descriptive data on the contribution of deprivation operations to the observation of within-session changes. In two experiments, four food deprived pigeons were exposed to a multiple variable-interval 30-s variable-interval 30-s schedule of grain presentation, wherein each grain presentation was kept constant at 5s. In Condition I, a session-entry criterion was in place that permitted the pigeon access to the daily session only if its body weight fell within controlled limits. Within-session rates of responding were generally stable. In Condition II, the entry criterion was removed and experimental sessions were conducted 6 days per week. The effect of removing the session-entry criterion was to increase body weight for all birds and decrease food intake across conditions. With no session entry criterion, robust within-session changes were observed for three of the four pigeons. The results suggest that rich schedules of reinforcement often used in the analysis of within-session changes can produce substantial reductions in deprivation levels that require up to several days to reverse. Future experiments in this area should take precautions to insure that deprivation is tightly controlled and report such measures to eliminate potential errors in replication.

Herrnstein's Equation: Data from 110 Rats

110 rats were trained under a series of variable-interval schedules of sucrose reinforcement (0.6 M, 50 μl), covering a wide range of scheduled interreinforcement intervals. Response and reinforcement rates recorded during the last five sessions of exposure to each schedule were used to fit Herrnstein's (1970) hyperbolic ‘response strength’ equation to the data from each rat The equation accounted for >80% of the data variance in 90%, and >90% of the variance in 60% of the sample. The distribution of the values of Rmax, the asymptote of the hyperbolic curve, did not depart significantly from normality. However, the distribution of the values of KH, the reinforcement rate needed to maintain the half-maximum response rate, was markedly skewed; logarithmically transformed values of KH conformed to a normal distribution. The data provide further support for the applicability of Herrnstein's equation to variable-interval performance; it is suggested that studies involving comparison of the parameters of the equation between groups of subjects should adopt logarithmic transformation of the values of KH.

Interaction between antidepressants andd-amphetamine on variable-interval performance

Four experiments were carried out investigating the interactions between some antidepressant drugs (imipramine, desipramine, fluvoxamine, trazodone (4 and 8 mg/kg) and d-amphetamine (0.1-3.2 mg/kg) on operant behaviour maintained under a variable-interval 80-s schedule of sucrose reinforcement; each experiment employed 12 rats. d-Amphetamine exerted a dose-related suppressant effect on response rate. Imipramine and desipramine given alone had no effect on response rate, whereas fluvoxamine (both doses) and the higher dose of trazodone produced significant increases in response rate. Pretreatment with imipramine, desipiramine or fluvoxamine significantly potentiated the suppressant effect of d-amphetamine on responding; pretreatment with trazodone had no significant effect. The potentiating effect of imipramine and desipramine may be related to their well known uptake blocking actions. The fact that fluvoxamine, a selective inhibitor of 5-hydroxytryptamine (5HT) uptake, also potentiated the effect of d-amphetamine suggests that the suppressant effect of d-amphetamine on operant behaviour may involve 5HT as well as catecholamine release. The lack of effect of trazodone may reflect its failure to influence uptake mechanisms. On the basis of a formal model couched in terms of Herrnstein's (1970) equation, it is suggested that imipramine, desipramine and fluvoxamine may have enhanced d-amphetamine's ability to reduce response capacity; it is suggested that the data do not provide evidence for an interaction between the antidepressants and the putative "motivation-enhancing" effect of d-amphetamine.

Constraints on the process of interresponse-time reinforcement as the explanation of variable-interval performance

Computer modelling was used to investigate the adequacy of the principle of reinforcement of single interresponse-times (IRTs) as an explanation of variable-interval performance. Variations in three components of models employing IRT reinforcement were simulated; these were (i) rules for selecting previously-reinforced IRTs from the IRT memory as “candidates” for output (acceptance rules), (ii) processes involving the transformation of previously-reinforced IRTs to generate behavioural variability, and (iii) the role of the minimum IRT value permitted by the models. In general, simulation output resembled experimental data only when (i) acceptance rules discriminated strongly against long, and in favour of short, IRTs; and (ii) when IRTs were transformed by distributions in which standard deviation was a large fraction (50–125%) of the mean, thus generating substantial IRT variability. Simulation results were analyzed by means of Herrnstein's hyperbolic equation, and in general, it was found that the output of IRT reinforcement models was insensitive to the rate of reinforcement received, as well as conforming less well to Herrnstein's equation than did experimental data. Overall, therefore, it was concluded that models of the reinforcement of single IRTs could provide an adequate simulation of VI performance only when the processes within them operated within narrow constraints. Even then, models involving the reinforcement of single IRTs could adequately simulate only those VI data in which response rate was insensitive to reinforcement rate.

Serial properties of behavior and their chemical modification

Certain features of operant behavior that are sensitive to the adverse effects of chemical exposure can be obscured by exclusive experimental reliance on global measures such as response rate. Temporal patterning of behavior is the clearest example. Reinforcement schedules studied in behavioral pharmacology and toxicology reveal novel consequences of chemical treatment when subjected to analyses of their temporal and serial properties. Fixed-ratio performance, for example, undergoes not only changes in response rate, but also displays distinctive shifts in inter-response time patterning, changes in the interresponse time distribution, and deterioration in what might be termed the cohesiveness of the ratio. Variable-interval performance also may change in distinctive ways that produce altered response patterning without marked changes in rate. Sequential dependencies, as measured by techniques of time series analysis, may also reveal effects not reflected by response rates. Spaced responding and autoregressive schedules provide examples. The serial and temporal properties alluded to above can be described and analyzed by a variety of quantitative techniques that also yield information of theoretical interest.

Evidence that background GABAergic tone and possible ligand release may alter the action of the 'neutral' benzodiazepine-receptor antagonist, Ro 15-1788, in hypothalamic self-stimulation

Upward or downward shifts in the level of brain GABAergictransmission have been held to be necessary and sufficient to promote release of endogenous ligands ('endocoids') for the benzodiazepine (BZD) recognition site. To investigate this possibility, variable-interval self-stimulation performance was used to monitor 'intrinsic' benzodiazepine-like and anti- benzodiazepine activity by the 'neutral' benzodiazepine-receptor antagonist, Ro 15-1788 (flumazenil) (10 or 30 mg/kg intraperitoneally). Rats were pretreated with either a GABA synthesis blocking agent (isoniazid, 130 mg/kg subcutaneously), or with a GABA agonist (progabide, 30 or 100 mg/kg intraperitoneally). The lower dose of Ro 15-1788 (10 mg/kg), without pretreatment, did not affect self-stimulation; higher doses (30 mg/kg) caused a brief (<20 min) depression. Isoniazid (130 mg/kg) depressed self-stimulation, but did not modify the activity of Ro 15-1788. In rats pretreated with progabide (100 mg/kg), low doses of Ro 15- 1788 (10 mg/kg) that were previously without effect now caused a sharp fall in responding. These findings can be interpreted as showing that even low doses of Ro 15-1788 may affect self-stimulation under certain conditions, and that they do so by competing with an endogenous ligand for the benzodiazepine site, released by upward shifts in GABAergic activity. Alternative explanations in terms of altered receptor function seem less feasible. The results imply that the action of the endogenous ligand would not resemble that of a typical benzodiazepine, but that of an inverse agonist (that is, proconflict and proconvulsant); this conclusion agrees with recent biochemical evidence.

Interaction of methadone, reinforcement history, and variable-interval performance.

In the present study, we examined how a reinforcement schedule history that generated high or low rates of responding influenced the effects of acute (Experiment 1) and chronic (Experiment 2) methadone administration. Initially, key-peck responses of pigeons were maintained under a variable-interval 90-s schedule of food presentation, and a methadone dose-response curve was determined with doses of 0.6, 1.2, and 2.4 mg/kg. The pigeons were then exposed, for at least 40 sessions, to either a fixed-ratio 50 schedule or a differential-reinforcement-of-low-rate 10-s schedule, or were given continued exposure to the variable-interval schedule. The methadone dose-response curve was redetermined after all pigeons again were responding under the variable-interval schedule. The effects of two different daily methadone doses (9.0 and 12.0 mg/kg/day) and withdrawal precipitated by naloxone also were assessed. Experience with a fixed-ratio or differential reinforcement of low rate schedule did not result in significantly different response rates under the variable-interval schedule and, in general, the acute effects of methadone did not have differential effects correlated with schedule history. However, for 2 of 4 subjects the rate-decreasing effects of methadone on rates of key pecking were greater following a history of low-rate responding, suggesting a possible interaction between schedule history and effects of methadone. Daily methadone administration under the variable-interval schedule revealed that pigeons with experience under the differential reinforcement of low rate schedule developed more rapid and complete tolerance to the rate-decreasing effects of methadone. Three of the 4 subjects in this group showed rate increases above drug-free baselines during chronic methadone dosing. Pigeons with a history of fixed-ratio responding also developed tolerance to the rate-decreasing effects of methadone but without the subsequent rate increases seen by subjects with low-rate histories. No subjects with variable-interval histories showed complete recovery of drug-free baselines, suggesting that interpolated training under other schedules may attenuate the rate-altering effects of chronically administered drugs. Naloxone (1.0 mg/kg), administered during the chronic methadone phase, resulted in greater disruption of responding by pigeons with a history of low-rate responding, as compared to subjects in the other two groups.(ABSTRACT TRUNCATED AT 400 WORDS)

DSP4 alters the effect of d-amphetamine on variable-interval performance: analysis in terms of Herrnstein's equation.

The effect of d-amphetamine (0.1-3.2 mg/kg) on performance in variable-interval 1-min and variable-interval 12-min schedules of positive reinforcement was examined in ten rats treated with the selective noradrenaline neurotoxin DSP4 and 12 control rats. In the control group d-amphetamine had a dose-dependent suppressant effect on response rates maintained under variable-interval 1-min; under variable-interval 12-min, response rates were increased by low doses and suppressed by higher doses of the drug. In the case of both schedules, lower doses of d-amphetamine were more suppressant and higher doses less suppressant in the DSP4-treated group than in the control group. The levels of noradrenaline in the parietal cortex, hippocampus and cerebellum (determined by high-performance liquid chromatography) were reduced to approximately 15% of control levels in the DSP4-treated rats. The results indicate that treatment with DSP4 attenuated both the facilitatory and the suppressant effects of d-amphetamine on variable-interval performance. A formal model couched in terms of Herrnstein's (1970) equation is put forward to account for these results. It is suggested that the noradrenergic pathways emanating from the locus caeruleus are involved in both the facilitatory and suppressant effects of d-amphetamine on operant behaviour.

Effect of pimozide on variable-interval performance: Failure to obtain support for the anhedonia hypothesis

Effect of pimozide on variable-interval performance: Failure to obtain support for the anhedonia hypothesis

The effect of pimozide on variable-interval performance: a test of the 'anhedonia' hypothesis of the mode of action of neuroleptics.

A quantitative behavioural test system based on Herrnstein's (1970) equation was used to test a prediction derived from the "anhedonia" hypothesis of neuroleptic action, that pimozide should increase the value of the behavioural parameter KH (the reinforcement frequency needed to maintain the half-maximal response rate in variable-interval schedules). On the basis of theoretical considerations, it was shown that the equation implies that a drug which exerts such an effect on KH must have a more profound suppressant effect on performance maintained by low reinforcement frequencies than on performance maintained by high reinforcement frequencies. Fifteen rats were trained under variable-interval 10-s and variable-interval 100-s schedules, and the effect of pimozide (0.125, 0.25, 0.33, and 0.5 mg/kg) was tested on performance maintained under both schedules in a dose-dependent manner, and there was no tendency for the drug to exert a greater effect on performance maintained under the lower reinforcement frequency. These results do not provide any evidence that the effect of pimozide on variable-interval performance is due to an "anti-hedonic effect"; rather, they are compatible with the hypothesis that pimozide impairs the capacity to respond.

The effect of pentobarbitone on variable-interval performance: analysis in terms of Herrnstein's equation.

Six rats were exposed to a series of variable-interval schedules of reinforcement, each schedule specifying a different reinforcement frequency. The effects of two doses of pentobarbitone (24.16 and 40.27 mumol/kg) upon performance maintained under each schedule were examined. The data were analyzed in terms of the goodness of fit of Herrnstein's (1970) equation. Under control conditions (no injection or injection of vehicle alone) response rate was an increasing negatively accelerated function of reinforcement frequency, the data obtained from each rat conforming closely to Herrnstein's equation. In the presence of pentobarbitone there was a reduction in the value of Rmax (the theoretical maximum response rate) in all six rats. The effect of pentobarbitone upon the value of KH (the reinforcement frequency needed to obtain the half-maximal response rate) differed between animals, the value being increased in some rats and decreased in others. This was shown to reflect a systematic relationship between the baseline value of KH and the change in KH produced by pentobarbitone, high baseline values being associated with reductions, and low baseline values with elevations, in the presence of the drug. The data were also analyzed using conventional rate-dependency analysis. In some animals there was a tendency for lower response rates to be suppressed to a greater degree than higher rates, while the reverse was true in other animals. In general, rate-dependency functions provided poor descriptions of the data obtained from individual subjects.

The effects of age, exposure, preexposure, and noise conditions on variable interval performance

The present study was designed to investigate the differential effects of environmental factors on performance in young and aged rats. Specifically, within- and between-session habituation, environmental preexposure, and noise conditions were examined. Consistent age differences were found in response to environmental change and sensory condition. Variable-interval response rates of the well-habituated old animal were well below those of the young. Stimulus change and increments in noise elevated these behaviors in the aged to levels well above those seen in the young. Aged animals appeared incapable of maintaining higher response rates without corresponding conditions of novelty, contextual change, or increments in ambient sensory conditions. Preexposure to sensory conditions similar to those used in the operant chamber reduced response rates in the old, as did habituation with repeated exposure to the chamber.

Differential punishment of other behavior: Effects of pause-dependent shock on variable-interval performance

Three albino rats were exposed to a differential-punishment-of-other-behavior shock schedule superimposed on a variable-interval schedule of food presentation. With this arrangement, failure to barpress for a specified interval since the previous response resulted in delivery of shock. This procedure reliably decreased the number of specified pauses and was accompanied by increases in barpress rates and shifts in the distribution of responses. Subsequent exposure to noncontingent shock produced similar, but quantitatively smaller, changes for two of the three subjects. Training with barpress-dependent shock and reexposure to noncontingent shock further diminished these effects. This study demonstrates that pauses are conditionable units of behavior insofar as they are sensitive to a punishment contingency. In terms of the targeted unit of behavior, the current findings are similar to those obtained with barpress-dependent or interresponse-time-dependent shock schedules and suggest a continuum of response specification.

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons were investigated in a series of three experiments. Experiment 1 compared the effects of a fixed, variable, or variable signaled changeover delay on interchangeover times and responding during and after the changeover delay. The duration of the changeover delays was systematically varied in Experiment 2, and the relative reinforcement frequencies were manipulated in Experiment 3. Interchangeover times were found to be shorter when changeover delays of variable duration were compared with those of fixed duration. Changeover delays of fixed duration produced higher response rates during the changeover delay than after the changeover delay had elapsed; changeover delays of variable duration produced such differences to a lesser extent. It was concluded that the changeover delay in concurrent variable-interval schedules of reinforcement functionally acts as a delay period to the next opportunity for reinforcement, possibly serving as a conditioned reinforcer for the behavior preceding it (the interchangeover time) and as a discriminative stimulus for the behavior in its presence (response rates during the delay).

AN ANALYTIC COMPARISON OF HERRNSTEIN'S EQUATIONS AND A MULTIVARIATE RATE EQUATION

Herrnstein's equations are approximations of the multivariate rate equation at ordinary rates of reinforcement and responding. The rate equation is the result of a linear system analysis of variable-interval performance. Rate equation matching is more comprehensive than ordinary matching because it predicts and specifies the nature of concurrent bias, and predicts a tendency toward undermatching, which is sometimes observed in concurrent situations. The rate equation contradicts one feature of Herrnstein's hyperbola, viz., the theoretically required constancy of k. According to the rate equation, Herrnstein's k should vary directly with parameters of reinforcement such as amount or immediacy. Because of this prediction, the rate equation asserts that the conceptual framework of matching does not apply to single alternative responding. The issue of the constancy of k provides empirical grounds for distinguishing between Herrnstein's account and a linear system analysis of single alternative variable-interval responding.

D-Amphetamine, operant history, and variable-interval performance

The effects of d-amphetamine on the bar-pressing of rats maintained under a variable-interval schedule of water reinforcement were examined as a function of the operant history of the subjects. One group of rats initially received 51 sessions of exposure to a fixed-ratio 20 schedule, while a second group received equivalent exposure to an interresponse-time-greater-than-12-sec schedule. Mean group response rate when stable was over ten times as high under the fixed-ratio schedule as under the interresponse-time-greater-than-12-sec schedule. Response rates of the two groups largely converged across 47 sessions of exposure to a variable-interval 60-second schedule, at which time response rates for both groups appeared stable. Acute administrations of d-amphetamine sulfate similarly affected mean response rates of both groups: A 0.25 mg/kg dose did not obviously affect rate, while doses of 0.5, 1.0, and 2.0 mg/kg produced dose-dependent rate decreases. These results indicate that the efficacy of operant history as a determinant of drug effects may be limited to circumstances where current contingencies do not exercise powerful and direct control over behavior.

Effects of phencyclidine, atropine and physostigmine, alone and in combination, on variable-interval performance in the squirrel monkey

The role played by cholinergic activity in the effects of phencyclidine (PCP) on schedule-controlled responding was studied in three squirrel monkeys trained to respond on a variable-interval (VI) 100 sec schedule of food presentation. A low dose of PCP (0.08 mg/kg IM) produced small increases in rates of responding. Higher doses (0.16–0.64 mg/kg) produced dose-dependent decreases in rates of responding. Atropine (0.05–3.2 mg/kg IM) and physostigmine (0.025–0.20 mg/kg IM) caused only decreases in response rates, the dose-response curve for atropine being particularly flat over a wide range of doses. When atropine was combined with PCP, no significant interaction was obtained. When physostigmine was combined with PCP, a complex interaction was observed. Evidence for partial antagonism of PCP by physostigmine was obtained only at the highest PCP dose tested. Atropine-physostigmine combinations resulted in response rates suggestive of antagonism.

A MULTIVARIATE RATE EQUATION FOR VARIABLE‐INTERVAL PERFORMANCE

A value-like parameter is introduced into a rate equation for describing variable-interval performance. The equation, derived solely from formal considerations, expresses rate of responding as a joint function of rate of reinforcement and "reinforcer power." Preliminary tests of the rate equation show that it handles univariate data as well as Herrnstein's hyperbola. In addition, a form of Herrnstein's hyperbola can be derived from the equation, and it predicts forms of matching in concurrent situations. For the multivariate case, reinforcer values scaled in concurrent situations where matching is assumed to hold are taken as determinations of reinforcer power. The multivariate rate equation is fitted to an appropriate set of data and found to provide a good description of variable-interval performance when both rate and power of reinforcement are varied. Rate and power measures completely describe reinforcement. The effects of their joint variation are not predicted and cannot be described by Herrnstein's equation.