Multiple Variable-interval Schedules Research Articles

Reduction of Cocaine Self-Administration and D3 Receptor-Mediated Behavior by Two Novel Dopamine D3 Receptor-Selective Partial Agonists, OS-3-106 and WW-III-55

Dopamine D3 receptor (D3R)-selective compounds may be useful medications for cocaine dependence. In this study, we identified two novel arylamide phenylpiperazines, OS-3-106 and WW-III-55, as partial agonists at the D3R in the adenylyl cyclase inhibition assay. OS-3-106 and WW-III-55 have 115- and 862-fold D3R:D2 receptor (D2R) binding selectivity, respectively. We investigated their effects (0, 3, 5.6, or 10 mg/kg) on operant responding by using a multiple variable-interval (VI) 60-second schedule that alternated components with sucrose reinforcement and components with intravenous cocaine reinforcement (0.375 mg/kg). Additionally, we evaluated the effect of OS-3-106 (10 mg/kg) on the dose-response function of cocaine self-administration and the effect of WW-III-55 (0-5.6 mg/kg) on a progressive ratio schedule with either cocaine or sucrose reinforcement. Both compounds were also examined for effects on locomotion and yawning induced by a D3R agonist. OS-3-106 decreased cocaine and sucrose reinforcement rates, increased latency to first response for cocaine but not sucrose, and downshifted the cocaine self-administration dose-response function. WW-III-55 did not affect cocaine self-administration on the multiple-variable interval schedule, but it reduced cocaine and sucrose intake on the progressive ratio schedule. Both compounds reduced locomotion at doses that reduced responding, and both compounds attenuated yawning induced by low doses of 7-OH-DPAT (a D3R-mediated behavior), but neither affected yawning on the descending limb of the 7-OH-DPAT dose-response function (a D2R-mediated behavior). Therefore, both compounds blocked a D3R-mediated behavior. However, OS-3-106 was more effective in reducing cocaine self-administration. These findings support D3Rs, and possibly D2Rs, as targets for medications aimed at reducing the motivation to seek cocaine.

Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat

BackgroundOperant hyperactivity, the emission of reinforced responses at an inordinately high rate, has been reported in children with ADHD and in the Spontaneously Hypertensive Rat (SHR), the most widely studied animal model of ADHD. The SHR emits behavior at hyperactive levels, relative to a normoactive strain, only when such behavior is seldom reinforced. Because of its dependence on rate of reinforcement, operant hyperactivity appears to be driven primarily by incentive motivation, not motoric capacity. This claim was evaluated in the present study using a novel strategy, based on the organization of behavior in bouts of reinforced responses separated by pauses.MethodMale SHR, Wistar-Kyoto (WKY) and Wistar rats (WIS) were exposed each to a multiple variable-interval schedule of sucrose reinforcement (12, 24, 48, 96, and 192 s) between post-natal days (PND) 48 and 93. Responding in each schedule was examined in two epochs, PND 58-62 and 89-93. Parameters of response-reinforcement functions (Herrnstein's hyperbola) and bout-organized behavior were estimated in each epoch.ResultsSHR emitted higher response rates than WKY and WIS, but only when rate of reinforcement was low (fewer than 2 reinforcers per minute), and particularly in the second epoch. Estimates of Herrnstein's hyperbola parameters suggested the primacy of motivational over motoric factors driving the response-rate differential. Across epochs and schedules, a more detailed analysis of response bouts by SHR revealed that these were shorter than those by WKY, but more frequent than those by WKY and WIS. Differences in bout length subsided between epochs, but differences in bout-initiation rate were exacerbated. These results were interpreted in light of robust evidence linking changes in bout-organization parameters and experimental manipulations of motivation and response-reinforcement contingency.ConclusionsOperant hyperactivity in SHR was confirmed. Although incentive motivation appears to play an important role in operant hyperactivity and motoric capacity cannot be ruled out as a factor, response-bout patterns suggest that operant hyperactivity is primarily driven by steeper delay-of-reinforcement gradients. Convergence of this conclusion with theoretical accounts of ADHD and with free-operant performance in children with ADHD supports the use of SHR as an animal model of ADHD.

A further application of the active time model to multiple concurrent variable-interval schedules

In this experiment we show that the active time model (ATM) accurately predicts probe data from multiple concurrent VI VI schedules. Subjects were trained under a concurrent VI 30-s VI 60-s and a concurrent VI 60-s VI 120-s schedule. Two types of unreinforced probes were then conducted. The first paired the two VI 60-s stimuli. These stimuli, while equivalent in their associated absolute rates of reinforcement, differed in their relative rates of reinforcement. The second probe paired the VI 30-s stimulus with the relatively rich VI 60-s stimulus. In contrast with the first probe, these stimuli differed in their absolute rates of reinforcement, while being similar in their relative rates. During the first set of probes, birds preferred the VI 60-s stimulus trained with the VI 120-s schedule. During the second set of probes, birds were indifferent to the two stimuli. These results are less extreme than others reported in the literature. Nonetheless, we found that ATM accurately fit individual subject data in both sets of probes. In contrast a variant of scalar expectancy theory did not fit the data at either the individual or group level.

Within-session patterns in variable-interval schedule performance: Variation with deprivation level

Previous research has shown that patterns in operant responding may change within the course of individual experimental sessions. The proper interpretation of such changes is controversial. At least one source of this controversy may lie in unstated experimental practices across laboratories, as published reports often have failed to detail important particulars of deprivation operations. The present study was aimed at gathering descriptive data on the contribution of deprivation operations to the observation of within-session changes. In two experiments, four food deprived pigeons were exposed to a multiple variable-interval 30-s variable-interval 30-s schedule of grain presentation, wherein each grain presentation was kept constant at 5s. In Condition I, a session-entry criterion was in place that permitted the pigeon access to the daily session only if its body weight fell within controlled limits. Within-session rates of responding were generally stable. In Condition II, the entry criterion was removed and experimental sessions were conducted 6 days per week. The effect of removing the session-entry criterion was to increase body weight for all birds and decrease food intake across conditions. With no session entry criterion, robust within-session changes were observed for three of the four pigeons. The results suggest that rich schedules of reinforcement often used in the analysis of within-session changes can produce substantial reductions in deprivation levels that require up to several days to reverse. Future experiments in this area should take precautions to insure that deprivation is tightly controlled and report such measures to eliminate potential errors in replication.

RESISTANCE TO CHANGE OF RESPONDING MAINTAINED BY UNSIGNALED DELAYS TO REINFORCEMENT: A RESPONSE‐BOUT ANALYSIS

Previous experiments have shown that unsignaled delayed reinforcement decreases response rates and resistance to change. However, the effects of different delays to reinforcement on underlying response structure have not been investigated in conjunction with tests of resistance to change. In the present experiment, pigeons responded on a three-component multiple variable-interval schedule for food presented immediately, following brief (0.5 s), or following long (3 s) unsignaled delays of reinforcement. Baseline response rates were lowest in the component with the longest delay; they were about equal with immediate and briefly delayed reinforcers. Resistance to disruption by presession feeding, response-independent food during the intercomponent interval, and extinction was slightly but consistently lower as delays increased. Because log survivor functions of interresponse times (IRTs) deviated from simple modes of bout initiations and within-bout responding, an IRT-cutoff method was used to examine underlying response structure. These analyses suggested that baseline rates of initiating bouts of responding decreased as scheduled delays increased, and within-bout response rates tended to be lower in the component with immediate reinforcers. The number of responses per bout was not reliably affected by reinforcer delay, but tended to be highest with brief delays when total response rates were higher in that component. Consistent with previous findings, resistance to change of overall response rate was highly correlated with resistance to change of bout-initiation rates but not with within-bout responding. These results suggest that unsignaled delays to reinforcement affect resistance to change through changes in the probability of initiating a response bout rather than through changes in the underlying response structure.

Resistance to Extinction in the Steady State and in Transition.

Three experiments with pigeons explored the constancy of reinforcer omission during extinction conjectured by rate estimation theory. Experiment 1 arranged 3-component multiple variable-interval (VI) schedules with a mixture of food and extinction trials within each session. Reinforcers omitted to an extinction criterion increased with food-trial reinforcer rate. Experiment 2 arranged 3-component multiple VI schedules where components differed in rate or number of reinforcers. Resistance to extinction depended on the training reinforcer rate but not on the number of reinforcers omitted. Experiment 3 replicated the partial-reinforcement extinction effect within subjects in a discrete-trial procedure and found that more reinforcers were omitted in continuous- than in partial-reinforcement trials. A model of extinction based on behavioral momentum theory accounted for all the data.

Reinforcement context and resistance to change.

Eight pigeons responded in a multiple variable-interval (VI) schedule in which a constant component always delivered 40rft/h, and an alternated component was either rich (200rft/h) or lean (6.67rft/h) in different conditions. Four tests of resistance to change were conducted in each condition: prefeeding, full extinction, constant-component-only extinction, and response-independent food. Resistance to both prefeeding and full extinction in the constant component varied inversely with the reinforcement rate in the alternated component, but resistance to response-independent food did not. The extinction and response-independent food results were consistent with [J. Exp. Psychol.: Anim. Behav. Proc. 25 (1999) 256] behavioral momentum model. Maintaining reinforcement in the alternated component increased resistance to extinction in the constant component, as predicted by the behavioral momentum model but not accounts of multiple-schedule performance based on [J. Exp. Anal. Behav. 13 (1970) 243] equation. Overall, the momentum model gave a good account of the results with the exception of the prefeeding data. Possible ways to reconcile the prefeeding results with behavioral momentum theory are considered.

Improved multiparameter models of drug effects on response rate under multiple variable interval schedules: evidence from rat studies.

The functional analysis of the behavioral effects of drugs has lagged behind more biological research approaches in elucidating how drugs influence behavior. Part of the problem is the scarcity of mathematical models of responding under standardized behavioral procedures. A two-parameter model derived from the matching law to account for response rate as a function of reinforcement rate in single response alternative procedures provides a good description of responding under multi-component multiple variable interval (multVI) schedules of reinforcement, both in the absence of drugs and in the presence of a variety of drugs. This paper explores how well the two-parameter model and its exponentiated variant (three-parameter model) describe drug effects on responding under multVI schedules. Data from seven published papers concerning the effects of drugs on multVI schedules are reanalyzed. The three-parameter model describes the effects of drugs on responding more accurately than does the two-parameter model. The three-parameter model also describes well the type of relation between response rates under drug and no-drug conditions that the rate dependency hypothesis is usually invoked to explain. The three-parameter model can be improved by adding a parameter to account for base response rate, that is, response rate when reinforcement rate is zero. The parameters of the best descriptive model could be used to devise a classification system for the behavioral effects of drugs under multVI schedules. The parameters should be conceptualized by relating the drug-induced changes in parameter value to converging changes in measurements of brain function. This would connect the behavioral and the biological analyses of drug action to the benefit of both research fields.

Motivation for alcohol assessed by multiple variable interval schedule behaviour: effects of reward size and alcohol cues.

Altered motivation for drugs of abuse is a central feature of most definitions of drug dependence and the impact of drug-related cues on motivation is of current interest. However, since most studies of cue-reactivity have not used behavioural measures of motivation, their results are often difficult to interpret in motivational terms. In the current paper we describe two experiments in which a behavioural technique, based on multiple variable interval (VI) schedules of reinforcement, was used to study motivation for alcohol in human subjects. In both experiments, subjects attended for several sessions and, during each session, were exposed to a 6-ply VI schedule (values ranged from 1 to 720 s), during which they earned points that were later exchanged for a preferred beer or lager. In Experiment 1 the procedure was validated by showing that changes in the magnitude of reinforcement altered behaviour appropriately. In Experiment 2 we found evidence that the presence of an alcohol-related cue increased the value of alcohol rewards. The results are discussed with reference to a model for the behavioural effects of drug-related cues in triggering relapse and a number of problems we found in using the multiple variable interval schedule procedure.

Multiple variable interval schedule behaviour in humans: effects of ethanol, mood, and reinforcer size on responding maintained by monetary reinforcement.

Ethanol is an effective reinforcer but, in common with other drugs of abuse, it may derive some of its reinforcing properties from the effects it has on behaviour maintained by other reinforcers. However, any assessment of ethanol's hypothesized effect on behaviour maintained by other reinforcers must take into account the fact that ethanol may have multiple mechanisms of action. In order to address this problem the experiments reported herein used a procedure based upon Herrnstein's Matching Law which allowed joint assessment of subjects' motor capacity and reinforcer sensitivity. The effect of ethanol (0, 0.3, and 0.6 g/kg) on motor capacity and reinforcer sensitivity was assessed by studying behaviour maintained by monetary reinforcement. In the first experiment the procedure was validated by showing that the behaviour of subjects was sensitive to changes in reinforcer value and in the second experiment 0.6 g/kg ethanol reduced motor capacity but did not affect reinforcer sensitivity. As a secondary hypothesis we also studied the effect of mood on reinforcer sensitivity and motor capacity. It was found that lower mood scores (lower hedonic tone) were associated with reduced reinforcer sensitivity and that male subjects showed higher motor capacity than females. However, there was also a mood by sex interaction, which indicated that higher motor capacity in males was only found in the presence of lower mood scores. The results are discussed in relation to the mechanisms of ethanol's dopaminergic effects, interactions between ethanol and other drugs of abuse, and the changes in reinforcer sensitivity which are thought to occur in depression.

Behavioral contrast using different reinforcers: effect of baseline rate of reinforcement

The present study determined whether behavioral contrast would occur when different reinforcers were delivered in the different components and whether its size would vary at different baseline rates of reinforcement. Pigeons pecked keys on a multiple variable-interval schedule. Mixed grain was the reinforcer in one component and wheat was the reinforcer in the other component. In contrast conditions, the rate of wheat reinforcement was increased or decreased, from the baseline delivery rate, by a factor of four. Contrast was studied at four different baseline rates of reinforcement. Contrast was usually observed and its size almost always varied directly with the programmed baseline rate of reinforcement. The present results indicate that changes in the condition of reinforcement of a different reinforcer can produce contrast. They also broaden the potential implications of behavioral contrast.

Ethanol’s Effects on Operant Responding: Differentiating Reinforcement Efficacy and Motor Performance

PETRY, N. M. Ethanol’s effects on operant responding: Differentiating reinforcement efficacy and motor performance. Physiol Behav 64(1) 117–122, 1998. The purpose of this study was to examine the effects of ethanol on operant responding in rats. Drugs may affect response rates by several mechanisms, including altering motor performance or sensitivity to reinforcement. These two processes can be dissociated by applying a quantitative model of reinforced responding the response–strength equation, to response rates obtained during multiple variable-interval (VI) schedules of reinforcement. This equation has two fitted parameters. One ( k) is thought to measure changes in motor performance, and the other ( R e) is thought to measure changes in reinforcement efficacy. In experimental sessions, rats earned sucrose reinforcement on a seven-component multiple VI operant schedule. The average intervals varied from 108 to 3 s and provided reinforcement rates from 30 to 1200/h. Following stable baseline performance, rats were injected intraperitoneally prior to sessions with different doses of ethanol: 0.0, 0.3, 0.6, and 0.9 g/kg. Low to moderate doses of ethanol (0.3 and 0.6 g/kg) significantly decreased response rates on several of the VI schedules but did not alter either of the fitted parameters, suggesting that these doses of ethanol did not affect motor performance or sensitivity to reinforcement. The 0.9 g/kg dose decreased responding maintained by many of the VI schedules and significantly increased the R e parameter, suggesting that the sucrose became relatively less efficacious in ethanol compared to vehicle sessions. This relatively high dose of ethanol, however, did not alter responding maintained by the richest reinforcement schedule, and it did not affect motor performance as defined by the k parameter. These results suggest that, within this paradigm, ethanol affects motivation to respond rather than ability to respond.

Operant schedule transformations and human behavioral momentum

Behavioral momentum, which can be defined as the persistence of behavior under altered environmental contingencies, is derived from Newtonian physics and operant psychology, and has relevance to behavior therapy in terms of shaping strong behaviors and ensuring effective relapse prevention strategies. The present study investigated whether changing operant schedule contingencies affects how humans respond to different stimuli when reinforcement density is systematically manipulated. Fifteen subjects participated in a computer study, in which each of two keys in a baseline condition was associated with the same schedule of reinforcement, multiple variable interval schedules, the only difference being that one reinforcer was ten times larger than the other. After six sessions, the contingency schedule changed to either an extinction condition, a variable time schedule, or a changed variable interval schedule to assess how subjects' responses persisted when reinforcement contingencies were systematically changed. Results of this study were found to be consistent with the general predictions of behavioral momentum. Subjects not only biased responding in favor of the more densely reinforcing key, but when contingencies changed, subjects showed continued biased responding. Implications for behavioral momentum for behavior modification and behavior therapy are discussed, and it is concluded that behavioral momentum has significant implications for designing new and comprehensive behavior change programs.

CLOSED‐ECONOMY MULTIPLE‐SCHEDULE PERFORMANCE: EFFECTS OF DEPRIVATION AND SESSION DURATION

Three pigeons responded for food reinforcement on multiple variable-interval schedules in which the total consumption of food was entirely determined by the subjects' interaction with the schedules (a closed economy). The finding of overmatching, where response allocation between components is more extreme than the distribution of reinforcers, was reconfirmed. Generalized-matching sensitivity decreased from overmatching to undermatching values typical of conventional multiple schedules when food deprivation was increased by decreasing session duration, but not when deprivation was increased by decreasing overall reinforcer rate. Sensitivity also increased from undermatching to overmatching as session duration increased from 100 min to 24 hr, while deprivation was held constant by decreasing overall reinforcer rate. These results can be understood in terms of increases in the value of extraneous reinforcers relative to food reinforcers as deprivation decreases or as the economy for extraneous reinforcers becomes more closed. However, no published quantitative expression of the effects of extraneous reinforcers is entirely consistent with the results.

On the contributions of responding and reinforcement to within-session patterns of responding

Operant responding may change systematically within experimental sessions even when the contingencies of reinforcement remain constant across the session. In two experiments, eight pigeons pecked a key for mixed grain delivered by variable interval or multiple variable interval schedules. Experiment 1 showed that changes in the rate at which reinforcers were delivered altered the within-session patterns even when the total amount of reinforcement presented per session was held constant. Experiment 2 showed that changing the overall rate of responding did not alter within-session response patterns. These results question the idea that within-session patterns of responding are produced by simple interpretations of satiation or fatigue. If within-session changes in responding were produced by satiation, then subjects receiving equal amounts of reinforcement should have shown similar within-session patterns of responding, but they did not (Experiment 1). If within-session changes were produced by fatigue, then the patterns should have differed when the overall response rate differed, but they did not (Experiment 2). The present study confirms that within-session patterns of responding are sensitive to changes in rate of reinforcement.

WITHIN‐SESSION CHANGES IN KEY AND LEVER PRESSING FOR WATER DURING SEVERAL MULTIPLE VARIABLE‐INTERVAL SCHEDULES

Rats pressed keys or levers for water reinforcers delivered by several multiple variable-interval schedules. The programmed rate of reinforcement varied from 15 to 240 reinforcers per hour in different conditions. Responding usually increased and then decreased within experimental sessions. As for food reinforcers, the within-session changes in both lever and key pressing were smaller, peaked later, and were more symmetrical around the middle of the session for lower than for higher rates of reinforcement. When schedules provided high rates of reinforcement, some quantitative differences appeared in the within-session changes for lever and key pressing and for food and water. These results imply that basically similar factors produce within-session changes in responding for lever and key pressing and for food and water. The nature of the reinforcer and the choice of response can also influence the quantitative properties of within-session changes at high rates of reinforcement. Finally, the results show that the application of Herrnstein's (1970) equation to rates of responding averaged over the session requires careful consideration.

BEHAVIORAL CONTRAST IN A TWO‐OPTION ANALOGUE TASK OF FINANCIAL DECISION MAKING

The effects of an alternative course of action on sustained escalation and persistence in the face of failure was investigated using a computerized stock investment task. Subjects invested in "stock" in two "markets" that yielded returns according to two-component multiple variable-interval schedules. Both markets yielded equal but intermittent return rates during the first phase. In the second phase, one market ceased to yield returns, while the return rate for the other market was unchanged. During the second phase, behavioral contrast effects were evident. Investing in the market that ceased to yield returns dropped precipitously, and investing in the unchanged market increased significantly. Although the behavior may be economically "irrational," it is predictable from the matching law and shows that interactions among a history of intermittent returns in a course of action, current return rate, and currently available alternative courses of action are important determinants of persisting in, or withdrawing from, a failing course of action.

On the dynamics of behavior allocation between simultaneously and successively available reinforcer sources

The assumption of constant rates of extraneous reinforcement during changes in explicit reinforcer rates made by Herrnstein (1970) must logically be incorrect. An alternative theory that suggests constant extraneous-schedule values, rather than reinforcer rates, is suggested. In a test of this theory, six pigeons were trained on multiple variable-interval schedules with either no alternative reinforcers available, or with various variable-ratio or variable-interval schedules of reinforcement arranged concurrently, in 26 experimental conditions. A modification of McLean and White's (1983) multiple-schedule model that assumed a constant extraneous variable-ratio schedule fitted the obtained data, and those from two previous experiments, well. However, the model is dynamic, and allows the subject a range of adaptations to variations in arranged reinforcer rates. It is argued that this range of adaptations is both appropriate and necessary.

Within-session responding as a function of post-session feedings

Ten rats pressed levers or keys for food reinforcers delivered by a multiple variable interval schedule. The delay between the end of the session and the delivery of a post-session feeding varied from 0 to 240 minutes. Contrary to the results reported by Bacotti (1976), response rates were not significantly higher when post-session feedings were delayed than when they were immediate. Response rates also increased and then decreased within the session, regardless of the delay to post-session feedings. These results suggest that subjects do not always integrate rewards across locations. Therefore, theories need not always include the temporal location of post-session feedings in the context of variables that determines responding within the session. Experiments must also take care to ensure that changes in response rates within the session do not confound the interpretation of their results.

Behavioral contrast in pigeons and rats: A comparative analysis

The effects of reinforcement rate on behavioral contrast were examined in pigeons and rats. Each species was exposed to a series of 12 multiple variable-interval schedules, divided into four 3-schedule series. Each series consisted of a standard contrast manipulation, and baseline schedules provided a different rate of reinforcement in each of the series. The functions relating reinforcement rate to the magnitude of contrast were different across species. Rats showed a U-shaped function, with reliable contrast occurring only at high reinforcement rates. Pigeons showed an inverted U-shaped function, with contrast occurring on all schedules except the schedule providing the lowest rate of reinforcement. Pigeons discriminated between schedule components better than rats did, although differences in discrimination were probably not responsible for the differences in contrast. The results suggest that behavioral contrast in rats may be a different phenomenon from behavioral contrast in pigeons. The results cannot be explained by current theories, which view contrast as the product of a single general process.