Intermittent Reinforcement Schedules Research Articles

Ethanol-reinforced behavior in P, NP, HAD and LAD rats: differential genetic regulation of reinforcement and motivation.

Genetic differences exist in whether animals find ethanol or other drugs positively reinforcing. Through the use of P, NP, HAD and LAD rats, animals selectively bred for high (P, HAD) or low (NP, LAD) ethanol consumption in a two-bottle choice procedure, the present results show that multiple factors determining ethanol consumption can be identified. Ethanol-preferring P rats consumed more ethanol across all conditions involving fixed ratio or progressive schedules than did non-preferring NP, preferring HAD or non-preferring LAD rats. In both non-preferring NP and preferring HAD rats, responding decreased similarly as fixed ratio size was increased. However, under a progressive ratio schedule, NP rats responded to higher break points than did HAD rats. In LAD rats, ethanol has not consistently been shown to be a reinforcer and these rats exhibited the lowest levels of responding for ethanol across all conditions. These findings extend the conditions under which P rats have been shown to consume substantial amounts of ethanol. In addition the results illustrate genetic differences with regard to the propensity of animals to maintain ethanol-reinforced behaviors under different schedules of intermittent reinforcement, and provide further evidence that ethanol drinking in a preference paradigm is not highly predictive of whether or to what extent ethanol may come to serve as a positive reinforcer. The results suggest that the phenomenon of ethanol drinking is a composite of at least three genetically independent factors governing different aspects of ethanol-seeking behavior: (1) a permissive factor allowing animals to readily overcome the aversive sensory effects of ethanol; (2) a reinforcement factor that determines whether a substance can come to serve as a positive reinforcer; and (3) a motivational factor related to the subjects' drive state with regard to drug seeking.

Operant Conditioning of the Gray Starling, Sturnus cineraceus

Gray Starlings Sturnus cineraceus are one of the most notorious agricultural pests in Japan. We trained three Gray Starlings by operant conditioning to peck a response key to obtain a piece of mealworm. Birds were carefully trained to get used to the experimental situation and new schedules were gradually introduced. All the birds were successfully trained to peck the key. Two of them were further trained by an intermittent reinforcement schedule. The techniques developed here to condition the Gray Starling should be useful for further research aimed to measure aversiveness of visual or auditory stimuli for bird scaring.

Schedule-induced stereotypy

The phenomena of the induction and entrainment of adjunctive behaviours was investigated in eight people with severe or profound mental retardation who exhibited stereotypic behaviours. The occurrence of collateral behaviours was investigated under extinction and massed reinforcer baselines and a variety of periodic or intermittent schedules of reinforcement for the performance of a simple experimental task. Seven of the eight subjects demonstrated evidence of schedule-induced stereotypic behaviour in the experimental conditions; five of the eight subjects also showed evidence of the entrainment of these behaviours within the first third of the interreinforcement interval. The results are discussed in relation to the analysis and intervention of problem behaviours in people with severe or profound mental retardation.

Reinforcing effects of enantiomers of N,N-dimethylamphetamine in squirrel monkeys

Reinforcing effects of the (+)- and (-)-enantiomers of N,N-dimethylamphetamine, a drug that is abused in humans, were studied in squirrel monkeys (Siamiri sciureus) trained under schedules of intermittent cocaine reinforcement. During training, each 30th response produced an injection of cocaine (fixed-ratio schedule), which was followed by a 1-min period during which lights were out and responses had no scheduled consequences (timeout). Sessions in which cocaine injections were scheduled alternated in an irregular sequence with sessions in which saline injections were scheduled. After training, response rates were well maintained by cocaine (response rates approximated 1.4 responses per second) but occurred relatively infrequently during sessions in which saline was injected (response rates approximated 0.3 responses per second). Doses of 10 to 56 micrograms/kg/injection of (+)-N,N-dimethylamphetamine maintained rates of responding significantly higher than those maintained by saline. The (-)-enantiomer did not maintain rates of responding that were higher than those maintained by saline when tested at doses up to 100 micrograms/kg/injection. These findings support previous results indicating that N,N-dimethylamphetamine will function as a reinforcer in laboratory animals. Further, they suggest that the significant reinforcing activity of this drug is restricted to the (+)-enantiomer.

Automated behavioral procedures for infant monkeys

A method is described for testing infant monkeys on a variety of operant tasks as soon as they can self-feed, typically within the first week of life. Each infant was housed during the 16-21-hour experimental session in a cage to which operant behavioral equipment was attached. Computer control of the experimental contingencies and data acquisition allowed a relatively large number of monkeys to be tested simultaneously, as well as detailed analysis of response parameters. Infant monkeys are capable of learning a number of tasks that assess learning and memory, including visual discrimination and reversal, simultaneous discrimination, and spatial and nonspatial matching to sample. Infant monkeys also perform like older animals on intermittent schedules of reinforcement. The long experimental sessions allowed determination of feeding pattern over the course of the night. Analyses of these variables have proved sensitive to the effects of developmental exposure to neurotoxicants such as lead and caffeine.

Measuring threshold of reinforcing brain stimulation by the method of constant stimuli

A new method for measuring threshold of reinforcing electrical brain stimulation is described and results of a parametric study using this method are presented. Two groups of rats were trained under a concurrent FR-CRF (fixed ratio-continuous reinforcement) schedule of reinforcement provided by electrical stimulation of the lateral hypothalamus. The invariant intermittent FR schedule of reinforcement was used to maintain a baseline of behavior while a superimposed concurrent CRF schedule was used to measure reinforcement magnitude by varying the intensity of the CRF stimulus between zero and a maximum. Increasing and decreasing stimulus intensity on the CRF schedule leads to a gradual disappearance, respectively reappearance, of post-reinforcement pauses (PRPs) on the concurrent FR schedule, providing a criterion for changeover in schedule control, and thus, for threshold of reinforcement. To illustrate the measurement of threshold according to psychophysical requirements of the Method of Constant Stimuli, different CRF intensities were given in a randomized order. In one group of animals FR and CRF stimuli were given through the same electrode. Another group received FR and CRF stimuli through separate electrodes implanted in different hemispheres of the brain. For both groups the duration of the PRP was used as the dependent variable. The data of both groups showed a high negative correlation between the intensity of the CRF stimulus and the duration of the PRP, which is consistent with the results of experiments in which a Method of Limits procedure was used. On the basis of this relationship between CRF current intensity and PRP duration a threshold for reinforcing brain stimulation was calculated.

Quantification of operant behavior

The study of performance on intermittent schedules of reinforcement has proved to be a powerful tool in the fields of experimental psychology and behavioral pharmacology and presently is proving equally valuable in behavioral toxicology. The ability to specify precisely contingencies of reinforcement allows a careful and detailed quantification of performance. Intermittent schedules of reinforcement may be used in behavioral toxicology in a number of ways. A baseline of performance may be established and utilized to monitor acute effects or to track effects of chronic exposure to a toxic agent. Alternatively, schedules of reinforcement may be used in experiments requiring group comparisons, where both terminal performance and acquisition of performance may be of interest. The use of different schedules, generating different rates and patterns of performance, may be compared to elucidate behavioral mechanisms. Use of computers for schedule control and data acquisition allows a detailed analysis of performance, thus increasing the probability of the detection of subtle effects.

A comparison of two behaviour reduction procedures: traditional extinction alone and interpolated reinforcement followed by extinction.

Extinction responding of children with severe retardation was compared when one component of a multiple schedule (VR-20/VR-20) was interpolated with one session of additional reinforcement. Either variable ratio 10 (VR-10), variable ratio 5 (VR-5) or continuous reinforcement (CRF) were interpolated in one component prior to extinction in both. Interpolated reinforcement resulted in an extinction process at least as efficient as traditional methods. Total responding in the VR-10 and VR-5 components was less than in the comparable unchanged components. Results support and extend research with animals and humans. Results also suggest the feasibility of using such a procedure to gain control of behaviours that are controlled by intermittent schedules of reinforcement, are under multiple or inconsistent stimulus control or occur infrequently.

Opiate antagonists and rewarding brain stimulation

This review examines the literature on the effects of opiate antagonists on brain stimulation (ICSS) reward. Antagonists should have predictable effects if endogenous opioids modulate ICSS. Naloxone is the antagonist most often used, and it has produced inconsistent results in some ICSS paradigms. When schedules of intermittent reinforcement are used, however, naloxone reliably reduces the rate of responding. It reverses the effects of opiate agonists on ICSS behavior, and it also attenuates the effects of psychomotor stimulants, such as amphetamine. The results produced by naloxone are consistent with a modulatory effect of endogenous opioid systems on reward, and suggest that the opiate and dopamine systems together exert significant control over ICSS. Further research is needed to characterize better the actions of the antagonists on ICSS behavior, and productive research directions are proposed. Data obtained in future studies might suggest how the endogenous opioid systems modulate both natural and brain stimulation reward.

Bilateral augmentation of dopaminergic and serotonergic activity in the striatum and nucleus accumbens induced by conditioned circling

The involvement of dopaminergic (DA) and serotonergic (5-HT) systems in circling was assessed by determining the neurochemical correlates of circling induced and maintained by two different schedules of water reinforcement. The conditioned circling paradigm was employed in an attempt to replicate reports that levels of DA and 3,4-dihydroxyphenylacetic acid (DOPAC) were increased in the striatum and nucleus accumbens septi (NAS) contralateral to the direction of circling. Rats trained to circle using a continuous schedule of reinforcement did not exhibit any changes in concentrations of DA, DOPAC, or homovanillic acid (HVA). Bilateral increases in 5-HT concentrations were observed in the striatum. Use of an intermittent schedule of reinforcement (FR-2) produced higher rates of circling. In rats maintained on the FR-2 schedule, no changes in DA or its metabolites were observed in the striatum. The ratio of HVA to DA was, however, increased bilaterally, suggesting a bilateral augmentation of DA utilization. Concentrations of DA were lower in the NAS contralateral to direction of turning. While NAS levels of HVA were elevated bilaterally when compared to non-circling controls, HVA was lower in the NAS contralateral to the direction of circling. DA utilization, as estimated by HVA: DA ratios, was increased bilaterally in the NAS. None of the measures of DA activity within the olfactory tubercle (OT) were influenced by circling. Turnover of 5-HT, as estimated by the ratio of 5-HT to 5-hydroxyindoleacetic acid (5-HIAA), was increased bilaterally in the striatum, NAS, and OT.(ABSTRACT TRUNCATED AT 250 WORDS)

A pitfall for the expectancy theory of human eyelid conditioning.

Two simple eyeblink conditioning experiments with random intermittent reinforcement schedules were performed. In Experiment 1, subjects had to rate their expectancy for an unconditioned stimulus (US) on a seven-level scale prior to each trial. As anticipated, expectancy for US increased with a successive conditioned stimulus (CS) alone, and decreased with successive CS-US pairings. However, Experiments 1 and 2 showed that the frequency of eyeblink conditioned responses (CRs) evolved in a direction opposite to that of expectancy changes: CRs increased, whereas expectancy for US decreased, and vice versa. The possible effect of sensitization on eyeblink response was ruled out by the lack of a run effect in an unpaired control group in Experiment 2. These results tend to disconfirm the expectancy theory of conditioning. Although they were explicitly predicted by the conventional "strength" theory of conditioning, an alternative interpretation is proposed within a cognitive framework.

Exchange, Perceived Alternatives, and Reinforcement Schedule: Cohesiveness Among Unmarried Couples

Abstract The extent to which partners of premarital dyads (N = 60 couples) believe they could just as easily have their needs met outside of the relationship was predicted and found to be associated with couple cohesiveness. As also predicted, this relationship was strongest when the exchange distribution between partners was imbalanced. Partner predictability was used as an indicator of the extent to which a couple tended to be on a continuous as opposed to an intermittent reinforcement schedule. I hypothesized that intermittent reinforcement (low predictability) is a better predictor of cohesiveness than continuous reinforcement (high predictability). The zero-order results showed a strong relationship in the direction opposite to that hypothesized. The longer the partners had been together, however, the weaker this relationship became, reaching near zero among the couples that had been together the longest.

Overshadowing of responding on ratio and interval schedules by an independent predictor of reinforcement

This experiment replicated previous demonstrations that interposing a brief stimulus between reinforced responses and the presentation of the reinforcer reduces responding maintained by intermittent reinforcement schedules. Furthermore, we could find no significant difference between the relative size of the reduction during training on ratio and interval schedules when the predictive significance of the response and stimulus was controlled by a yoking procedure.

Self-administration of methionine enkephalin into the nucleus accumbens

Microinfusions of the endogenous opiate neurohumor, methionine enkephalin, into the nucleus accumbens initiated a reinforcing stimulus in a dose-related manner. The reinforcing nature of this intracranial self-administration was evaluated with intermittent schedules of reinforcement and a two-lever discrimination procedure. Opiate receptors are likely responsible for the initiation of this reinforcing stimulus since naloxone effectively blocked self-administration. These data suggest the mediation of opiate reinforcement through interactions with opiate receptors in brain regions outside the ventral tegmental area, questioning the current dopamine hypothesis for the initiation of these reinforcement processes.

Schedule-induction of nicotine self-administration

Nicotine, although assumed to be an important factor in maintaining the use of tobacco products, has produced equivocal results when tested in drug self-administration studies using standard procedures with laboratory animals. Several recent studies however, have demonstrated nicotine self-administration using a procedure of schedule-induction. Schedule-induced behaviors occur as an adjunct to behavior controlled by an intermittent schedule of reinforcement and are thus not under control of the scheduled contingencies. Using schedule-induction procedures; oral, intravenous and inhaled self-administration of nicotine has been shown in rats, both rats and rhesus monkeys, and humans respectively. Although the self-administration of some doses of nicotine occurred without a concurrent schedule of intermittent reinforcement, schedule-induction results in responding maintained by lower doses of the drug and much more rapid initiation of self-administration. The result of such studies suggest an interaction between environmental factors, such as an intermittent schedule of other reinforcers, and nicotine's pharmacological effects. This interaction may be important in understanding the etiology and maintenance of human tobacco use.

ニホンザルの見本合わせ行動における弁別刺激コンテクスト

Transfer of matching-to-sample behavior by Japanese monkeys was investigated in a freeoperant situation in order to reexamine stimulus control of the behavior. Following the author's earlier study (Kojima, 1979), the same two monkeys were retrained on a pattern matching-to-sample and shifted to three successive transfer situations. For a proper evaluation of transfer effects, an intermittent reinforcement schedule (VI 60s) for a correct matching trial maintained the rate of reinforcement relatively low and kept it almost unchanged in the transfer situations. The results showed that both monkeys attained successful tranfser of matching behavior in the second (intradimensional) transfer situation, but exhibited performance disruption in the third (extradimensional) transfer situation using color stimuli. The present findings suggested that they are capable of learning a concept of “sameness”, but with little generality.

Effects of motivational variables and contextual stimuli on schedule-induced behavior

Hooded rats were trained in two experiments on either DRL 10- or 30-sec schedules of reinforcement to assess the effects of deprivation level, deprivation state, and available environmental stimuli on schedule-induced behavior. The reinforcer was an 8% sucrose solution, and the adjunctive inducing stimuli provided were air tubes and wood blocks. The effects of water deprivation level were studied in Experiment 1 by comparing water-deprived and nondeprived conditions in an ABA reversal procedure. Deprivation state, hunger or thirst, was assessed similarly in Experiment 2. In both studies, air licking developed spontaneously as an excessive, schedule-induced adjunctive activity only under thirst states, and diminished with decreased deprivation levels. Wood gnawing did not develop as an excessive or schedule-induced behavior but occurred more frequently under hunger states. The findings suggest that when state-appropriate environmental stimuli are provided during intermittent reinforcement schedules, relevant, redirected behaviors rather than irrelevant displacement activities emerge.

What is meant by the term “schedule-induced,” and how general is schedule induction?

The term “schedule-induced” implies that the overall frequency of a behavior is greater in the presence of an intermittent schedule of reinforcement than in the absence of such a schedule. Consequently, the occurrence of interreinforcement behavior is not in itself sufficient evidence of schedule induction: a test of induction requires comparison between an intermittent-schedule condition and a nonschedule baseline. The relative merits of different types of nonschedule baseline are examined, and it is concluded that the best test of schedule-induction involves both an extinction and a massed-reinforcer baseline. A working definition of schedule-induction is suggested on this basis. Studies purporting to show schedule induction of activities other than drinking are critically reviewed, and it is concluded that schedule induction may be less general than is usually supposed. It may therefore be more fruitful to seek an explanation of schedule-induced drinking which focuses specifically on the interaction between food and water ingestion in the rat, rather than an explanation involving concepts such as stress, frustration, or arousal.

The ambiguous-cue problem: The effects of reduced similarity between trials and the schedule of reinforcement

In a series of seven experiments pigeons were trained on the two-choice ambiguous-cue problem that contains PA and NA trials. On PA trials choice of P (the positive stimulus) was reinforced and choice of A (the so-called ambiguous stimulus) was not reinforced; on NA trials choice of A was reinforced and choice of N (the negative stimulus) was not reinforced. When the two types of trials were distinguished by only the presence of P or N and each trial was terminated by one response, subjects quickly attained high levels of performance on NA trials but showed only chance levels of performance on PA trials. PA performance was markedly improved when a variety of additional stimuli distinguished the two types of trials and when a variety of schedules of intermittent reinforcement were employed; these manipulations had little effect on NA performance.

Extinction and dopamine receptor blockade after intermittent reinforcement training: failure to observe functional equivalence.

Response decrements in an operant task produced by either extinction or by the dopamine receptor blocker pimozide were examined in three experiments which employed intermittent reinforcement schedules. In contrast to the congruency between these treatments previously observed following continuous reinforcement training, treatment with pimozide was markedly more effective than extinction in decreasing performance after training with variable interval, fixed interval, and fixed ratio reinforcement. The two treatments also produced substantially different patterns of responding. A shift from extinction to pimozide did not alter the progressive decline in response rate over days, but a shift from pimozide to extinction caused a pronounced increase of performance. These results indicate that the pimozide and extinction treatment did not produce functionally equivalent effects, and that the role of dopamine on reward processes should not be inferred from comparisons between pimozide and extinction.