Extinction Schedule Research Articles

Relationship between impulsivity, hyperactivity and working memory: a differential analysis in the rat

BackgroundImpulsivity is a behavioural trait that comprises several distinct processes. It is a key feature of many psychopathologies such as mania, addictive disorders or attention deficit-hyperactivity disorders. To date, the aspects of impulsiveness involved in these pathologies have not yet been explicitly defined. In these disorders, sensation or drug seeking and cognitive deficits are closely related, but the nature of these relationships remains largely unknown. A new animal model of impulsiveness based on spontaneous inter-individual differences is proposed here to help clarify the relationship between characteristic aspects of impulsive-related pathologies.MethodsRats were divided into sub-groups according to their scores in three operant tasks with varying degrees of behavioural inhibition, timing and motor vs. cognitive impulsivity demands. These tasks included a fixed consecutive number schedule (ability to complete an action to receive a reinforcer), a multiple fixed-interval/extinction schedule of reinforcement (high level of responding), and a delayed reward task (delay discounting). In addition, measurements of locomotor responses to novelty and to amphetamine in a circular corridor, and working memory in an 8-arm radial maze were obtained.ResultsSubstantial behavioural inter-individual differences were observed in each task, whereas few inter-task relationships were found. Impulsive rats, as defined in a task requiring inhibition of premature responses, presented a higher increase in amphetamine-induced locomotion. Reduced working memory performance was only observed in hyperactive rats in an extinction schedule.ConclusionThis novel approach shows that distinct aspects of impulsiveness and hyperactivity can be expressed based on large inter-individual differences that vary from poorly to highly adapted behaviours ones in a normal population of rats. Inhibitory deficit was related to a higher response to psychostimulants a characteristic of rats predisposed to amphetamine self-administration and related to higher limbic dopaminergic activity, whereas working memory capacity was only related to hyperactivity. This approach allows for the identification of particular individuals presenting distinct behavioural characteristics of impulsive-related psychopathologies. These individuals may be of great interest in the modelling of these disorders and the exploration of their neurobiological bases.

The effects of acquisition training schedule on extinction and reinstatement of cocaine self-administration in male rats.

Partial reinforcement is known to increase resistance to extinction (Rn) relative to training with continuous reinforcement. This phenomenon, referred to as the partial reinforcement extinction effect, is one of the most robust in learning and conditioning studies. Experiment 1 investigated manipulations known to affect the partial reinforcement extinction effect and determined their possible relevance for drug use patterns. Male rats received intravenous cocaine self-administration training under partial reinforcement (FR-10) training or continuous reinforcement (FR-1) conditions with either a low (0.25 mg/kg infusion) or a high cocaine dose (1.00 mg/kg infusion). Animals were placed on an extinction (recurrent nonreward) schedule for 10 days (1-hr sessions) prior to being tested for cue-induced reinstatement (single 2-hr session). Experiment 2 involved acquisition of cocaine self-administration under FR-1 conditions of short training (15 days) or extended training (30 days) with a low dose (0.25 mg/kg infusion) or a medium dose (0.50 mg/kg infusion) of cocaine reward prior to extinction or reinstatement. Experiment 1 showed that rats trained with FR-10-high dose outcomes exhibited greater Rn than the remaining groups. Additionally, FR-10-high dose and FR-10-low dose rats were more likely to return to active drug seeking during the reinstatement test. In Experiment 2, rats trained under FR-1-medium dose conditions were more persistent during extinction following short acquisition training than comparable rats experiencing extended acquisition training. The reinstatement test was conducted following extinction, in which it was observed that overtraining under FR-1-medium dose reward schedules resulted in a decrease in the tendency to return to active drug seeking.

Effects of Behavioral History on Resistance to Change

Two experiments examined whether differential resistance to change would occur under identical variable-interval schedules as a function of a differential behavioral history. In Experiment 1, each of 3 pigeons first pecked at different rates under a multiple variable-ratio differential-reinforcement-of-low-rate schedule. In a subsequent condition, a multiple variable-interval variable interval schedule operated in the presence of the same training stimuli, during which there were three 4-session prefeeding probes each occurring during a different stage of training. Pecking generally was more resistant to change in the presence of the stimulus previously correlated with the differential-reinforcement-of-low-rate schedule, and this result was more reliable during the earlier probes. In Experiment 2, each of 3 pigeons first was exposed to a multiple variable-interval extinction schedule, after which a variable-interval schedule replaced extinction. During 3 probes, each conducted during a different stage of multiple variable-interval variable-interval schedule training, food was delivered response independently between components. In the first probe, pecking was more resistant to change for each pigeon under the variable-interval schedule that had been in effect longer, but during the final 2 probes this differential resistance was absent. The present results, therefore, show that under identical schedules a behavioral history can influence resistance to change differentially, but that these history effects tend to dissipate with continued exposure to the identical schedules.

The effects of continuous and partial reward on the vigilance task performance of adults with attentional deficits: A pilot investigation

The effects of reward schedule (100% and 30%) and extinction on attention (reaction time to auditory stimuli) and frustration levels (pressure exerted on a response key) of 15 adults with attentional disorders and 21 normal adults were examined using a continuous performance task. We predicted, based on Douglas and Parry ((1994). Journal of Abnormal Child Psychology, 22, 281), that adults with attentional deficits would (a) perform similar to comparisons when rewarded on a continuous schedule, (b) exhibit higher levels of frustration when that continuous schedule was moved to an extinction schedule, and (c) experience more frustration than comparisons when rewarded on a partial schedule. Overall, adults with attentional deficits were slower to respond and their responses were more variable than typical comparisons across trials, similar to what is observed for children. Continuous reward resulted in poorer performance earlier in the reward phase and continued throughout an extinction phase. The frustration levels of adults with attentional deficits did not differ from comparisons across schedule conditions. Results are discussed in terms of the role of arousal in mediating responding to various schedules of reward.

Cue exposure in the treatment of resistant bulimia nervosa.

It was hypothesized that binge eating (bulimia nervosa [BN]) may be caused by the anticipatory and immediate anxiety associated with certain types of food. Consequently, an extinction schedule should reduce binge eating. Cue exposure was carried out with 6 bulimic women who had responded poorly or not at all to the usual pharmacologic or cognitive-behavioral treatments. Binge eating and vomiting were almost totally suppressed in the 6 patients. Symptom suppression was maintained at two follow-ups, one at 4-20 months and another at 2.5-3 years. Cue exposure may be effective with BN that is resistant to conventional treatments. The anxiety associated with food plays an important role in provoking and/or maintaining binge eating. Motivation to change is likely to be an important mediator.

Effects of a dopamine D3 receptor ligand, BP 897, on acquisition and expression of food-, morphine-, and cocaine-induced conditioned place preference, and food-seeking behavior in rats.

The present study addressed the role of dopaminergic D(3) receptors (D(3)R) in motivational processes in rats. The effects of the selective D(3)R partial agonist, BP 897 (0.25-1 mg/kg, i.p.), on the establishment and the expression of conditioned place preference (CPP) supported by food, morphine (4 mg/kg, s.c.), or cocaine (2 mg/kg, s.c.) were investigated using an unbiased, one-compartment, place-conditioning procedure. When administered alone, BP 897 (0.05-2 mg/kg, i.p.) did not support CPP; on the contrary, conditioned place avoidance (CPA) was observed at 1 mg/kg, suggesting that this dose of BP 897 could be perceived as aversive. When given before each cocaine injection during the conditioning phase, BP 897 (1 mg/kg) prevented the establishment of CPP, and a single administration of BP 897 (0.5 and 1 mg/kg) before the test session impaired the expression of cocaine CPP. In contrast, neither the establishment nor the expression of food- and morphine-CPP were significantly altered by BP 897 (up to 1 mg/kg), whereas the full but less selective D(3)/D(2)R agonists, 7-OH-DPAT (0.5-2 mug/kg, s.c.) and quinelorane (1 mug/kg, s.c.), prevented the acquisition of food CPP. In a within-session extinction schedule of lever pressing for food, BP 897 (0.06-2 mg/kg) was ineffective in potentiating response reinstatement induced by the noncontingent delivery of two food pellets, in contrast with quinelorane and 7-OH-DPAT where previous studies showed to be efficient in this respect (Duarte et al, 2003). These results indicate that BP 897 has no positive appetitive value on its own, and that a moderate degree of stimulation of D(3)R is not sufficient to modulate food-primed food-seeking behavior or alter incentive motivation for food, morphine, and/or their associated cues. However, D(3)R are likely involved in the perception of the rewarding value of cocaine and cocaine-paired cues. This suggests that the appetitive effects of cocaine are subserved by mechanisms different, at least in part, from those of morphine and food, and that D(3)R play a role only in the former.

Hyperactivity and impulsiveness in rats fed diets supplemented with either Aroclor 1248 or PCB-contaminated St. Lawrence river fish

We examined whether exposure to polychlorinated biphenyls (PCBs) around puberty would produce hyperactivity and impulsiveness in adult Sprague–Dawley rats. Randomly assigned groups consumed food containing environmental concentrations of Aroclor 1248, PCB-contaminated St. Lawrence River carp, or corn oil (control). All received operant training to a final multiple (mult) 120-s, fixed interval (FI), 5-min extinction (EXT) schedule. Pressing rates of both exposed groups for drops of water averaged more than 1.5 times that of controls, especially toward the end of the 120-s interval. This overactivity included bursts with short (≤0.5 s) interresponse times (IRTs), behavior characteristic of hyperactive boys and genetically hyperactive rats. The exposed groups also overreacted to the decreases in reinforcement density associated with transition to the final schedule. The results were interpreted in terms of the possible alterations in the animals’ reinforcement mechanisms and the possible neurotoxic effects of PCB exposure.

Superstitious responding and reinforcement rate under concurrent variable-interval extinction schedules

To examine superstitious responding, four pigeons key pecked under multiple concurrent variable-interval 45 s variable-interval 90 s concurrent variable-interval 90 s variable-interval 180 s schedules in the absence of a changeover delay. The two variable-interval 90 s schedules then were replaced by extinction, and key-peck responding during extinction was examined as a function of the prevailing reinforcement rate. During the first several sessions, extinction-key responding was maintained closer to baseline levels in the presence of the higher reinforcement rate, and this effect dissipated or even reversed with continued exposure to extinction. Although extinction-key responding generally decreased to near-zero levels after several sessions, in a few instances, it continued for 30 and 45 sessions. These results demonstrate how concurrent variable-interval extinction schedules can be used to investigate what often has been labeled superstitious responding.

Competition, superstition and the illusion of control

The effects of reinforcement schedule and competition on generating superstitious behaviors and beliefs were examined in 72 people. Superstition was induced by having participants respond to turn on a tone under a concurrent 2—lever Variable Interval (VI) Extinction (EXT) schedule. During the session, stimulus lights would occasionally be illuminated, although they did not signal any change in contingency. Attributing importance to the inactive lever, a pattern of switching between levers, or to the illumination of the lights were considered to be superstitious beliefs. Participants were either run alone or in pairs, and manipulation of the reinforcement schedule resulted in groups which were matched in probability of reinforcement, as well as in groups which were mismatched. Reinforcement schedule (VI 30” versus VI 60”) and competitive situation did not affect degree of superstitious belief, except when people were placed in a “winning” condition. However, Superstition was associated with participants’ belief in improved future performance and with participants’ perceived skill relative to their opponents. Results are discussed in terms of relationships between superstition, the illusion of control, and self-efficacy. Differences between experimentally-induced and commonly held superstitions are also discussed.

Developmental exposure to methylmercury alters behavioral sensitivity to d-amphetamine and pentobarbital in adult rats

Female rats were exposed to 0, 0.5, or 6.4 ppm methylmercury in their drinking water before mating, and throughout gestation and lactation. When the female offspring were 4–6 months old, they were trained to respond under a multiple differential reinforcement of high rate (DRH) 9:4 — Extinction schedule of reinforcement. No differences among exposure groups were apparent in steady-state behavior. Drug challenges were conducted with multiple doses of d-amphetamine, scopolamine, pentobarbital, haloperidol, and dizocilpine, drugs selected for their different pharmacological effects. The ED 50 values for amphetamine's reinforcement rate-reducing effects for the control, 0.5-, and 6.4-ppm groups were 3.1, 1.9, and 0.9 mg amphetamine/kg body weight, respectively, demonstrating an increased sensitivity to d-amphetamine in methylmercury-exposed rats. Rats in the 6.4-ppm group also demonstrated a relative insensitivity to pentobarbital. Further, these exposed rats exhibited an inverted U-shaped dose–effect curve under the pentobarbital dose–effect determination, while controls showed only a declining curve. Exposed rats did not respond differentially to haloperidol, scopolamine, or dizocilpine, suggesting specificity. The present data suggest an involvement of catecholaminergic and GABAergic activity in methylmercury's neurotoxicity.

Aging unmasks adverse effects of gestational exposure to methylmercury in rats

The consequences of developmental exposure to methylmercury on behavior in aged animals were investigated. Methylmercury exposure was arranged by placing 0, 0.5 or 6.4 ppm Hg in the drinking water of female rats at least 4 weeks before mating and continuing until post-natal (PN) day 16. Brain Hg concentrations in cohorts of low- and high-dose offspring were 0.5 and 9.1 ppm at birth and 0.04 and 0.52 ppm at weaning (described in another report). Lever pressing of female offspring was maintained under a Multiple Differential Reinforcement of High Rate 9:4 Extinction schedule of food reinforcement (Mult DRH 9:4 EXT). Under the DRH 9:4 schedule, a food reinforcer was delivered when nine responses occurred within 4 s. Under the Extinction schedule, responding had no programmed consequences. No exposure-related differences in reinforcement rate under the DRH schedule or discrimination between the DRH and extinction components were apparent initially. At 950 days of age, the overall response rates of controls had shown a gradual decline over the previous 500 days to about 80% of their beginning levels, but, otherwise, most controls were healthy. A gradual decline in the reinforcement rate began to appear in low- and high-dose rats at about 500 and 800 days of age, respectively. Microanalyses of the nine-response burst maintained by the DRH schedule revealed that the lever-press duration increased, the inter-response time (IRT) was unaffected, and the time between response bursts increased. Overall, the nine-response burst remained intact as a coherent response unit. The increased time between response bursts caused the decline in reinforcement rate. All rats displayed these effects as they aged, but the mercury-exposed rats did so sooner.

Inherited Hypohedonia Leads to Learned Helplessness: A Conjecture Updated

The reformulated learned helplessness theory posits depressogenic explanatory style (DES) as a vulnerability factor in depression. Early-life adversity has been suggested as the source of DES, but this is contradicted by empirically established features of the learned helplessness phenomenon itself and by the evidence for human resilience in the face of adversity. This article extends a conjecture first advanced by P. E. Meehl (1975) and argues that an inherited defect in hedonic capacity would be sufficient to produce DES by causing intermittent schedules of reinforcement to be experienced as extinction schedules, resulting in a pervasive and unremitting sense of helplessness. This proposed hedonic defect also provides a means for integrating the original and reformulated versions of learned helplessness theory and for integrating learned helplessness with emerging research and theory on the biology of vulnerability to depression.

The Role of Corticotropin-Releasing Factor and Corticosterone in Stress- and Cocaine-Induced Relapse to Cocaine Seeking in Rats

We have shown previously that footshock stress and priming injections of cocaine reinstate cocaine seeking in rats after prolonged drug-free periods (Erb et al., 1996). Here we examined the role of brain corticotropin-releasing factor (CRF) and the adrenal hormone corticosterone in stress- and cocaine-induced reinstatement of cocaine seeking in rats. The ability of footshock stress and priming injections of cocaine to induce relapse to cocaine seeking was studied after intracerebroventricular infusions of the CRF receptor antagonist D-Phe CRF12-41, after adrenalectomy, and after adrenalectomy with corticosterone replacement. Rats were allowed to self-administer cocaine (1.0 mg/kg/infusion, i.v) for 3 hr daily for 10-14 d and were then placed on an extinction schedule during which saline was substituted for cocaine. Tests for reinstatement were given after intermittent footshock (10 min; 0.5 mA) and after priming injections of saline and cocaine (20 mg/kg, i.p.). Footshock reinstated cocaine seeking in both intact animals and animals with corticosterone replacement but not in adrenalectomized animals. The CRF receptor antagonist D-Phe CRF12-41 blocked footshock-induced reinstatement at all doses tested in both intact animals and animals with corticosterone replacement. Reinstatement by priming injections of cocaine was only minimally attenuated by adrenalectomy and by pretreatment with D-Phe CRF12-41. These data suggest that brain CRF plays a critical role in stress-induced, but only a modulatory role in cocaine-induced, reinstatement of cocaine seeking. Furthermore, the data show that although reinstatement of cocaine seeking by footshock stress requires minimal, basal, levels of corticosterone, stress-induced increases in corticosterone do not play a role in this effect.

Chronic l-deprenyl treatment alters brain monoamine levels and reduces impulsiveness in an animal model of Attention-Deficit/Hyperactivity Disorder

Effects of chronic l-deprenyl administration on hyperactive behaviour and brain monoamine levels were studied in spontaneously hypertensive rats (SHR) and Wistar–Kyoto (WKY) rats. SHR were hyperactive, impulsive and had impaired sustained attention when tested with a multiple 2-min fixed interval (FI) 5-min extinction (EXT) schedule of reinforcement. Even low, 0.25 mg/kg, doses of chronically-administered l-deprenyl reduced the impulsiveness (bursts of responses with short interresponse times) of SHR, without altering the general hyperactivity or the impaired sustained attention. The drug had no effect on WKY behaviour. The levels of noradrenaline (NA), dopamine (DA), serotonin (5-hydroxytryptamine, 5-HT) and their metabolites, measured in neostriatum, nucleus accumbens and frontal cortex, showed that l-deprenyl effectively inhibited monoamine oxidase (MAO) activity. These results suggest that impulsiveness is a behavioural component that may be operating independent of the other components, like hyperactivity and deficient sustained attention, and that can be reduced by chronic MAO-B inhibition with l-deprenyl in this strain of rats. The positive effect of l-deprenyl on impulsiveness is discussed as due either to normalization of an asymmetric dopaminergic activity in the nucleus accumbens, or to a restoration of normal DA function in the prefrontal cortex.

Sex differences in operant discrimination behaviour in an animal model of attention-deficit hyperactivity disorder

The present study was aimed at determining whether the behaviour of the spontaneously hypertensive rat (SHR), an animal model of attention-deficit hyperactivity disorder (ADHD), showed sex differences parallel to those seen in ADHD children. The experimental protocol contained an operant discrimination task, a two-component multiple (mult) 2-min fixed interval (FI) 5-min extinction (EXT) schedule of water reinforcement, a reliable behavioural paradigm for testing activity levels, discrimination behaviour and impulsiveness. The results indicated that both male and female SHRs show some of the most important behavioural traits of ADHD. Both were hyperactive and showed discrimination problems in terms of a behavioural extinction deficit towards the end of the EXT component. Still their behaviour differed markedly, which was probably due to quite different underlying mechanisms. The behavioural characteristics of the female SHRs may be compatible with an attention-deficit interpretation, whereas the behavioural characteristics of the male SHRs may be due to a shorter than normal delay-of-reinforcement gradient. The present study strengthens the position of SHR as an animal model of ADHD for future studies that may elucidate details in the underlying neurobiological deficits and for testing various treatment strategies.

Inherited hypohedonia leads to learned helplessness: A conjecture updated.

The reformulated learned helplessness theory posits depressogenic explanatory style (DES) as a vulnerability factor in depression. Early-life adversity has been suggested as the source of DES, but this is contradicted by empirically established features of the learned helplessness phenomenon itself and by the evidence for human resilience in the face of adversity. This article extends a conjecture first advanced by P. E. Meehl (1975) and argues that an inherited defect in hedonic capacity would be sufficient to produce DES by causing intermittent schedules of reinforcement to be experienced as extinction schedules, resulting in a pervasive and unremitting sense of helplessness. This proposed hedonic defect also provides a means for integrating the original and reformulated versions of learned helplessness theory and for integrating learned helplessness with emerging research and theory on the biology of vulnerability to depression.

PERFORMANCE ON CONCURRENT VARIABLE‐INTERVAL EXTINCTION SCHEDULES

Six homing pigeons were trained on concurrent variable‐interval extinction schedules in a switching‐key procedure. The discriminative stimuli and associated schedules in operation were selected randomly after each switch and each reinforcer. More than 80 daily sessions were arranged in each of five experimental conditions that varied the reinforcer rate on the variable‐interval schedule. Behavior allocation remained nonexclusive even after extended training and did not change systematically as a function of the reinforcer rate. Both of these findings are predicted by a contingency‐discriminability description of choice and are incompatible with a generalized matching description.

60 Hz electric fields and incandescent light as aversive stimuli controlling the behavior of rats responding under concurrent schedules of reinforcement

Several reports have shown that animals will sometimes engage in behaviors that reduce their exposure to a 60 Hz electric field (E-field). The field, therefore, can function as an aversive stimulus. In other studies, the E-field at equivalent strengths failed to function as an aversive stimulus. The present experiment, using rats, demonstrates how factors other than field strength can influence whether a subject engages in behavior that reduces field exposure. The general design consisted of giving the rat a choice between two alternatives, one of which sometimes included an added stimulus. Each subject was trained to press each of two levers to obtain food. Pressing one lever was reinforced intermittently under a variable interval 2 min schedule (VI 2); pressing the other lever was reinforced by a second VI 2 schedule operating independently of the first. Under this concurrent schedule the rat spent 50% of the daily 50 min session responding to each of the levers, indicating that they were equally "valued." Next, while the schedules remained in effect, the first response to one of the levers turned on a 100 kV/m E-field which remained on until the rat pressed the other lever. The time spent responding under the schedule associated with the field was reduced by about 5-10%. When the procedure was changed so that no lever presses produced food, i.e., extinction, but the added stimulus contingency remained, the rats spent even less time in the presence of the field. Similar outcomes were observed during both the concurrent food or extinction schedules when incandescent light was used. Thus, both an E-field and incandescent light functioned as aversive stimuli, but the magnitude of the aversiveness was small. Aversiveness depended not only on stimulus intensity, but also on behavioral factors.

Intrinsic reinforcing properties of putatively neutral stimuli in an instrumental two-lever discrimination task

Four experiments examined the influence of a stimulus presented after one response in a two-lever choice task. In Experiment 1, food-deprived rats trained on a concurrent variable-interval extinction schedule responded more often on the extinction lever when such responding periodically produced a visual stimulus than when it did not. In Experiments 2 and 3, a similar signal-induced enhancement effect was found even when food was delivered randomly with respect to responding on both levers or when no food was presented. In Experiment 4, a response-contingent visual stimulus elevated responding to the lever on which it was presented, but an auditory cue suppressed responding. These findings indicate that visual stimuli may possess intrinsically reinforcing properties for rats.

Differential Outcome Effect Does Not Control Performance in a Two-Choice Discrimination Task with Visual Stimuli

Three experiments examined the extent to which a differential outcome effect controlled performance in a two-choice instrumental discrimination. In Experiment 1, hungry rats responded on a concurrent variable interval (VI) 60-s extinction (ext) schedule; food reinforcement was presented for responses to the VI 60-s component after a 5-s delay. When a 2-s localized visual stimulus (illumination of the response lever) was presented after the to-be-reinforced response on the VI lever, improved discrimination performance (i.e., more responding to the VI than the ext lever) compared to an unsignaled control condition was noted. In Experiment 2, the same visual stimulus presented after responses to the extinction lever retarded emergence of the discrimination compared to an unsignaled condition (Phase 1). A similar retardation of responding to the correct lever was noted in Phase 2 when the stimulus was a 2-s overhead light flash. In Experiment 3, rats with a 2-s response-contingent overhead light flash associated with the VI component learned to discriminate the VI from the ext component more readily than did a group of rats with no stimulus. However, a group of rats with the overhead light presented after incorrect responses to the ext component demonstrated retarded discrimination learning. These results suggest that a differential outcome effect does not control performance in a delayed reward discrimination task when visual stimuli are employed.