Lever-press Training Research Articles

Goal-Directed Action Is Initially Impaired in a hAPP-J20 Mouse Model of Alzheimer's Disease.

Cognitive-behavioral testing in preclinical models of Alzheimer's disease has failed to capture deficits in goal-directed action control. Here, we provide the first comprehensive investigation of goal-directed action in a transgenic mouse model of Alzheimer's disease. Specifically, we tested outcome devaluation performance in male and female human amyloid precursor protein (hAPP)-J20 mice. Mice were first trained to press left and right levers for pellet and sucrose outcomes, respectively (counterbalanced), over 4 d. On test, mice were prefed one of the outcomes to satiety and given a choice between levers. Devaluation performance was intact for 36-week-old wild-types of both sexes, who responded more on the valued relative to the devalued lever (Valued > Devalued). By contrast, devaluation was impaired (Valued = Devalued) for J20 mice of both sexes, and for 52-week-old male mice regardless of genotype. After additional lever press training (i.e., 8-d lever pressing in total), devaluation was intact for all mice, demonstrating that the initial deficits were not a result of a nonspecific impairment in reward processing, depression, or locomotor activity in J20 or aging mice. Follow-up analyses revealed that microglial expression in the dorsal CA1 region of the hippocampus was associated with poorer outcome devaluation performance on initial, but not later tests. Together, these data demonstrate that goal-directed action is initially impaired in J20 mice of both sexes and in aging male mice regardless of genotype, and that this impairment is related to neuroinflammation in the dorsal CA1 hippocampal region.

Outcome-selective reinstatement is predominantly context-independent, and associated with c-Fos activation in the posterior dorsomedial striatum

Research from human and animal studies has found that after responding has been successfully reduced following treatment it can return upon exposure to certain contexts. An individual in recovery from alcohol use disorder, for example, might relapse to drinking upon visiting their favourite bar. However, most of these data have been derived from experiments involving a single (active) response, and the context-dependence of returned responding in situations involving choice between multiple actions and outcomes is less well-understood. We thus investigated how outcome-selective reinstatement – a procedure involving choice between two actions and outcomes – was affected by altering the physical context in rats. In Experiment 1, rats were trained over 6 days to press a left lever for one food outcome (pellets or sucrose) and a right lever for the other outcome. Then, rats received an extinction session in either the same context (A) as lever press training, or in a different context (B). Rats were tested immediately (5 min) after extinction in Context A or B such that there were four groups in total: AAA, ABB, ABA, and AAB. Reinstatement testing consisted of one food outcome being delivered ‘freely’ (i.e. unearned by lever pressing and unsignalled by cues) to the food magazine every 4 min in the following order: Sucrose, Pellet, Pellet, Sucrose. Selective reinstatement was considered intact if pellet delivery increased pressing selectively on the pellet lever, and sucrose delivery selectively increased pressing on the sucrose lever. This result (Reinstated > Nonreinstated) was observed for rats in group AAA and ABB, but not rats in groups ABA and AAB. Experiment 2 was conducted identically, except that rats received two extinction sessions over two days and tested one day later. This time, all groups demonstrated intact outcome-selective reinstatement regardless of context. Analysis of c-Fos expression in several brain regions revealed that only c-Fos expression in the posterior dorsomedial striatum (pDMS) was related to intact reinstatement performance. Overall, these results suggest that outcome-selective reinstatement is predominantly context-independent, and that intact reinstatement is related to neuronal activity in the pDMS.

Does DRO type matter?: Cycle versus resetting contingencies in eliminating responding

Following lever-press training on variable-interval 30-s schedules, rats were exposed to three types of schedules designed to eliminate lever pressing. The first two were variations on what is called a differential-reinforcement-of-other-behavior (DRO, “zero rate”, or [target response] omission schedule) schedule. Under both variations, reinforcers were scheduled to occur in different conditions after either fixed or variable inter-reinforcer intervals (IRIs). Under one variation each lever press reset the time interval (i.e., “resetting DRO”) and under the other a reinforcer delivery scheduled at the end of an IRI was cancelled by the first response during the IRI (i.e., “cycle DRO”). In another condition reinforcers were delivered independently of responding after fixed or variable time periods. Each of the DRO procedures reduced response rates quickly and to near zero across ten sessions. The time schedules also reduced responding, albeit at a slower rate. The results extend the analogy of omission training to freeoperant avoidance to shock-deletion avoidance schedules.

Dietary effects on the determinants of food choice: Impulsive choice, discrimination, incentive motivation, preference, and liking in male rats

The current study sought to understand how long-term exposure to diets high in saturated fat and refined sugar affected impulsive choice behavior, discrimination abilities, incentive motivation, food preferences, and liking of fat and sugar in male rats. The results showed that 8 weeks of dietary exposure impaired impulsive choice behavior; rats exposed to diets high in processed fat or sugar were more sensitive to changes in delay, a marker of impulsivity. For the high-fat group, these deficits in impulsive choice may stem from poor time discrimination, as their performance was impaired on a temporal discrimination task. The high-fat group also showed reduced magnitude sensitivity in the impulsive choice task, and they earned fewer rewards during lever press training indicating potentially reduced incentive motivation. The high-fat group also developed a preference for high-fat foods compared to the chow and high-sugar group who both preferred sugar. In contrast, dietary exposure did not alter the liking of fat or sugar as measured by a taste reactivity task. Together, the results suggest that the alterations in impulsive choice, time discrimination, incentive motivation, and food preferences induced by consumption of a high-fat diet could make individuals vulnerable to overeating, and thus obesity.

Stress as a context: Stress causes relapse of inhibited food seeking if it has been associated with prior food seeking

Three experiments with rats explored the hypothesis that inhibited food-seeking can be reinstated by stress if stress has been part of the context of earlier food-seeking. In all experiments, rats first learned to lever press for sucrose pellets and then had the response inhibited through extinction (where responding no longer yielded sucrose pellets). In a final test, inhibited responding was tested after exposure to a stressor or not. Previous research indicates that stress during testing does not normally reinstate extinguished food-seeking, although it reliably does so when animals are responding for drugs. In Experiment 1, stress caused a reinstatement of food seeking if and only if the rats had been exposed to stressors prior to sessions of lever press training. In Experiment 2, a new stressor that had not been associated with response acquisition also caused reinstatement if other stressors had been associated with response acquisition. Experiment 3 then established that stressors must be associated with the acquisition of lever pressing, rather than extinction, in order to allow a stressor to cause relapse of extinguished food seeking. The results support the view that stress can cause relapse of inhibited food seeking if it has been part of the context of original food seeking. The effect is therefore an example of the ABA renewal effect in which inhibited responding recovers after extinction when the response is returned to its training context. Implications for understanding relapse to overeating and other “addictive” behaviors are discussed.

A SYSTEMATIC REPLICATION OF SKINNER (1938) USING A HOT AIR BLAST AS THE PUNISHER

The present study was a systematic replication of the classic study by Skinner (1938) using a hot air blast (HAB) as the punisher. After lever press training, six rats underwent two extinction sessions. During the initial 10 min of extinction in the first session, half of the subjects received a HAB for every lever press (punishment). Subjects that received punishment made fewer lever presses during first extinction session, but the total number of responses in both groups was equivalent by the end of the second extinction session. The present data corroborate the findings of Skinner (1938) that the punishment may have partial and temporary effects in certain contexts. We discuss the ways in which the elicitation of competitive responses, the intensity and nature of the stimulus, and the duration of exposure to the aversive contingency are important factors that may explain the divergent results in the literature.Keywords: punishment, replication, competitive responses, suppression, rats, hot air blast

Ethanol Seeking by Long Evans Rats Is Not Always a Goal-Directed Behavior

BackgroundTwo parallel and interacting processes are said to underlie animal behavior, whereby learning and performance of a behavior is at first via conscious and deliberate (goal-directed) processes, but after initial acquisition, the behavior can become automatic and stimulus-elicited (habitual). With respect to instrumental behaviors, animal learning studies suggest that the duration of training and the action-outcome contingency are two factors involved in the emergence of habitual seeking of “natural” reinforcers (e.g., sweet solutions, food or sucrose pellets). To rigorously test whether behaviors reinforced by abused substances such as ethanol, in particular, similarly become habitual was the primary aim of this study.Methodology/Principal FindingsMale Long Evans rats underwent extended or limited operant lever press training with 10% sucrose/10% ethanol (10S10E) reinforcement (variable interval (VI) or (VR) ratio schedule of reinforcement), or with 10% sucrose (10S) reinforcement (VI schedule only). Once training and pretesting were complete, the impact of outcome devaluation on operant behavior was evaluated after lithium chloride injections were paired with the reinforcer, or unpaired 24 hours later. After limited, but not extended instrumental training, lever pressing by groups trained under VR with 10S10E and under VI with 10S was sensitive to outcome devaluation. In contrast, responding by both the extended and limited training 10S10E VI groups was not sensitive to ethanol devaluation during the test for habitual behavior.Conclusions/SignificanceOperant behavior by rats trained to self-administer an ethanol-sucrose solution showed variable sensitivity to a change in the value of ethanol, with relative insensitivity developing sooner in animals that received time-variable ethanol reinforcement during training sessions. One important implication, with respect to substance abuse in humans, is that initial learning about the relationship between instrumental actions and the opportunity to consume ethanol-containing drinks can influence the time course for the development or expression of habitual ethanol seeking behavior.

Facilitation of extinction of operant behaviour in C57Bl/6 mice by chlordiazepoxide and D-cycloserine.

Effects on the extinction of GABAergic drug, chlordiazepoxide (CDP), and glutamatergic drug, D: -cycloserine (DCS), in C57BL/6 mice were compared. Following a palatability test (Experiment 1), Experiments 2-6 involved food-reinforced lever press training followed by extinction sessions at 1- or 4-day intervals. The effects of drugs were examined. Experiment 7 involved a two-lever task. CDP did not affect food palatability (Experiment 1), but facilitated extinction when administered prior to extinction sessions via intracerebral (Experiment 2) or peripheral administration at 1-day (Experiments 3-7) or 4-day intervals (Experiment 6). Reducing the amount of training prior to extinction reduced the delay in the effect of CDP typically seen, and CDP had a larger effect in early sessions on mice that had received less training (Experiment 3). There was some evidence that CDP could be blocked by flumazenil (Experiment 4), and CDP withdrawal reversed extinction facilitation (Experiments 5 and 7). With 4-day intervals, DCS administered immediately following extinction sessions, or pre-session CDP, facilitated extinction with 48-trial sessions (experiment 6B). With six-trial sessions, the co-administration of post-session DCS enhanced facilitation produced by pre-session CDP (experiment 6A). Finally, CDP facilitated extinction in a dose-related fashion following training on a two-lever food-reinforced task (Experiment 7). The findings are consistent with the hypotheses that two neurotransmitter systems have different roles in operant extinction and that glutamatergic systems are involved in extinction learning and GABAergic systems involved in the expression of that learning. This parallels findings with extinction following Pavlovian conditioning, which has been more extensively investigated.

Excitatory backward conditioning in an appetitive conditioned reinforcement preparation with rats

Four experiments were conducted to examine appetitive backward conditioning in a conditioned reinforcement preparation. In all experiments, off-line classical conditioning was conducted following lever-press training on two levers. Presentations of a sucrose solution by a liquid dipper served as an unconditioned stimulus (US) and two auditory stimuli served as conditioned stimuli (CSs); one was paired with the US in either a forward (Experiment 1a) or a backward (Experiments 1b, 2, and 3) relationship, and the other served as a control CS, which was not paired with the US. In testing, each lever-press response produced a presentation of one of the CSs instead of appetitive reinforcers. The response to a lever was facilitated, compared to the response to another lever, when the response produced the backward CS presentation as well as when it produced the forward CS presentation; that is, the backward CS served as an excitatory conditioned reinforcer.

The long-term effect of high- and low-rate responding histories on fixed-interval responding in rats.

Tell rats were given extended lever-press training on a fixed-interval (FI) 30-s food reinforcement schedule from the outset or following exposure to one or two previous reinforcement schedules. For 4 rats the previots schedule was either fixed-ratio 20, which generated high response rates, or differential-reinforcement-of-low-rate 20 s, which produced low response rates. For 4 additional rats the extended training on FI 30 s was preceded by experience with two schedules: fixed-ratio 20 followed by differential-reinforcement-of-low-rate 20 s; or the same two schedules in the reverse order. Fixed-interval response rates were initially affected by the immediately preceding schedule, but after 80 to 100 sessions, all traces of prior schedule history had disappeared. The results also showed no long-term effect of schedule history on the interfood-interval patterns of responding on the FI 30-s schedule. These results support one of the most central tenets of the experimental analysis of behavior: control by the immediate consequences of behavior.

Oral operant ethanol self-administration in 5-HT1b knockout mice

The present experiment examined oral ethanol self-adminstration in 5-HT1b knockout (KO) mice and 5-HT1b wild-type (WT) control mice using a continuous access operant procedure. After lever press training, adult 5-HT1b KO and 5-HT1b WT mice were placed in operant chambers on a 23 h per day basis with access to food (FR1), 10% v/v ethanol (FR4), and water from a sipper tube. KO mice displayed higher rates of responding on the ethanol-associated lever compared to WT mice. KO mice also consumed greater amounts of water. Food responding was the same in both genotypes. Following 30 sessions, ethanol concentration was altered every 5 days. Response patterns were determined using 0, 5, and 20% v/v ethanol concentrations. Ethanol responding (0, 5, 10, and 20% v/v) was also examined after the addition of 0.15% saccharin. KO mice and WT mice showed similar response rates for all ethanol concentrations. Since KO mice showed greater levels of ethanol responding only for unsweetened 10% v/v ethanol, and showed modest ethanol self-adminstration overall, the present results are not consistent with the notion that 5-HT1b KO have a generally greater preference for ethanol than 5-HT1b WT mice.

NMDA receptor antagonists prevent conditioned activation of intracranial self-stimulation in rats

Rats with bipolar electrodes implanted unilaterally into the ventral tegmental area were trained to lever-press for response contingent electrical stimulation (continuous reinforcement). After preliminary lever-press training, two types of daily sessions were held on 10 consecutive days: type T +, during which current intensity was set at the Threshold level and each response was accompanied by the visual signal (stimulus lights above the lever briefly went off); and type ST −, during which current was set at the Sub Threshold level and there were no visual stimuli. On day 11, combination of the subthreshold current intensities and stimulus lights previously associated with the threshold stimulation (session type ST +) resulted in significantly elevated response rates compared to the performance under the subthreshold current without visual stimuli (session type ST −). This effect was dose dependently blocked by competitive NMDA receptor antagonist (±)-CPP ((±)-3-(2-carboxypiperazin-4-yl)-propyl-1-phosphonic acid) and CGS 19755 ( cis-4-(phosphonomethyl) piperidine-2-carboxylic acid). The present findings suggest that the activation of intracranial self-stimulation induced by a conditioned visual stimulus is dependent on the NMDA receptor functioning.

Classical conditioning of electrical self-stimulation of ventral tegmental area to brief visual stimuli in rats

Two groups of rats with bipolar electrodes implanted unilaterally into the left ventral tegmental area were trained to lever-press for the response contingent electrical stimulation. After preliminary lever-press training, two types of daily sessions were held on 10 consecutive days: type T +, current intensity set at the T hreshold level and each response was accompanied by the visual signal (stimulus lights above the lever briefly went off); and type ST −, current set at the S ub T hreshold level and no visual stimuli. On day 11, combination of the subthreshold current intensities and stimulus lights previously associated with the threshold stimulation (session type ST +) resulted in a significantly elevated response rates compared to the performance under the subthreshold current without visual stimuli (session type ST −). The extinction and recovery of this phenomenon are also demonstrated.

The effect of quisqualic acid-induced lesions of the nucleus basalis magnocellularis on latent inhibition.

Latent inhibition (LI) is a reduction in the rate of acquisition of a Pavlovian conditioned response that results from prior nonreinforced preexposure to a conditioned stimulus (CS). LI has been suggested to reflect the operation of mechanisms involved in stimulus selection for subsequent cognitive processing. The present experiment was conducted to assess the effect of bilateral lesions of the nucleus basalis magnocellularis (NBM) on LI employing a conditioned emotional response paradigm. Bilateral lesions of the NBM were produced by administration of 0.12 M quisqualic acid and resulted in decreased cortical acetylcholinesterase staining, as well as a 40% reduction in cortical choline acetyltransferase activity. Following lever press training, preexposed animals received 40 presentations of a 60-s tone CS. Nonpreexposed animals received no tone presentations. Acquisition of conditioned suppression was then assessed over the course of 4 tone-shock (0.6 mA, 0.5 s) pairings. Control, preexposed animals displayed a retarded rate of acquisition in comparison to nonpreexposed controls, thereby demonstrating that the parameters used in the present experiment produced LI. In contrast, lesioned animals preexposed to the CS acquired conditioned suppression as readily as nonpreexposed lesioned animals. However, the acquisition of conditioned suppression in both lesioned groups was found to be similar to that displayed in the preexposed control group. This pattern of results was interpreted as being attributable to a lesion-induced impairment in the ability to maintain stimulus processing, rather than a deficit in the ability to filter a stimulus.

Lead/ethanol interactions I: Rate-depressant effects

Adult male rats were exposed to a diet containing 500 ppm added lead as lead acetate (group lead-diet) or a control diet containing no added chemicals (group control-diet) for 61 days prior to commencing fixed-ratio 32 (FR 32) lever press training for water reinforcement. After steady state responding was achieved, all animals received serial administrations of acute doses of ethanol prior to the daily training session. Specifically, lead-diet and control-diet rats received i.p. injections of .25, .5, .75, 1.0, and 1.25 g/kg ethanol, in ascending order, alternating daily with injections of saline. The results revealed a dose-dependent rate-depressant effect, with higher doses of ethanol producing more behavioral suppression than lower doses for both groups. In addition, at the dose of 1.0 g/kg it was observed that the suppressive effects of ethanol on schedule-controlled responding were reduced among lead-treated animals relative to controls. These data are discussed in terms of lead-induced attenuation of the pharmacologic effects of ethanol.

Intracranial self-stimulation motivates treadmill running in rats

Most animal running models have traditionally used aversive motivators to induce exercise tasks. This study demonstrates treadmill running motivated by reinforcement of intracranial self-stimulation (ICSS), providing an alternative model with which to study physiological responses to exercise. Twenty-nine male Sprague-Dawley rats were stereotaxically implanted with bipolar electrodes aimed at the ventral tegmental area of the brain. After 7 days of operant lever-press training for ICSS, rats that pressed at least 50 presses/min were randomly divided into three conditions: exercise-reinforcing brain stimulation (Ex-St), exercise-aversive shock (Ex-Sh), and sedentary controls (C). Ex-St and Ex-Sh ran for 30 min at 25 m/min at 5% grade for 2 wk with ICSS and electric shock as the motivator, respectively, while C did not run. At the end of 2 wk, Ex-St and Ex-Sh performed an endurance run. Results show that Ex-St ran longer than Ex-Sh [63 +/- 10 vs. 42 +/- 10 (SD) min; P less than 0.05]. HR was higher in Ex-St than in C (P less than 0.05). Rectal temperature increased similarly in both exercise groups. This model provides a highly effective method to motivate treadmill running in rats and as such can be used to characterize physiological responses to exercise without the potentially confounding influence of stress associated with an aversive shock motivator.

Autoshaping a leverpress in rats with lateral, medial, or complete septal lesions

Rats with either control operations or lateral, medial, or complete septal lesions received 600 trials of leverpress training using an autoshaping procedure, i.e., food delivery followed a 10 s illuminated lever presentation, or occurred immediately after a leverpress. Rats with complete septal lesions acquired the leverpress faster than controls and had more food-tray entries per minute during the first 100 trials than the other groups. Rats with lateral or medial septal lesions had leverpress and food-tray entries equivalent to controls. The facilitation of autoshaping a leverpress may partially be explained by the general increase in motor reactivity to stimuli found following septal lesions.

Differential reinforcement effects in rats reared in enriched and impoverished environments

Comparison of the effects of environmental enrichment (EC) and impoverishment (IC) on learning in the rat is hindered by difficulty in distinguishing, experimentally, EC/IC differences in learning capacity per se from differences in performance-related variables such as sensory or motor ability, activity and motivation. Experiments are described which analyse EC/IC performance differences in a simple lever-press training procedure. In particular the hypothesis that EC and IC rats may be differentially reinforced by the same response-contingent events is examined. The data firmly support this hypothesis and possible explanations of the differential reinforcement effect are examined.

The role of contextual cues in operant responding in rats

Following lever-press training on a variable-interval 60-second schedule of food presentation, groups of rats either remained in their home cages or were exposed to the operant chamber, from which lever and food had been removed, for five sessions. The lever was replaced in the chamber and rats from Group 1 (exposure to chamber) and Group 3 (home cage) were returned to the variable-interval schedule. Although response rates in test sessions were somewhat lower than at the end of training, there was no statistically significant difference in rates for either group. Rats in Group 2 (exposure to chamber) and Group 4 (home cage) received two test sessions of extinction. During the first session, Group 2 rates of lever pressing were significantly higher than Group 4 rates. These findings do not support the view that associations between contextual cues and the reinforcer serve to energize instrumental behavior (Pearce & Hall, 1979), and provide only minimal support for the view that contextual cues control responses that compete with the operant (Mills, 1980).

Conditioned suppression in the adult rat following chronic exposure to lead

Conditioned suppression was examined for control (N = 8) and lead-treated (N = 8) adult rats. Treated animals were exposed daily to 10 mg/kg lead administered via the diet as lead acetate. 40 days of oporant lever press training on a multiple schedule using food as reward was given. Lever press training continued for an additional 5 days but during this period subjects experienced superimposed 3 min tone presentations followed by electric shock (1.3 mA, 1.5 s duration) to a grid floor. Tone presentations were observed to be significantly more disruptive of operant steady state responding for treated subjects as compared to controls. This finding supports earlier claims that lead exposure enhances emotionality.