Variable-interval Reinforcement Research Articles

The pleasure of art as a matter of fact.

> All other forms of perception divide a man, because they are exclusively based either on the sensuous or on the intellectual part of his being; only the perception of the Beautiful makes something whole of him, because both his natures must accord with it. > — Schiller, 1793–1795, Letters on the Aesthetic Education of Man , p. 138, Letter 27 Can art make us better people? Better members of society? In her recent article, ‘Pleasure junkies all around! Why it matters and why “the arts” might be the answer: a biopsychological perspective’, Christensen [1] claims that engaging with art can promote healthy choices, choices that balance short-term pleasure goals with long-term general well-being. She suggests that many modern life conveniences, such as social media, computer games or online shopping, have the potential to turn us into ‘pleasure junkies’ because they maximize short-term pleasures. But art, Christensen argues, is a safe choice and a means to remedy this unhealthy addiction to pleasure. Christensen's argument is grounded on three claims about the sort of pleasure we get from art. First, in contrast to low-level pleasure , which Christensen conceives as ‘a mere perceptual stimulation leading to a rewarding sensation (food, sex, etc.)’ ([1], p. 2), pleasure from art is presented as a kind of higher-order pleasure that engages ‘broader neural networks implied in the attribution of meaning’ (p. 2), presumably leading to ‘long-term maintenance of healthy bodily function’ (p. 2). Second, ‘the arts do not induce states of craving without fulfilment—as do activities with reinforcement schedules which are prone to create habits and addictions such as intermittent variable ratio or interval reinforcement schedules (e.g. social media, gambling, football, extreme sports, drugs’ (p. 4). Third, ‘the arts do not search for a perceptual “Bliss point” […]. They do not just …

Using Variable Interval Reinforcement Schedules to Support Students in the Classroom: An Introduction With Illustrative Examples

When behaviors are reinforced with a variable interval reinforcement schedule, reinforcement is available only after an unknown period of time. These types of reinforcement schedules are most useful for reinforcing slow and steady responding and for differentially reinforcing behaviors that are incompatible with some problematic behaviors. This review helps define variable interval reinforcement schedules, uses the example of a strategy to manage thumb-sucking behavior to illustrate the implementation of these schedules, and describes potential applications in school and clinical settings.

E-mail Anonymous: A Physician's Addiction

DOI: 10.1200/JCO.2014.58.9705 The next time you are sitting in Grand Rounds, standing in the cafeteria line, rounding with your team, or giving a talk, take a look at your colleagues. Chances are, they are staring at a screen, and many are probably checking e-mail. E-mail has changed how physicians communicate. Whether related to patient care, academic pursuits, or administrative tasks, our interactions are now frequently conducted electronically. This form of communication has a new pressure attached: who would not prefer to collaborate with someone who responds within minutes? The patient-based aspect of e-mail reinforces the importance of a prompt reply—patient care is paramount, and therefore, checking e-mail frequently feels like something that good doctors should do. Whether the shift from live dialogue to e-mail has led to improved communication, enhanced productivity, or—most importantly—improved patient outcomes is unknown. The true extent and impact of daily e-mail use by physicians has also not been formally quantified. However, my own e-mail usage can be formally qualified. I am an e-mailaholic. It is hard for me to know exactly when the addiction began, but by the time I became an attending physician, I would check my e-mail at nearly every moment. The e-mail browser would be open in my office or clinic throughout the day, and when not at a computer, my e-mail would be just a few seconds away, accessible via the phone in my righthand pocket. Before going to bed and first thing on waking, while waiting for my coffee to brew or walking through the parking lot, I was checking e-mail. It was an action devoid of thought; it just became second nature for my thumb to push, slide right, enter password, and push once more—I can open e-mail with my eyes closed. Most of us can. A retrospectively self-administered CAGE questionnaire (the name of which is derived from questions used to screen for alcohol addiction; these questions involve the words cut down, annoyed, guilty, and eye-opener) would have easily produced three “yes” responses: I often thought about the need to cut down my e-mail usage and spend more time on the task at hand or with the people around me; my children were at times annoyed with me for not paying attention to our conversations; and I felt uneasy without a daily eye-opener: a check of my e-mail soon after waking up. Constant communication may have its benefits. Queries are answered more quickly, both by colleagues and patients (who may feel less intimidated getting in touch with their physician via e-mail), and perhaps projects and tasks are advanced a little faster. But e-mail also decreases spoken and face-to-face interactions, arguably eroding relationships. Questions answered by e-mail may result in misunderstandings that require many follow-up e-mails. This ongoing e-mailing leads to interruptions and multitasking: patient care, writing, reading, thinking, and generally being present in any given moment become fragmented. Many knowledge workers frequently multitask, but some believe that the term multitasking is a misnomer and that one is actually rapidly switching from task to task, which is a hindrance to creativity and intense focus. What is the cost of this constant switching of thought and attention? Although no prospective and unequivocal studies exist, e-mail has been associated with stress and loss of control, and in an academic environment, users spent approximately 23% of their time on e-mail, checking it up to 36 times per hour. In older adults, multitasking led to significant working memory disruption. When e-mail was shut off for 13 knowledge workers, they tended to multitask less and exhibited longer task focus; stress was also lower without e-mail. Interestingly, some companies are now asking their employees to shut off e-mail for certain lengths of time during the day to increase productivity. The gravitational pull of e-mail is strong and draws on the variable-interval reinforcement schedule concept—reward is unpredictable and e-mail becomes addictive. Furthermore, a recent study in default-mode processing (inward-directed thought) revealed that people not only do not seem to enjoy sitting quietly and thinking with no distractions, but that 43% (67% of men and 25% of women) selfadministered electric shocks rather than simply sit for 15 minutes without anything to do (all of them previously said they would pay money to avoid such JOURNAL OF CLINICAL ONCOLOGY A R T O F O N C O L O G Y VOLUME 33 NUMBER 3 JANUARY 2

Saccadic reaction time distributions follow the matching law in a concurrent variable interval reinforcement schedule

Saccadic reaction time distributions follow the matching law in a concurrent variable interval reinforcement schedule

Contextual control of operant behavior: evidence for hierarchical associations in instrumental learning.

Recent research has suggested that operant responses can be weakened when they are tested in new contexts. The present experiment was therefore designed to test whether animals can learn a context-(R-O) relation. Rats were given training sessions in context A, in which one response (R1; lever pressing or chain pulling) produced one outcome (O1) and another response (R2; chain pulling or lever pressing) produced another outcome (O2) on variable interval reinforcement schedules. These sessions were intermixed with training in context B, where R1 now produced O2 and R2 produced O1. Given the arrangement, it was possible for the animal to learn two distinct R-O associations in each specific context. To test for them, rats were then given aversion conditioning with O2 by pairing its presentation with lithium-chloride-induced illness. Following the aversion conditioning, the rats were given an extinction test with both R1 and R2 available in each context. During testing, rats showed a selective suppression in each context of the response that had been paired with the reinforcer subsequently associated with illness. Rats could not have performed this way without knowledge of the R-O associations in effect in each specific context, lending support to the hypothesis that rats learn context-(R-O) associations. However, despite a complete aversion to O2, responding was not completely suppressed, leaving the possibility open that rats form context-R associations in addition to context-(R-O) associations.

Pausing as an operant: Choice and discriminated responding

The effects of intermittent schedules of reinforcement for pausing were evaluated in two experiments. In Experiment 1, across a series of conditions, a variable-interval (VI) baseline schedule, in which pigeons' key pecks produced food, alternated with conditions in which food was delivered according to a concurrent VI (for key pecking) tandem variable-time differential-reinforcement-of-other-behavior (DRO) 5-s schedule. Time spent pausing within a session was proportional to the reinforcement rates associated with the tandem schedule. To examine the control of pausing by antecedent events, Experiment 2 arranged a multiple schedule in which pecking and pausing in either component were maintained according to concurrent schedules like those used in the first experiment. The availability of reinforcement for pausing was signaled in one component while signals uncorrelated with reinforcement were presented in the other. Signaled reinforcement for pausing, relative to the presentation of uncorrelated signals, decreased time spent pausing, a finding consistent with existing research on the effects of signaled VI reinforcement for key pecking in pigeons. The results of the two experiments show that pausing functions as an operant in much the same way that discrete responses, like key pecks, do, and that pausing and other operants are similarly affected by both antecedent and consequent events.

Inhibiting PKMζ reveals dorsal lateral and dorsal medial striatum store the different memories needed to support adaptive behavior

Evidence suggests that two regions of the striatum contribute differential support to instrumental response selection. The dorsomedial striatum (DMS) is thought to support expectancy-mediated actions, and the dorsolateral striatum (DLS) is thought to support habits. Currently it is unclear whether these regions store task-relevant information or just coordinate the learning and retention of these solutions by other brain regions. To address this issue, we developed a two-lever concurrent variable-interval reinforcement operant conditioning task and used it to assess the trained rat's sensitivity to contingency shifts. Consistent with the view that these two regions make different contributions to actions and habits, injecting the NMDA antagonist DL-AP5 into the DMS just prior to the shift impaired the rat's performance but enhanced performance when injected into the DLS. To determine if these regions support memory content, we first trained rats on a biased concurrent schedule (Lever 1: VI 40" and Lever 2: VI 10"). With the intent of "erasing" the memory content stored in striatum, after this training we inhibited the putative memory-maintenance protein kinase C isozyme protein kinase Mζ (PKMζ). Infusing zeta inhibitory peptide (ZIP) into the DLS enhanced the rat's ability to adapt to the contingency shift 2 d later, whereas injecting it into the DMS had the opposite effect. Infusing GluR2(3Y) into the DMS 1 h before ZIP infusions prevented ZIP from impairing the rat's sensitivity to the contingency shift. These results support the hypothesis that the DMS stores information needed to support actions and the DLS stores information needed to support habits.

CONTEXTUAL INFLUENCES ON RESISTANCE TO DISRUPTION IN CHILDREN WITH INTELLECTUAL DISABILITIES

Training context can influence resistance to disruption under differing reinforcement schedules. With nonhumans, when relatively lean and rich reinforcement schedules are experienced in the context of a multiple schedule, greater resistance is found in the rich than the lean component, as described by behavioral momentum theory. By contrast, when the schedules are experienced in separated blocks of sessions (i.e., as single schedules), resistance is not consistently greater in either component. In the current study, two groups of 6 children with intellectual disabilities responded to stimuli presented in relatively lean or rich components. For both, reinforcers were delivered according to the same variable-interval reinforcement schedule; additionally, the rich component included the delivery of response-independent reinforcers. The Within group was trained on a multiple schedule in which lean and rich components alternated regularly within sessions; the Blocked group was trained on two single schedules in which sessions with either the lean or rich schedule were conducted in successive blocks. Disruption tests presented a concurrently available alternative stimulus disrupter signaling the availability of tangible reinforcers. All 6 Within participants showed greater resistance to disruption in the rich component, consistent with behavioral momentum theory. By contrast, there was no consistent or significant difference in resistance for Blocked participants. This finding is potentially relevant to the development of interventions in applied settings, where such interventions often approximate single schedules and include response-independent reinforcers.

CHOICE IN QUAIL NEONATES: THE ORIGINS OF GENERALIZED MATCHING

Although newborns have surprised scientists with their learning skills, proficiency on concurrent schedules of reinforcement requires (in effect) the ability to integrate and compare behavior-consequence relations over time. Can very young animals obey the quantitative relation that applies to such repeated choices, the generalized matching law? The provenance of the skill is not well understood, and this study provides the first investigation of matching in neonates. Northern bobwhite (Colinus virginianus) hatchlings pecked left and right targets on a touchscreen for heat delivery on a concurrent variable-interval reinforcement schedule. Within 5 days after hatching, the chicks showed sensitivity levels significantly greater than zero, but short of typical adult levels. However, stable sequential patterns emerged almost immediately, including a consistent choose-rich tendency after unreinforced responses, one that entails some degree of temporal integration. These exploratory data suggest that the basic ability to match develops quickly in this precocial species, but that more extensive experience may be required to achieve the higher sensitivities typically seen in adults.

Human choice behaviour in a frequently changing environment

The present experiment provided a replication in humans of an experimental procedure that has been used frequently with nonhumans to investigate choice behaviour in a changing environment. Six volunteers played a computer game, which required tracking of a moving balloon on two simultaneously available response panels for monetary reinforcers. Each of the 15 sessions randomly arranged the following concurrent variable-interval reinforcement schedules, which were in effect until six reinforcers had been obtained: 27:1, 9:1, 3:1, 1:1, 1:3, 1:9, and 1:27. Although many aspects of human performance appeared to be qualitatively similar to that of nonhumans on this procedure, such as the rapid preference shifts towards the within-session reinforcer ratios and the presence of local effects of reinforcers, values of sensitivity to reinforcement were very variable in the present study, as commonly reported in human choice studies. Future variations and refinements of the experimental methods are needed to explore how this variability may be reduced.

Slave to the striatal habit (Commentary on Tricomi et al.)

Colloquially, we all know what it means to perform an action by ‘habit’. But scientifically, characterizing behaviors as such is not so straightforward. Indeed, while addictive behaviors are often framed in the context of habit learning (Graybiel, 2008), this remains a controversial issue in the neurobiology of addiction literature (Berridge, 2007; Robbins & Everitt, 2007). For example, does one light up a cigarette as an instinctive response to cues that are generally associated with that motor program (due to prior stimulus-response reinforcement learning)? Or rather, do such behaviors reflect a goal-directed process in which the craving of a particular desired outcome (i.e. the nicotine high) is used to flexibly determine any complex sequence of actions needed to achieve it? Both of these processes likely contribute to habit-like behavior, but teasing them apart (at both cognitive and neurobiological levels) is critical. In animal research, habits are typically operationalized as behaviors that are so ingrained as to be insensitive to their consequent outcomes, as evaluated with a devaluation procedure. Animals are first trained to press a lever to produce a reward. Next, the reward itself is devalued so that it is no longer desirable to them (e.g. by providing free access to the reward so that they are satiated). The animal is then presented with the lever to determine whether they respond in a ‘goal-directed’ manner by reducing lever presses, indicative that their actions are sensitive to the (no longer desired) outcome. However, given sufficient prior training, animals will continue to press the lever ‘by habit’ despite the fact that they do not want the reward. In rats, it is now well established that the dorsolateral striatum is responsible for ingraining stimulus-response associations that lead to habit-like behavior, whereas prefrontal cortical regions are required for updating action-value representations and allowing the animal to exert goal-directed behavior (Yin et al., 2004). By contrast, despite years of research on procedural and reinforcement learning, evidence for veridical habit learning in humans is lacking. In this issue of EJN, Tricomi et al. (2009) investigated this question with functional magnetic resonance imaging (fMRI) and behavioral testing over multiple sessions. They used a variable interval reinforcement schedule known to promote habitual behavior in rodents. Participants were presented with fractal stimulus cues and learned to press one of two different keys as often as they wished. Each cue, if responded to with the correct key and at the appropriate time, would yield a different food reward (corn chips or chocolate bars). After the training session, the devaluation procedure was administered: participants were given free access to one of the two rewards so that they were satiated to that food. In the critical test phase, when presented with the fractal cues, those who had undergone only a single day of training behaved in a goal-directed manner: they responded selectively less to the stimulus associated with the devalued reward. In stark contrast, those who had undergone three days of training continued to respond ‘by habit’ to both cues as if the devaluation procedure had never occurred. What are the brain mechanisms of such habitual responding? Notably, in the imaging analysis, an area within the dorsolateral striatum – specifically the right posterior putamen and the globus pallidus to which it projects – showed greater activation at the end of the three day training procedure than at the beginning. These findings are remarkably consistent with those implicating homologous areas in rodent habitual behavior (Yin et al., 2004). Furthermore, activations in the ventromedial prefrontal cortex (vmPFC), which were previously shown to be sensitive to changes in subjective value following devaluation (Valentin et al., 2007) appeared to ‘ramp up’ on each trial in anticipation of reward outcomes – consistent with an action-outcome goal-directed process. How do brain areas involved in habitual versus goal-directed behavior interact? Whereas basal ganglia activations increased across training, suggestive of a progressive reinforcement learning process, the putative action-outcome representations in vmPFC continued to ramp up in anticipation of reward throughout all training sessions. Thus habitual responding appears not to result from a decrease in the anticipation of reward outcomes across trials, but rather due to strengthening of stimulus-response links. This interpretation is consistent with that embedded within computational models in which habitual responding emerges even as both striatal and prefrontal systems represent their respective estimates of reward likelihood for the action at hand, but with striatum progressively exerting a dominant bias on behavioral control (Daw et al., 2005; Frank & Claus, 2006). In summary, these data provide compelling evidence for common neurobiological substrates of habit learning in humans and rodents. Future studies should investigate the degree to which these processes are altered by clinical disorders including obsessive compulsive disorder and addiction.

Oscillations following periodic reinforcement

Three experiments examined behavior in extinction following periodic reinforcement. During the first phase of Experiment 1, four groups of pigeons were exposed to fixed interval (FI 16s or FI 48s) or variable interval (VI 16s or VI 48s) reinforcement schedules. Next, during the second phase, each session started with reinforcement trials and ended with an extinction segment. Experiment 2 was similar except that the extinction segment was considerably longer. Experiment 3 replaced the FI schedules with a peak procedure, with FI trials interspersed with non-food peak interval (PI) trials that were four times longer. One group of pigeons was exposed to FI 20s PI 80s trials, and another to FI 40s PI 160s trials. Results showed that, during the extinction segment, most pigeons trained with FI schedules, but not with VI schedules, displayed pause-peck oscillations with a period close to, but slightly greater than the FI parameter. These oscillations did not start immediately after the onset of extinction. Comparing the oscillations from Experiments 1 and 2 suggested that the alternation of reconditioning and re-extinction increases the reliability and earlier onset of the oscillations. In Experiment 3 the pigeons exhibited well-defined pause-peck cycles since the onset of extinction. These cycles had periods close to twice the value of the FI and lasted for long intervals of time. We discuss some hypotheses concerning the processes underlying behavioral oscillations following periodic reinforcement.

Dynamic behavioural changes in the Spontaneously Hyperactive Rat: 1. Control by place, timing, and reinforcement rate

Several existing theoretical models of Attention Deficit Hyperactivity Disorder (ADHD) predict that a high or low learning rate contributes to some cases of ADHD; and that this is controlled by the dopamine signal. The Spontaneously Hypertensive (Hyperactive) Rat has an abnormal Dopamine Transporter (DAT), which is likely to be hypofunctional, thereby increasing the dopamine concentration and the learning rate. We therefore test the prediction that the SHR learns too fast. Using a variable interval reinforcement schedule, we looked for increased responding within each interval, at around the durations of the last three intervals. At the time predicted by the previous interval, SHR responded significantly (10%) more than their baseline rate, and control WKY rats similarly less than their baseline. At the time predicted by the second (and third) previous intervals, there were minor trends (and no change) in the same direction. In summary, at the times predicted, SHR respond more than control rats. The effect size is insufficient to account for all the differences from control rats, such as their nearly threefold greater operant responding. At the times when SHR increase responding, WKY suppress it. This demonstrates similar learning to the SHR but the opposite response, probably due to anxiety about the bang accompanying reinforcer delivery: WKY are known to have a very negative response to stress. Interval schedules with randomised interval lengths have considerable potential for assessing multiple causes of behaviour, particularly those acting over short timecourses. Alterations in learning rate are not a sufficient explanation for either SHR or ADHD difference from controls.

Dynamic behavioural changes in the Spontaneously Hyperactive Rat: 3. Control by reinforcer rate changes and predictability

Variable intervals are widely believed to produce steady rates of responding. However, based on the calming effect of unpredictability in attention deficit hyperactivity disorder (ADHD) we hypothesised that an animal model of this disorder, the Spontaneously Hyperactive (or Hypertensive) Rat, would become less active following particularly variable sequences of interval-lengths in a variable interval schedule. From a large dataset of holepokes and tray-reports by rats in a variable interval reinforcement schedule, we extracted numerous short sequences of intervals on the basis of the first, second, and third derivatives of reinforcement timing (i.e. rate, acceleration, and jerk) in recent intervals. Sets of selected intervals were compared with one another to elucidate the effect of these different derivatives on behaviour in the current interval. Results show that SHR are more active after richer recent reinforcement; after decelerating reinforcers; and after predictable reinforcers. The hypothesis is supported. In conclusion, SHR behaviour largely complies with the Extended Temporal Difference model which in turn has been previously validated against published data in ADHD. The Extended TD model therefore is able to account for two species’ behaviour in a wide range of experimental paradigms. SHR are similar in several respects to group averages of children with ADHD, except that SHR have reduced variability and perform actions faster than controls. Hyperactivity in the SHR is very dependent on momentary environmentally determined states, which is an important area for future investigation of ADHD.

Dynamic behavioural changes in the Spontaneously Hyperactive Rat: 2 Control by novelty

An important aspect of attention deficit hyperactivity disorder (ADHD) is its temporary amelioration by novelty and by stimulant medication. Extant learning-based accounts of ADHD are unable to account for the temporary amelioration effect. One possible mechanism is a drive for novelty. Computational simulations have previously shown that such a drive can result from a “dopamine appetite”. Empirical demonstration of such a process requires, as a first step, the development of standard criteria for identifying drive-based, versus learning-based, changes in behaviour. Using a variable-interval reinforcement schedule, we looked over a range of timescales for behavioural changes showing putative characteristics of drives, namely: low information content, unidirectionality, saturability, spontaneous reversibility in less than a day, and cycle stability. SHR lacks normal down-regulation of responding when the schedule becomes sparser. SHR appears to be re-learning the schedule length during the days of each calendar week. SHR hyperactivity is specific to the operant and develops gradually over the first five minutes of each session. Empirical within-session results were replicated by a simple simulation containing two interacting reward systems, one for water and the other for stimulation (including novelty). To summarise, enhanced sensation-seeking (or a subtype of it, novelty-seeking) provides the best available account of changes in SHR activity within sessions, though not of changes over longer timecourses. Sensation-seeking appears to be associated with low anxiety in the SHR. SHR propensity to display multiple influences on their behaviour makes them ideal for further pharmacological, genetic, and behavioural investigation.

Concurrent performances: Extinction, noncontingent reinforcement andvariably delayed reinforcement

Transitions from variable-interval (VI) reinforcement of a pigeon’s pecks on one key to extinction, noncontingent reinforcement or variably delayed reinforcement of those pecks were arranged with and without concurrent reinforcement of pecks on a second key. Within sessions, the two keys were sometimes available singly and sometimes concurrently. Response rates during extinction and during noncontingent reinforcement decreased comparably. Similar rate decrements occurred early after initial exposure to variably delayed reinforcement; thereafter, delayed reinforcers maintained relatively higher rates of responding that produced little if any loss of scheduled reinforcers. Extinction, noncontingent reinforcement and delayed reinforcement typically maintained lower rates of responding when operating concurrently with VI reinforcement than when operating alone. Responding was maintained by the contingent relation between responses and reinforcers whether that contingency involved immediate or variably delayed reinforcers. Responding was not maintained when that contingency was broken whether reinforcers were completely discontinued or were delivered independently of responses (noncontingently). These results add to the data supporting a distinction between the effects of initiating or terminating response-reinforcer contingencies and those of initiating or terminating reinforcer deliveries. When responding appears to be maintained by noncontingent reinforcement, in so-called superstition or adventitious reinforcement, it is likely either that the maintenance will be transient or that other variables are operating, as when reinforcer deliveries have eliciting or adjunctive effects.

ＶＩ強化によって維持しているラットのレバー押し反応に対するＶＲ罰の効果

The effects of reinforcement and punishment on response suppression under variable-interval reinforcement and variable-ratio punishment schedules were investigated. At baseline, lever pressing in rats was maintained by a variable-interval food reinforcement schedule. In the punishment condition, responding was punished by a grid shock under a variable-ratio schedule in which three rats experienced four or five punishment rates and a fixed reinforcement rate. Results indicated that there was a negative linear correlation between the variable-ratio punishment rate and the response rate. Results of three rats that experienced five or six reinforcement rates with a fixed punishment rate indicated that punishment suppressed responding at all reinforcement rates. These findings suggest that the punishment rate was strongly related to response suppression under both variable-interval reinforcement and variable-ratio punishment schedules.

Extracellular level of basolateral amygdalar dopamine responding to reversal of appetitive-conditioned discrimination in young and old rats

Young and old rats, aged 3 and 24 months old, respectively, were conditioned to press a lever under lamp-on conditions for reward acquisition and lamp-off for no reward using a variable interval reinforcement schedule that averaged 15 s; i.e., the minimal requirement was 4 responses/min. Over a 30-day period, young and old groups showed increased response to lamp-on from 22 to 35/min and from 10 to 23/min, respectively, and shortened response to lamp-off after initial training. Response to lamp-on as a percentage of total response to lamp-on and -off (the discrimination ratio) was over 80%. For the next 30 days, reversal learning was imposed to reinforce discrimination of the lamp-off state. Young rats showed a steadily increasing discrimination ratio from 40% to 70%, and old rats from 30% to 60%. In comparison with the initial training, young rats showed a total response increase from 50% to 60%, while old rats showed an approximately 5% decrease without loss of reward-obtaining efficiency. In vivo microdialysis during reversal revealed that young rats had higher dopamine transmission in the basolateral amygdala than old rats. The dopamine level was positively correlated with the number of responses to state of reward in young rats and negatively with the numbers of both NRF and RF responses to lamp-on and -off states in old rats. These results suggest that in reversal discrimination, basolateral amygdalar dopamine efflux correlates with the manner of age-related conditioned response rather than the ability to learn.

Variable-interval reinforcement schedule value influences responding following REM sleep deprivation.

The effects of rapid-eye movement sleep deprivation (REMSD) in rats were studied in relation to variable-interval (VI) reinforcement schedule value. Initially, lever pressing was maintained on a VI 30-s schedule of food pellet delivery. After a baseline was established, rats were repeatedly exposed to 96 hr of REMSD and control conditions of an equivalent duration. Responding decreased following REMSD but not after exposure to control conditions. Lever pressing was then maintained on a VI 15-s schedule of food pellet delivery and exposure to the REMSD and control conditions was repeated. Under this condition following repeated REMSD exposures, rates of lever pressing became similar to baseline responding. A VI 30-s schedule of food pellet delivery was then reinstated and REMSD and control conditions were repeated. Lever pressing following exposure to the REMSD condition decreased for 3 of 4 rats. Results suggest that VI schedule value influences the effects of REMSD on responding.

Bouts of responding from variable-interval reinforcement of lever pressing by rats.

Four rats obtained food pellets by lever pressing. A variable-interval reinforcement schedule assigned reinforcers on average every 2 min during one block of 20 sessions and on average every 8 min during another block. Also, at each variable-interval duration, a block of sessions was conducted with a schedule that imposed a variable-ratio 4 response requirement after each variable interval (i.e., a tandem variable-time variable-ratio 4 schedule). The total rate of lever pressing increased as a function of the rate of reinforcement and as a result of imposing the variable-ratio requirement. Analysis of log survivor plots of interresponse times indicated that lever pressing occurred in bouts that were separated by pauses. Increasing the rate of reinforcement increased total response rate by increasing the rate of initiating bouts and, less reliably, by lengthening bouts. Imposing the variable-ratio component increased response rate mainly by lengthening bouts. This pattern of results is similar to that reported previously with key poking as the response. Also, response rates within bouts were relatively insensitive to either variable.