Random-interval Schedule Research Articles

Random interval schedule of reinforcement influences punishment resistance for cocaine in rats

In an animal model of compulsive drug use, a subset of rats continues to self-administer cocaine despite footshock consequences and is considered punishment resistant. We recently found that punishment resistance is associated with habits that persist under conditions that typically encourage a transition to goal-directed control. Given that random ratio (RR) and random interval (RI) schedules of reinforcement influence whether responding is goal-directed or habitual, we investigated the influence of these schedules on punishment resistance for cocaine or food. Male and female Sprague Dawley rats were trained to self-administer either intravenous cocaine or food pellets on a seeking-taking chained schedule of reinforcement, with the seeking lever requiring completion of either an RR20 or RI60 schedule. Rats were then given four days of punishment testing with footshock administered at the completion of seeking on a random one-third of trials. For cocaine-trained rats, the RI60 schedule led to greater punishment resistance (i.e., more trials completed) than the RR20 schedule in males and females. For food-trained rats, the RI60 schedule led to greater punishment resistance (i.e., higher reward rates) than the RR20 schedule in female rats, although male rats showed punishment resistance on both RR20 and RI60 schedules. For both cocaine and food, we found that seeking responses were suppressed to a greater degree than reward rate with the RI60 schedule, whereas response rate and reward rate were equally suppressed with the RR20 schedule. This dissociation between punishment effects on reward rate and response rate with the RI60 schedule can be explained by the nonlinear relation between these variables on RI schedules, but it does not account for the enhanced resistance to punishment. Overall, the results show greater punishment resistance with the RI60 schedule as compared to the RR20 schedule, indicating that schedules of reinforcement are an influencing factor on resistance to negative consequences.

A reinforcement schedule with two types of temporal information

Decades of research on reinforcement schedules have demonstrated that temporal information regarding the arrival or nonarrival of biologically significant events controls animal behavior. The fixed interval (FI) schedule, which is a time-based reinforcement schedule, suggests that responses are regulated by the time elapsed since the last reinforcement. This raises the question of how behavior is controlled when two distinct temporal cues regarding the availability of reinforcers are simultaneously presented. We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information: the time elapsed since the last reinforcement, and the inter-reinforcement interval (IRI). Through long-term operant conditioning in rats, we examined how temporal information controls responses. When two temporal cues were available, we found that behavior was influenced by each cue. Moreover, we discovered that the effects of these cues could be analytically separated within the dynamics of response rates. The findings revealed that in controlling behavior, living organisms can utilize two temporal cues rather than relying on a single, clear cue.•We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information.•When two temporal cues were available, we found that behavior was influenced by each cue.•Rats can utilize two temporal cues rather than relying on a single, clear cue.

Effect of instructions on the microstructure of human schedule performance.

Three experiments examined the effect of instructions on human free-operant performance on random ratio (RR) and random interval (RI) schedules. Both rates of responding, and the microstructure of behavior, were explored to determine whether bout-initiation and within-bout responding may be controlled by different processes. The results demonstrated that responding in acquisition (Experiments 1 and 2) and extinction (Experiment 3) was impacted in line with given instructions. During acquisition, rates were higher on RR compared to RI for accurate and minimal instructions. During extinction, rates decreased when there were minimal instructions. However, instructions had a greater impact on within-bout responding, than they did on bout-initiation responding. Overall rates of responding, and within-bout rates, varied in line with the nature of the instructions, but bout-initiation responding did not (Experiments 1 and 2). Resistance to extinction was increased by instructions in terms of overall responding and within-bout rates, but not in terms of bout-initiation rates (Experiment 3). These data are consistent with the hypothesis that bout-initiation responding may be less impacted by instructions than within-bout responding, speculatively, the former is stimulus-driven, automatic/habitual, and less accessible to conscious processing. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Effects of an ethanol-paired conditioned stimulus on responding for ethanol suppressed by a conditioned-taste-aversion

Ethanol-Paired Conditioned Stimuli (CS) can increase ethanol-responding either in extinction or occurring at low rates late in a session. To examine the generality of CS-induced increases in ethanol-responding, we examined whether a CS could increase responding suppressed by Conditioned-Taste-Aversion (CTA), which presumably suppresses responding by changing ethanol's valence from positive to negative. Rats were trained to respond for ethanol under a Random Interval (RI) schedule. We then removed the lever and paired Random-Time ethanol deliveries with illumination of a stimulus light (i.e., CS) for 10 sessions. Results were compared with a Truly Random Control group, in which the light and ethanol deliveries occurred independently. In a subsequent experiment, rats were treated similarly, except the light served as a discriminative stimulus, as the lever was extended and ethanol deliveries were available under a RI during light presentations. After this training, the lever was returned and rats again responded for ethanol. Subsequently, sessions were followed by LiCl administration. When responding reached low levels, LiCl administration stopped and the light was occasionally illuminated during the session. Responding during the light presentation was compared to responding during the period preceding light presentation. Responding partially recovered across 10 sessions and was greater during light presentations than in the period before it in all three groups. Increases were not reliably different between the groups, indicating that explanations for these increases such as CS-induced increases in motivation or approach toward the light are unlikely to be correct. The most likely explanation for these light-induced increases is that during sessions in which the light had been presented previously, LiCl had never been presented and thus, the light had come to signal that ethanol was safe to drink.

Focused-attention mindfulness increases sensitivity to current schedules of reinforcement.

Four experiments explored the impact of focused-attention mindfulness training on human performance on free-operant schedules of reinforcement. In each experiment, human participants responded on a multiple random ratio (RR), random interval (RI) schedule. In all experiments, responding was higher on RR than RI schedules, despite equated rates of reinforcement. A 10-min focused-attention mindfulness intervention (focused attention) produced greater differentiation between schedules than relaxation training (Experiments 1, 2, and 4), or no intervention (Experiment 3). Focused-attention mindfulness improved learning when the schedules associated with components of the multiple schedule were reversed. This occurred irrespective of whether the focused-attention mindfulness was before (Experiment 2) or after (Experiments 3 and 4) initial training, or whether compared to relaxation (Experiments 2 and 4) or no intervention (Experiment 3). In Experiment 4, following multiple RR, RI training, focused-attention mindfulness increased sensitivity to contingency reversal and did not interfere with previous training in a group that did not receive a contingency reversal. In contrast, relaxation training did not facilitate reversal learning and interfered with previous learning. The results suggest that focused-attention mindfulness improves awareness of operative contingencies by focusing participants on the present, rather than reducing interference from previous learning. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Modeling VI and VDRL feedback functions: Searching normative rules through computational simulation.

We present the mathematical description of feedback functions of variable interval and variable differential reinforcement of low rates as functions of schedule size only. These results were obtained using an R script named Beak, which was built to simulate rates of behavior interacting with simple schedules of reinforcement. Using Beak, we have simulated data that allow an assessment of different reinforcement feedback functions. This was made with unparalleled precision, as simulations provide huge samples of data and, more importantly, simulated behavior is not changed by the reinforcement it produces. Therefore, we can vary response rates systematically. We've compared different reinforcement feedback functions for random interval schedules, using the following criteria: meaning, precision, parsimony, and generality. Our results indicate that the best feedback function for the random interval schedule was published by Baum (1981). We also propose that the model used by Killeen (1975) is a viable feedback function for the random differential reinforcement of low rates schedule. We argue that Beak paves the way for greater understanding of schedules of reinforcement, addressing still open questions about quantitative features of simple schedules. Also, Beak could guide future experiments that use schedules as theoretical and methodological tools.

The effect of brief mindfulness training on the micro-structure of human free-operant responding: Mindfulness affects stimulus-driven responding

Background and objectivesThe current study examines the extent to which mindfulness impacts on operant conditioning processes, and explores the suggestion that mindfulness training serves to make humans more sensitive to the current reinforcement contingencies with which they are presented. In particular, the effect of mindfulness on the micro-structure of human schedule performance was explored. It was expected that mindfulness might impact bout-initiation responding to a greater degree than within-bout responding, premised on the assumption that bout-initiation responses are habitual and not under conscious control, but within-bout responses are goal-directed and conscious. MethodsNonclinical participants experienced one of three brief (15min) interventions: focused attention breathing exercise (mindfulness), an unfocused attention breathing exercises, or no intervention. They then responded on a multiple random ratio (RR) random interval (RI) schedule. ResultsIn the no intervention and unfocused attention groups, overall and within-bout response rates were higher on the RR than the RI schedule, but bout-initiation rates were the same on the two schedules. However, for the mindfulness groups all forms of responding were higher for the RR than the RI schedule. Previous work has noted that habitual, and/or unconscious or fringe-conscious events, are impacted by mindfulness training. LimitationsA nonclinical sample may limit generality. ConclusionsThe current pattern of results suggests that this is also true in schedule-controlled performance, and offers an insight into the manner in which mindfulness alongside conditioning-based interventions, to bring all responses under conscious control.

Cannabidiol (CBD) drives sex-dependent impairments in omission, but has no effect on reinforcer devaluation

Habits are inflexible behaviors that persist despite changes in outcome value. While habits allow for efficient responding, neuropsychiatric diseases such as drug addiction and obsessive-compulsive disorder are characterized by overreliance on habits. Recently, the commercially popular drug cannabidiol (CBD) has emerged as a potential treatment for addictive behaviors, though it is not entirely clear how it exerts this therapeutic effect. As brain endocannabinoids play a key role in habit formation, we sought to determine how CBD modifies goal-directed behaviors and habit formation. To explore this, mice were administered CBD (20 mg/kg i.p.) or vehicle as a control and trained on random interval (RI30/60) or random ratio (RR10/20) schedules designed to elicit habitual or goal-directed lever pressing, respectively. Mice were tested for habitual responding using probe trials following reinforcer-specific devaluation as well as omission trials, where mice had to withhold responding to earn rewards. We found that while CBD had little effect on operant behaviors or reward devaluation, CBD inhibited goal-directed behavior in a sex-specific and context dependent manner during the omission task. Beyond drug treatment, we found an effect of sex throughout training, reward devaluation, and omission. This work provides evidence that CBD has no effect on habit formation in a reward devaluation paradigm. However, the omission results suggest that CBD may slow learning of novel action outcome contingencies or decrease goal-directed behavior. This work calls for further examination of sex-dependent outcomes of CBD treatment and highlights the importance of investigating sex effects in habit-related experiments.

Reinforcement History Dependent Effects of Low Dose Ethanol on Reward Motivation in Male and Female Mice.

Alcohol use disorders (AUDs) are more prevalent in men than in women, though AUD diagnoses in women are growing rapidly, making an understanding of sex differences in alcohol-related behaviors increasingly important. The development of AUDs involves the transition from casual, low levels of alcohol drinking to higher, maladaptive levels. The ability of low dose alcohol to drive reward and drug seeking may differ in males and females, and this could underlie differences in susceptibility to AUD. In this study we sought to determine whether a history of chronic, low dose ethanol exposure (0.5 g/kg; i.p.) could drive sucrose reward seeking and motivation, and whether this differed between male and female mice. Adult mice were trained to lever press for a liquid sucrose reward on two reinforcement schedules: a random interval (RI) schedule and a variable ratio (VR) schedule. After training, mice were tested on each of these levers for reward motivation using a progressive ratio test. We found that a history of low dose ethanol exposure increased sucrose reward motivation in male mice, but only on the RI lever and only when exposure occurred proximal to learning. Female mice were more motivated for sucrose on the RI lever than the VR lever regardless of ethanol exposure condition. These findings indicate that training on different reinforcement schedules affects reward motivation. Further, we show that males are more susceptible to the effects of low dose ethanol on sucrose reward motivation than females. These data broaden our understanding of sex differences in reward seeking as a result of ethanol exposure.

Modeling resurgence with an evolutionary theory of behavior dynamics

McDowell’s (2004) Evolutionary Theory of Behavior Dynamics (ETBD) is a computational theory that has reproduced a wide variety of behavioral phenomena observed in material reality. Here, we extended the generality of the ETBD by successfully replicating laboratory studies of resurgence with live animals using artificial organisms (AOs) animated by the theory. We ran AOs on concurrent random-interval random-interval (conc RI RI) schedules of reinforcement wherein one alternative (i.e., a target behavior) was reinforced while the other alternative (i.e., an alternative behavior) was not reinforced. Then, we placed the target behavior on extinction and reinforced the alternative response, producing a shift in allocation of responding from the target behavior to the alternative response. Finally, schedule thinning of the alternative response (i.e., downshifts) resulted in resurgence of target behavior. Our findings indicated that resurgence increased as a function of the relative downshift in reinforcement rate and magnitude, replicating findings from previous studies with live animals. These results further illustrate the utility of the ETBD for generating dynamic behavioral data and serve as a proof-of-concept for a novel computational approach for studying and understanding resurgence in future studies.

IMPULSIVE CHOICE IN SPONTANEOUSLY HYPERTENSIVE (SHR) AND LEWIS (LEW) RATS

The Spontaneously Hypertensive rat (SHR), the most widely accepted rodent-model of Attention Deficit/Hyperactivity disorder (ADHD), usually is compared with its normotensive control, the Wistar Kyoto (WKY) rat, seeking differences in impulsivity between strains. The utilization of the WKY as a control strain for the SHR, however, has been questioned. Dopamine deficits in nucleus accumbens may cause low tolerance to delayed consequences in the SHR. Locomotion and dopamine activities in the Lewis (LEW) rat are comparable to those in the SHR, suggesting that both strains will develop similar levels of impulsivity. This possibility was examined with SHRs and LEWs responding to concurrentchains procedures. Choice was measured in the initial link where two random interval schedules operating concurrently on two levers arranged entries to two terminal links, one delivering 1-food pellet immediately and the other 4-food pellets with several delays randomly presented during the session. Both strains learned to choose impulsively with training, but the SHRs developed faster changes in preference and more impulsive choices than the LEWs. The Hyperbolic-decay Model and the Generalized Matching Law described choice well. Positive correlations between estimates of discounting rate and sensitivity to the immediacy of reinforcement confirmed some consistencies between these models of choice.

Effect of depression and anxiety on human schedule performance

Two experiments examined the impact of depression and anxiety on the microstructure of human schedule responding. Human participants responded by pressing a computer key for points on a multiple random ratio (RR) random interval (RI) schedule. The RI schedule was yoked to the RR schedule in terms of reinforcement rate within subject. Overall response rates were higher on the RR compared to the RI schedule. In both experiments, the presence of psychometrically-measured depressive traits reduced overall levels of responding. Depressive traits also decreased within-bout responding, but increased levels of bout-initiation responding. These findings regarding the microstructure of responding were not noted for anxiety traits. These results suggest that depression impacts learning by working through a number of different mechanisms – both impacting the perceived value of the reinforcer, and possibly by creating a bias to attend and process external cues associated with reinforcement which are taken to control response-initiation rates.

Methodological improvements to a Procedure for Rapidly Establishing Steady-State Behavior.

We performed three experiments to improve the quality and retention of data obtained from a Procedure for Rapidly Establishing Steady-State Behavior (PRESS-B; Klapes et al.,2020). In Experiment 1, 120 participants worked on nine concurrent random-interval random-interval (conc RI RI) schedules and were assigned to four conditions of varying changeover delay (COD) length. The 0.5-s COD condition group exhibited the fewest instances of exclusive reinforcer acquisition. Importantly, this group did not differ in generalized matching law (GML) fit quality from the other groups. In Experiment 2, 60 participants worked on nine conc RI RI schedules with a wider range of scheduled reinforcement rate ratios than was used in Experiment 1. Participants showed dramatic reductions in exclusive reinforcer acquisition. Experiment 3 entailed a replication of Experiment 2 wherein blackout periods were implemented between the schedule presentations and each schedule remained in operation until at least one reinforcer was acquired on each alternative. GML fit quality was slightly more consistent in Experiment 3 than in the previous experiments. Thus, these results suggest that future PRESS-B studies should implement a shorter COD, a wider and richer scheduled reinforcement rate ratio range, and brief blackouts between schedule presentations for optimal data quality and retention.

Rats exposed to intermittent ethanol during late adolescence exhibit enhanced habitual behavior following reward devaluation.

The brain undergoes substantial maturation during adolescence, and repeated exposure to ethanol at this time has been shown to result in long-lasting behavioral and neural consequences. During the broad period of adolescence, different neuronal populations and circuits are refined between early and late adolescence, suggesting the possibility that ethanol exposure at these differing times may lead to differential outcomes. The goal of the current study was to evaluate the impact of adolescent intermittent ethanol (AIE) during early and late adolescence on the formation of goal-directed and habitual behavior in adulthood. Male and female Sprague-Dawley rats were exposed to ethanol via intragastric gavage (4.0g/kg, 25% v/v) every other day from postnatal day (P) 25-45 or P45-65, considered early and late adolescence, respectively. In adulthood (~P70 early or~P90 late), rats were gradually food-restricted and began operant training on a fixed ratio 1 schedule. Rats were then transitioned onto random interval schedules and eventually underwent a sensory-specific satiation procedure as a model of reward devaluation. Few differences as a result of adolescent ethanol exposure were found during instrumental training. Following reward devaluation, rats exposed to water and ethanol during early adolescence exhibited reductions in lever pressing, suggestive of a goal-directed response pattern. In contrast, late AIE males and females demonstrated persistent responding following both devalued and non-devalued trials, findings representative of a habitual behavior pattern. The shifts from goal-directed to habitual behavior noted only following late AIE contribute to the growing literature identifying specific behavioral consequences as a result of ethanol exposure during distinct developmental periods within adolescence. More work is needed to determine whether the greater habit formation following late AIE is also associated with elevated habitual ethanol consumption.

Astrocytic equilibrative nucleoside transporter type 1 upregulations in the dorsomedial and dorsolateral striatum distinctly coordinate goal-directed and habitual ethanol-seeking behaviours in mice.

Two distinct dorsal striatum regions, dorsomedial striatum (DMS) and dorsolateral striatum (DLS), are attributed to conditioned goal-directed and habitual reward-seeking behaviours, respectively. Previously, our study shows that the ethanol-sensitive adenosine transporter, equilibrative nucleoside transporter 1 (ENT1), regulates ethanol-drinking behaviours. Although ENT1 is expressed in both neurons and astrocytes, astrocytic ENT1 is thought to regulate adenosine levels in response to ethanol. However, the role of DMS and DLS astrocytic ENT1 in goal-directed and habitual ethanol-seeking is not well known. Here, we identified whether the upregulation of astrocytic ENT1 in the DMS and DLS differentially regulates operant seeking behaviours towards the 10% sucrose (10S); 10% ethanol and 10% sucrose (10E10S); and 10% ethanol (10E) in mice. Using 4days of random interval (RI), mice exhibited habitual seeking for 10S, but goal-directed seeking towards 10E10S. Using the same mice conditioned with 10E10S, we examined 10E-seeking behaviour on a fixed ratio (FR) for 6days and RI for 8days. On the other hand, during FR and the first 4days of RI schedules, mice showed goal-directed seeking for 10E, whereas mice exhibited habitual seeking for 10E during the last 4days of RI schedule. Interestingly, DMS astrocytic ENT1 upregulation promotes shift from habitual to goal-directed reward-seeking behaviours. By contrast, DLS astrocytic ENT1 upregulation showed no effects on behavioural shift. Taken together, our findings demonstrate that DMS astrocytic ENT1 contributes to reward-seeking behaviours.

Role of psychopathic personality traits on the micro-structure of free-operant responding: impacts on goal-directed but not stimulus-drive responses in extinction

The current study explored effects of psychopathic personality traits on micro-structure of free-operant responding. Non-clinical participants were recruited, and responded on a multiple random-ratio (RR) random-interval (RI) schedule for points, prior to being placed into extinction. They completed the TriPM to measure psychopathic personality, and were divided into those scoring lower and higher on the sub-scales of the TriPM (meanness, boldness, disinhibition). Responding was higher on RR than RI schedules, with no difference between rates of bout-initiation, but higher rates of within-bout responding the RR schedule. Extinction reduced responding after an initial response burst. No personality traits impacted acquisition of overall free-operant responding, nor its microstructure. However, meanness retarded the course of extinction after promoting an initial response burst. The results suggest psychotics are more goal-directed, consistent with meanness being associated with aggression and low tolerance to frustration, potentiating frustrative non-reward effects.

Factors controlling the micro-structure of human free-operant behaviour: Bout-initiation and within-bout responses are effected by different aspects of the schedule

Two experiments examined factors controlling human free-operant performance in relation to predictions based on the nature of bout-initiation and within-bout responding. Overall, responding was higher for a random ratio (RR) than a random interval (RI) schedule, with equal rates of reinforcement. Bout-initiation rates were not different across the two schedules, but within-bout rates were higher on the RR schedule. Response cost reduced overall rates of responding, but tended to suppress bout-initiation responding more than within-bout responding (Experiments 1 & 2). In contrast, reinforcement magnitude increased all forms of responding (Experiment 2). One explanation consistent with these effects is that bout-initiation responses are controlled by overall rates of reinforcement through their impact on the context (i.e. are stimulus-driven), but that within-bout responses are controlled by response reinforcement (i.e. are goal-directed). These current findings are discussed in the light of these theoretical suggestions.

Instrumental learning in a mouse model for obsessive-compulsive disorder: Impaired habit formation in Sapap3 mutants

It has been hypothesized that maladaptive habit formation contributes to compulsivity in psychiatric disorders such as obsessive-compulsive disorder (OCD). Here, we used an established animal model of OCD, Sapap3 knockout mice (SAPAP3-/-), to investigate the balance of goal-directed and habitual behavior in compulsive individuals and if altered habit formation is associated with compulsive-like behavior. We subjected 24 SAPAP3-/- and 24 wildtype littermates (WT) to two different schedules of reinforcement in a within-subjects design: a random-ratio (RR) schedule to promote goal-directedness, and a random-interval (RI) schedule, known to facilitate habitual responding. SAPAP3-/- acquired responding under both schedules, but showed lower response rates and fewer attempts to collect food pellets than WT, indicative of altered reward processing. As expected, WT were sensitive to sensory-specific satiety (outcome devaluation) following RR training, but not RI training, demonstrating schedule-specific acquisition of goal-directed and habitual responding, respectively. In contrast, SAPAP3-/- were sensitive to outcome devaluation after both RR and RI training, suggesting decreased engagement of a habitual response strategy. No linear relation was observed between increased grooming and behavior during the outcome devaluation test in SAPAP3-/-. Together, our findings demonstrate altered reward processing and impaired habit learning in SAPAP3-/-. We report a diminished propensity to form habits in these mice, which albeit inconsistent with the predominant idea of excessive habit formation in OCD, nonetheless points at dysregulation of behavioral automation in the context of compulsivity. Thus, the habit hypothesis of compulsivity should be updated to state that an imbalance of habitual and goal-directed responding in either direction can contribute to the development of compulsive behavior.

Unexpected food outcomes can return a habit to goal-directed action.

Three experiments examined the return of a habitual instrumental response to the status of goal-directed action. In all experiments, rats received extensive training in which lever pressing was reinforced with food pellets on a random-interval schedule of reinforcement. In Experiment 1, the extensively-trained response was not affected by conditioning a taste aversion to the reinforcer, and was therefore considered a habit. However, if the response had earned a new and unexpected food pellet during the final training session, the response was affected by taste aversion conditioning to the (first) reinforcer, and had thus been converted to a goal-directed action. In Experiment 3, 30 min of prefeeding with an irrelevant food pellet immediately before the test also converted a habit back to action, as judged by the taste-aversion devaluation method. That result was consistent with difficulty in finding evidence of habit with the sensory-specific satiety method after extensive instrumental training (Experiment 2). The results suggest that an instrumental behavior's status as a habit is not permanent, and that a habit can be returned to action status by associating it with a surprising reinforcer (Experiment 1) or by giving the animal an unexpected prefeeding immediately prior to the action/habit test (Experiment 3).

Goal-directed control on interval schedules does not depend on the action-outcome correlation.

When an organism's action is based on an anticipation of its consequences, that action is said to be goal-directed. It has long been thought that goal-directed control is made possible by experiencing a strong correlation between response rates and reward rates (Dickinson, 1985). To test this idea, we designed a set of experiments to determine whether the response rate-reward rate correlation is a reliable predictor of goal-directed control on interval schedules. In Experiment 1, rats were trained on random interval (RI) schedules in which the response rate-reward rate correlation was manipulated across groups. In tests of reward devaluation, rats behaved in a goal-directed manner regardless of the experienced correlation. In Experiment 2, rats once again experienced either a strong or weak correlation, but on RI schedules with lower overall reward densities. This time, behavior appeared habitual regardless of the experienced correlation. Experiment 3 confirmed that the density of the RI schedule influences goal-directed control, and also revealed that extensive training on these schedules resulted in goal-directed action. Finally, in Experiment 4 goal-directed responding was greater and emerged sooner on fixed than random interval schedules, but, again, was manifest after extensive training on the RI schedule. Taken together, our data suggest that goal-directed and habitual control are not determined by the correlation between response rates and reward rates. We discuss the importance of temporal uncertainty, action-outcome contiguity, and reinforcement probability in goal-directed control. (PsycINFO Database Record (c) 2019 APA, all rights reserved).