Fixed-interval Research Articles

A reinforcement schedule with two types of temporal information

Decades of research on reinforcement schedules have demonstrated that temporal information regarding the arrival or nonarrival of biologically significant events controls animal behavior. The fixed interval (FI) schedule, which is a time-based reinforcement schedule, suggests that responses are regulated by the time elapsed since the last reinforcement. This raises the question of how behavior is controlled when two distinct temporal cues regarding the availability of reinforcers are simultaneously presented. We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information: the time elapsed since the last reinforcement, and the inter-reinforcement interval (IRI). Through long-term operant conditioning in rats, we examined how temporal information controls responses. When two temporal cues were available, we found that behavior was influenced by each cue. Moreover, we discovered that the effects of these cues could be analytically separated within the dynamics of response rates. The findings revealed that in controlling behavior, living organisms can utilize two temporal cues rather than relying on a single, clear cue.•We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information.•When two temporal cues were available, we found that behavior was influenced by each cue.•Rats can utilize two temporal cues rather than relying on a single, clear cue.

Active and passive waiting in impulsive choice: Effects of fixed-interval and fixed-time delays.

Behavioral interventions to improve self-control, preference for a larger-later (LL) reward over a smaller-sooner (SS) reward, involve experience with delayed rewards. Whether they involve timing processes remains controversial. In rats, there have been inconsistent results on whether timing processes may be involved in intervention-induced improvements in self-control. Interventions that improved self-control with corresponding timing improvements used fixed-interval (FI) delays, whereas interventions that failed to find corresponding timing improvements used fixed-time (FT) delays. The FI schedule includes a response contingency (active waiting), whereas the FT schedule delivers reward automatically (passive waiting). The present study compared the effects of FI and FT schedules in interventions and impulsive choice tasks to evaluate effects on self-control and timing behavior. The impulsive choice task evaluated preference for an SS option (one pellet after 10-, 15-, 20-, 25-, and 30-s delays) versus an LL option (two pellets after a 30-s delay). The intervention task included forced-choice SS (one pellet after 10 s) and LL (two pellets after 30 s) sessions under FI or FT schedules. FI schedules produced greater sensitivity to SS delay in the impulsive choice task. Both FI and FT interventions increased LL choices. Following choice testing, temporal bisection and peak interval tasks revealed better timing precision for rats with an FI delay experience. Overall, the FI choice contingency was associated with improved temporal attention and timing precision.

The contributions of neonatal inhalation of copper to air pollution-induced neurodevelopmental outcomes in mice

Exposures to ambient ultrafine particle (UFP) air pollution (AP) during the early postnatal period in mice (equivalent to human third trimester brain development) produce male-biased changes in brain structure, including ventriculomegaly, reduced brain myelination, alterations in neurotransmitters and glial activation, as well as impulsive-like behavioral characteristics, all of which are also features characteristic of male-biased neurodevelopmental disorders (NDDs). The purpose of this study was to ascertain the extent to which inhaled Cu, a common contaminant of AP that is also dysregulated across multiple NDDs, might contribute to these phenotypes. For this purpose, C57BL/6J mice were exposed from postnatal days 4–7 and 10–13 for 4 hr/day to inhaled copper oxide (CuxOy) nanoparticles at an environmentally relevant concentration averaging 171.9 ng/m3. Changes in brain metal homeostasis and neurotransmitter levels were determined following termination of exposure (postnatal day 14), while behavioral changes were assessed in adulthood. CuxOy inhalation modified cortical metal homeostasis and produced male-biased disruption of striatal neurotransmitters, with marked increases in dopaminergic function, as well as excitatory/inhibitory imbalance and reductions in serotonergic function. Impulsive-like behaviors in a fixed ratio (FR) waiting-for-reward schedule and a fixed interval (FI) schedule of food reward occurred in both sexes, but more prominently in males, effects which could not be attributed to altered locomotor activity or short-term memory. Inhaled Cu as from AP exposures, at environmentally relevant levels experienced during development, may contribute to impaired brain function, as shown by its ability to disrupt brain metal homeostasis and striatal neurotransmission. In addition, its ability to evoke impulsive-like behavior, particularly in male offspring, may be related to striatal dopaminergic dysfunction that is known to mediate such behaviors. As such, regulation of air Cu levels may be protective of public health.

Effects of contingent and noncontingent reinforcement on the emotional behavior of domestic dogs (Canis familiaris)

Intermittent schedules of reinforcement are frequently used in dog training. During fixed-time (FT) schedules, reinforcement delivery is time-based and response-independent. In contrast, during fixed-interval (FI) schedules, a response produces a reinforcer if a certain amount of time has elapsed since the last reinforcer was delivered (i.e., FI schedules are time-based and response-dependent). FT schedules are typically used in noncontingent-reinforcement interventions with the aim of decreasing an undesired behavior. Although the effectiveness of these interventions has been reliably shown with human and nonhuman animal learners, little is known about the emotional behavior that is related to these contingencies. In the present study, the target behavior (stepping on a mat) of six dogs was initially trained (shaped) and maintained on a FI 15s schedule. In addition, dogs were exposed to a FT 15s schedule, a combination of FT 15s and FI 15s schedules, and an extinction-only condition (no reinforcement was delivered for the target response). We found that rates of emotional responses (e.g., lip/nose licking, vocalizing) varied in prevalence across dogs and phases. Although lip/nose licking was displayed in high rates across all dogs and phases, it was generally lower in baseline and extinction phases. This suggests that these phases produced less frustration-related behavior. The effect-size calculations for dogs engaging in pacing, sneezing, and vocalizing underscored the varied prevalence of the behaviors across phases and dogs and yielded mostly small- to medium-size effects. These results are especially relevant for the training of companion and working dogs and their well-being during training sessions. Future research should continue investigating emotional behavior in dogs across various schedules of reinforcement (e.g., variable schedules).

Motivated to time: Effects of reinforcer devaluation and opportunity cost on interval timing.

Prior research suggests that interval timing performance is sensitive to reinforcer devaluation effects and to the rate of competing sources of reinforcement. The present study sought to replicate and account for these findings in rats. A self-paced concurrent fixed-interval (FI) random-ratio (RR) schedule of reinforcement was implemented in which the FI requirement varied across training conditions (12, 24, 48 s). The RR requirement-which imposed an opportunity cost to responding on the FI component-was adjusted so that it took about twice the FI requirement, on average, to complete it. Probe reinforcer devaluation (prefeeding) sessions were conducted at the end of each condition. To assess the effect of contextual reinforcement on timing performance, the RR requirement was removed before the end of the experiment. Consistent with prior findings, performance on the FI component tracked schedule requirement and displayed scalar invariance; the removal of the RR component yielded more premature FI responses. For some rats, prefeeding reduced the number of trials initiated without affecting timing performance; for other rats, prefeeding delayed responding on the FI component but had a weaker effect on trial initiation. These results support the notion that timing and motivational processes are separable, suggesting novel explanations for ostensible motivational effects on timing performance.

Topographical aspects of brief-stimulus presentations: A re-examination of the problem of conditioned reinforcement

Three pigeons were exposed to second-order schedules in which responding under a fixed-interval (FI) component schedule was reinforced according to a variable-interval (VI) schedule of food reinforcement. Completion of each component resulted in either (1) brief presentation of a stimulus present during reinforcement (paired brief stimulus), (2) brief presentation of a stimulus not present during reinforcement (nonpaired brief stimulus), or (3) no stimulus presentation (tandem schedule). Under the two nonpaired brief stimulus conditions, either a change in keylight color or onset of houselight illumination was used as the brief stimulus. Similar patterns of keypecking occurred under tandem and nonpaired keylight brief-stimulus presentations, whereas nonpaired houselight brief-stimulus presentations generated positively accelerated within-component keypeck patterning for two pigeons. When the same keylight brief stimulus was paired with food, positively accelerated patterns of keypecking were obtained for all pigeons. Differences in the effects of nonpaired brief-stimulus presentations on second-order schedule performance suggest that component schedule patterning under nonpaired brief-stimulus procedures is a function of the particular type of stimulus used (i.e., houselight versus keylight). These results suggest that (1) brief houselight illumination may function as a sensory reinforcer, and (2) a briefly presented food-paired stimulus can function as an effective conditioned reinforcer.

Behavioral resistance as function of different levels of instruction completeness

The present study examined the effects of different levels of instruction completeness on the behavioral persistence of humans engaged in a computer task. Five undergraduate students responded in a three-component multiple schedule of reinforcement during baseline. In two components, responses produced points according to a fixed-interval (FI) 5 s schedule of reinforcement. In the third component, extinction was programmed, and no instruction was available (No-Instruction Component - NI). The complete instruction "Press once every 5 seconds to earn points" appeared on the computer screen during one FI component (Complete-Instruction Component – CI), and the minimal instruction "Press to earn points" appeared on the computer screen during the other FI component (Minimal-Instruction Component – MI). The reinforcement rate was equal between the FI components. Increases in response force disturbed responding during test relative to baseline. Overall, greater persistence occurred in the component correlated with the complete instruction (i.e., CI component), suggesting that different levels of instruction completeness can differentially affect behavioral persistence when the reinforcement rate is held constant.

A behavioral timing intervention upregulates striatal serotonergic markers and reduces impulsive action in adult male mice

Many studies support the hypothesis that time-based interventions reduce impulsive behavior in rodents. However, few studies have directly assessed 1) how such interventions affect impulsive action rather than impulsive choice, 2) if intervention effects differ by sex, and 3) how time-based interventions affect neurochemistry in regions mediating decision-making and reward. Thus, we assessed how a fixed-interval (FI) intervention initiated during late adolescence and extending into adulthood affected dopaminergic and serotonergic analytes in the frontal cortex and striatum and subsequent impulsive action in adult male and female mice. Beginning on postnatal day (PND) 45, mice were either trained on a progressive series of FI schedules (FI 20, 40, & 60 s) or remained in the home cage. Following the intervention, increases in striatal serotonergic analytes were found in FI-exposed males and females (n = 8/sex/group) with few changes found in the frontal cortex. Impulsive action was assessed in the remaining mice (n = 10/sex/group) using a fixed-ratio waiting-for-reward (FR-wait) task in which completion of an FR-25 component initiated a “free” pellet component in which pellets were delivered at increasing intervals according to a fixed delay increment that varied across sessions. Responses reset the additive delay and initiated a new FR-25 component. FI-exposed males, but not females, showed fewer delay resets and no-wait resets relative to control mice. Importantly, FI-exposure did not affect discrimination reversal performance in either sex. These data suggest that time-based interventions may reduce impulsive action in addition to impulsive choice perhaps with increased male sensitivity. Additionally, time-based interventions appear to operate through striatal serotonergic augmentation.

Spatial organization of rat’s behavior in a situation with three optional lever-pressing responses under continuous and fixed-interval schedules of food

We assessed spatiotemporal distribution of behavior as a function of the spatial and temporal arrangement of food delivery. Eight food-deprived rats were exposed in an extended experimental chamber to a continuous reinforcement schedule (CRf) or to a fixed interval (FI) 30 s, with three levers contiguous to food dispenser or placed on separate panels. Responses on any of the levers were equally operative on the ongoing schedule. All rats moved mainly near to dispenser and levers when these were contiguous, under both, CRf and FI schedules, although distribution of responses on different levers was more variant under FI than under CRf. With separate levers, rats traveled through a larger area of the experimental space, especially under FI. Distribution of responses showed more variation under FI than under CRf schedule. Under FI, response patterns consisted in a post-meal pause followed by an acceleration towards the next food. Findings are discussed in the context of studies in which have been reported inconsistent patterns under FI schedules when are manipulated the number of operanda, dispensers, and their locations. It is highlighted the relevance of continuous analysis of rats’ displacement, to account for the configuration of response patterns observed in this study.

Robust Kalman Smoothers With Linear Equality Constraints

It is well known that the Kalman filter (KF) performance is clearly degraded in real-life applications where the system model is misspecified to a certain extent, that is, when the assumed knowledge does not perfectly match the true system dynamics. A possible way to mitigate the impact of a system model mismatch is to resort to linear constraints. In that perspective, a general linearly constrained KF (LCKF) formulation has been recently derived and shown to provide an effective robust filtering solution. In this contribution, we extend the LCKF framework to linear smoothers. Adapting the Fixed-Interval (FI) smoother is easy with sequential state augmentation, but the obtained linearly constrained FI (LCFI) has quadratic complexity in the interval length. Hence the need for an efficient alternative: the linearly constrained Rauch-Tung-Striebel (LCRTS) smoother. Introduced in an intuitive manner, its consistency is proved by showing that the LCFI and LCRTS coincide. The mismatch mitigation capabilities and performance of the LCRTS, with respect to unconstrained smoothers, are shown through illustrative examples with different types of mismatch in both measurement and process equations.

Resurgence during transitions from variable- to fixed-interval schedules

The effects of local periods of extinction on resurgence following transitions from variable-interval (VI) to fixed-interval (FI) schedules were studied using four pigeons exposed to a within-session resurgence procedure. Each session was divided into a Training (T) Alternative-Reinforcement (AR), and Resurgence Test (RT) phase. During the T phase, key pecking was reinforced under a VI 60-s schedule on one key. In the AR phase, responses reinforced in the T phase were extinguished, while responses to a different key were reinforced under a VI 90-s schedule. Next, responding to the same key that produced reinforcers in the AR phase was reinforced according to four different RT conditions: RT phase I (FI 90 s), RT phase II (FI 180 s), RT phase III (FI 45 s), or RT phase IV (extinction). The frequency of resurgence generally was an inverse function of the rate of reinforcement in the RT phase. Resurgence occurred less often when reinforcers were delivered under the FI 45-s schedule and more often under leaner schedules in the RT phase, peaking under extinction. The results show that resurgence may occur during local periods of extinction, with larger and more consistent effects occurring when the rate of reinforcement in the RT condition is leaner than it was during the preceding AR phase.

Marc Richelle (1930–2021) and the Study of Temporal Regulation of Behaviour in Animals

This article discusses the contribution of Marc Richelle to the study of temporal regulation of behaviour in animals. Richelle was a pioneer of behavioural pharmacology in Europe in the 1960s, and some of his early pharmacological experiments, particular those involving chlordiazepoxide, are discussed. Richelle frequently tested drug effects on performance on fixed-interval (FI) and differential reinforcement of low rate (DRL) schedules. Much of his later work, conducted with Helga Lejeune, involved cross-species comparisons of performance on FI and DRL, and often focused on potential differences between “timing competence” and “timing performance”. His work provided an unrivalled body of research on operant behaviour in different species, involving research on animals as different as cats and fish. Much of the work was reviewed in Richelle and Lejeune’s 1980 book Time in Animal Behaviour, which contained particularly influential accounts of collateral behaviour and inter-species comparisons.

Assessment of the ‘timing’ function of schedule-induced behavior on fixed-interval performance.

It has been suggested that schedule-induced behaviors allow organisms to adapt better to temporal regularities of the environment. The main goal of the present study was to observe the effect of schedule-induced drinking (SID) on the performance in fixed-interval (FI) schedules. Rats were exposed to a FI 15-, 30-, or 60-s food reinforcement schedule, and only half of them had access to water in the experimental chamber. Rats with access to water developed SID, which occurred in the first part of the interval, regardless of the FI value, and was followed by an increase in lever pressing rate. There were no substantial differences in the quantitative measures of timing between groups that had or did not have access to water, except for the rats in the FI 15-s group with access to water, who showed longer postreinforcement pauses, possibly attributable to competition between SID and lever pressing. SID did not manifest the scalar property, contrary to lever pressing, but it is proposed that behaviors are displayed serially until the last behavior before the target operant response becomes a discriminative stimulus for that behavior. It is not assumed that the purpose of schedule-induced behaviors is to aid timing, but the development of behavioral patterns might determine the performance of organisms on temporal tasks. Additionally, in some cases competition between responses might exert more control on when the operant behavior occurs than timing. Timing seems to consist in the temporal organization of available behaviors that leads to a specific behavior occurring at a specified time, a single characteristic that typically had come to indicate accurate timing. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Timing and Intertemporal Choice Behavior in the Valproic Acid Rat Model of Autism Spectrum Disorder.

Recently it has been proposed that impairments related to autism spectrum disorder (ASD) may reflect a more fundamental disruption in time perception. Here, we examined whether in utero exposure to valproic acid (VPA) can generate specific behavioral deficits related to ASD and time perception. Pups from control and VPA groups were tested using fixed-interval (FI) temporal bisection, peak interval, and intertemporal choice tasks. In addition, the rats were assessed on motor function, perseverative and exploratory behavior, anxiety, and memory. The VPA group displayed a leftward shift in timing functions. VPA rats displayed no deficits on the motor and memory tasks, but were significantly different from controls on measures of perseveration and anxiety.

Stimulus generalization of behavioral history: Interspecies generality and persistence of generalization gradients.

Four pigeons were exposed to a tandem variable-interval (VI) fixed-ratio (FR) schedule in the presence of a 50-pixel (about 15 mm) square or an 80-pixel (about 24 mm) square and to a tandem VI differential-reinforcement-of-low-rate (DRL) schedule when a second 80-pixel or 50-pixel square was present. The values of the VI and FR schedules were adjusted to equate reinforcement rates in the two tandem schedules. Following this, a square-size continuum generalization test was administered under a fixed-interval (FI) schedule or extinction. In the first testing session, response frequency was a graded function of the similarity of the test stimuli to the training stimuli for all pigeons. These systematic generalization gradients persisted longer under the FI schedule than under extinction.

Activation of VTA GABA neurons disrupts reward seeking by altering temporal processing

The role of ventral tegmental area (VTA) dopamine in reward, cue processing, and interval timing is well characterized. Using a combinatorial viral approach to target activating DREADDs (Designer Receptors Exclusively Activated by Designer Drugs, hM3D) to GABAergic neurons in the VTA of male rats, we previously showed that activation disrupts responding to reward-predictive cues. Here we explored how VTA GABA neurons influence the perception of time in two fixed interval (FI) tasks, one where the reward or interval is not paired with predictive cues (Non-Cued FI), and another where the start of the FI is signaled by a constant tone that continues until the rewarded response is emitted (Cued FI). Under vehicle conditions in both tasks, responding was characterized by “scalloping” over the 30 s FI, in which responding increased towards the end of the FI. However, when VTA GABA neurons were activated in the Non-Cued FI, the time between the end of the 30 s interval and when the rats made a reinforced response increased. Additionally, post-reinforcement pauses and overall session length increased. In the Cued FI task, VTA GABA activation produced erratic responding, with a decrease in earned rewards. Thus, while both tasks were disrupted by VTA GABA activation, responding that is constrained by a cue was more sensitive to this manipulation, possibly due to convergent effects on timing and cue processing. Together these results demonstrate that VTA GABA activity disrupts the perception of interval timing, particularly when the timing is set by cues.

Effects of pre‐operant running and sucrose concentration on operant wheel running on a fixed interval schedule of reinforcement

Prior research proposed that temporal control over the pattern of operant wheel running on a fixed interval (FI) schedule of sucrose reinforcement is a function of automatic reinforcement generated by wheel running and the experimentally arranged sucrose reinforcement. Two experiments were conducted to assess this prediction. In the first experiment, rats ran for different durations (0, 30, 60, and 180 min) prior to a session of operant wheel running on a FI 120-s schedule. In the second experiment, the concentration of sucrose reinforcement on a FI 180-s schedule was varied across values of 0, 5, 15, and 25%. In Experiment 1, as the duration of pre-operant running increased, the postreinforcement pause before initiation of running lengthened while wheel revolutions in the latter part of the FI interval increased. In Experiment 2, wheel revolutions markedly increased then decreased to a plateau early in the FI interval. Neither manipulation increased temporal control of the pattern of wheel running. Instead, results indicate that operant wheel running is regulated by automatic reinforcement generated by wheel activity and an adjunctive pattern of running induced by the temporal presentation of sucrose. Furthermore, the findings question whether the sucrose contingency regulates wheel running as a reinforcing consequence.

Resurgence and repeated within-session progressive-interval thinning of alternative reinforcement.

Resurgence of a previously suppressed target behavior is common when reinforcement for a more recently reinforced alternative behavior is thinned. To better characterize such resurgence, these experiments examined repeated within-session alternative reinforcement thinning using a progressive-interval (PI) schedule with rats. In Experiment 1, a transition from a high rate of alternative reinforcement to a within-session PI schedule generated robust resurgence, but subsequent complete removal of alternative reinforcement produced no additional resurgence. Experiment 2 replicated these findings and showed similar effects with a fixed-interval (FI) schedule arranging similarly reduced session-wide rates of alternative reinforcement. Thus, the lack of additional resurgence following repeated exposure to the PI schedule was likely due to the low overall obtained rate of alternative reinforcement provided by the PI schedule, rather than to exposure to within-session reinforcement thinning per se. In both experiments, target responding increased at some point in the session during schedule thinning and continued across the rest of the session. Rats exposed to a PI schedule showed resurgence later in the session and after more cumulative alternative reinforcers than those exposed to an FI schedule. The results suggest the potential importance of further exploring how timing and change-detection mechanisms might be involved in resurgence.

COMPARING COORDINATED RESPONDING IN PAIRS OF RATS WHEN BOTH OPERANTS AND AGGREGATE PRODUCTS ARE SELECTED BY INTERMITTENT TEMPORAL CONSEQUENCES

Three pairs of rats worked in two adjacent operant chambers separated by a Plexiglas wall. In some experimental conditions, lever-presses were independent operants maintained by either a fixed-interval (FI) or variable interval (VI) schedule. In other conditions, each rat lever- press was part of two interlocked behavior contingencies as part of a metacontingency: its aggregate product, a brief flash of light, was maintained either by a FI or a VI schedule of mutual water presentation. The results clearly show that the interlocking of behavioral contingencies in the design of an experimental metacontingency established a pattern of cooperation between the rats.

Effects of benztropine analogs on delay discounting in rats.

Methylphenidate and d-amphetamine, medications used for treatment of attention deficit hyperactivity disorder (ADHD), are used recreationally and self-administered by laboratory animals. Benztropine (BZT) analogs, like those medications, increase synaptic dopamine levels but are less effective in maintaining self-administration, suggesting clinical utility with less abuse liability. The current study was designed to evaluate potential therapeutic effects of BZT analogs related to ADHD. Rats responded under a delay-discounting procedure in which responses on one lever produced immediate delivery of a single food pellet and alternative responses produced four food pellets either immediately or with various temporal delays, with those delays arranged in ascending or random orders in different groups of rats. Selection of the smaller more immediate reinforcer has been suggested as an aspect of "impulsivity," a trait with suggested involvement in ADHD. Other rats were studied under fixed-interval (FI) 300-s schedules to assess drug effects on behavior under temporal control. d-Amphetamine, methylphenidate, and the BZT analog AHN 1-055, but not AHN 2-005 or JHW 007, increased selection of the large, delayed reinforcer with either arrangement of delays. All drugs changed the temporal distribution of responses within the FI from one with responses concentrated at the end to a more uniform distribution. Changes in the temporal distribution of FI responding occurred with drugs that did not affect discounting suggesting that discounting does not arise directly from the same temporal control processes controlling FI responding. AHN 1-055 may be of clinical utility in the treatment of ADHD.