Bout Initiation Rate Research Articles

Effect of instructions on the microstructure of human schedule performance.

Three experiments examined the effect of instructions on human free-operant performance on random ratio (RR) and random interval (RI) schedules. Both rates of responding, and the microstructure of behavior, were explored to determine whether bout-initiation and within-bout responding may be controlled by different processes. The results demonstrated that responding in acquisition (Experiments 1 and 2) and extinction (Experiment 3) was impacted in line with given instructions. During acquisition, rates were higher on RR compared to RI for accurate and minimal instructions. During extinction, rates decreased when there were minimal instructions. However, instructions had a greater impact on within-bout responding, than they did on bout-initiation responding. Overall rates of responding, and within-bout rates, varied in line with the nature of the instructions, but bout-initiation responding did not (Experiments 1 and 2). Resistance to extinction was increased by instructions in terms of overall responding and within-bout rates, but not in terms of bout-initiation rates (Experiment 3). These data are consistent with the hypothesis that bout-initiation responding may be less impacted by instructions than within-bout responding, speculatively, the former is stimulus-driven, automatic/habitual, and less accessible to conscious processing. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The temporal structure of goal-directed and habitual operant behavior.

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

The effect of brief mindfulness training on the micro-structure of human free-operant responding: Mindfulness affects stimulus-driven responding

Background and objectivesThe current study examines the extent to which mindfulness impacts on operant conditioning processes, and explores the suggestion that mindfulness training serves to make humans more sensitive to the current reinforcement contingencies with which they are presented. In particular, the effect of mindfulness on the micro-structure of human schedule performance was explored. It was expected that mindfulness might impact bout-initiation responding to a greater degree than within-bout responding, premised on the assumption that bout-initiation responses are habitual and not under conscious control, but within-bout responses are goal-directed and conscious. MethodsNonclinical participants experienced one of three brief (15min) interventions: focused attention breathing exercise (mindfulness), an unfocused attention breathing exercises, or no intervention. They then responded on a multiple random ratio (RR) random interval (RI) schedule. ResultsIn the no intervention and unfocused attention groups, overall and within-bout response rates were higher on the RR than the RI schedule, but bout-initiation rates were the same on the two schedules. However, for the mindfulness groups all forms of responding were higher for the RR than the RI schedule. Previous work has noted that habitual, and/or unconscious or fringe-conscious events, are impacted by mindfulness training. LimitationsA nonclinical sample may limit generality. ConclusionsThe current pattern of results suggests that this is also true in schedule-controlled performance, and offers an insight into the manner in which mindfulness alongside conditioning-based interventions, to bring all responses under conscious control.

Simulating bout-and-pause patterns with reinforcement learning.

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice-an agent makes a choice between operant and other behaviors. The second mechanism is cost-a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

Factors controlling the micro-structure of human free-operant behaviour: Bout-initiation and within-bout responses are effected by different aspects of the schedule

Two experiments examined factors controlling human free-operant performance in relation to predictions based on the nature of bout-initiation and within-bout responding. Overall, responding was higher for a random ratio (RR) than a random interval (RI) schedule, with equal rates of reinforcement. Bout-initiation rates were not different across the two schedules, but within-bout rates were higher on the RR schedule. Response cost reduced overall rates of responding, but tended to suppress bout-initiation responding more than within-bout responding (Experiments 1 & 2). In contrast, reinforcement magnitude increased all forms of responding (Experiment 2). One explanation consistent with these effects is that bout-initiation responses are controlled by overall rates of reinforcement through their impact on the context (i.e. are stimulus-driven), but that within-bout responses are controlled by response reinforcement (i.e. are goal-directed). These current findings are discussed in the light of these theoretical suggestions.

Modeling bout–pause response patterns in variable-ratio and variable-interval schedules using hierarchical Bayesian methodology

Streams of operant responses are arranged in bouts separated by pauses and differences in performance in reinforcement schedules with identical inter-reinforcement intervals (IRIs) are primarily due to differences in within–bout response rate, not in bout–initiation rate. The present study used hierarchical Bayesian modeling as a new method to quantify the properties of the response bout. A Bernoulli distribution was utilized to express the probability to stay in bout/pause, while a Poisson distribution was utilized to quantify the within–bout response rates. We compared bout/pause patterns between variable–ratio (VR) and variable–interval (VI) schedules across IRIs. The model estimation revealed no difference in within-bout staying probability between schedules. However, response rates of within–bout responses were higher in VR than VI across IRIs. These results are consistent with previous analyses using a log–survivor plot to describe within–bout responses and bouts–initiation responses. In addition, a simulation study was performed to examine how sensitively the model estimate the parameters according to different bout initiation rates. These result showed that the within–bout staying probability was affected by changes in between–bout while within–bout response rate parameters were not. This suggests model estimation robustness of the model estimation to dissociate within–bout and between–bout parameters during different reinforcement schedules.

Human performance on random interval schedules.

Four experiments explored the factors controlling human responding on random interval (RI) schedules of reinforcement. All experiments identified 2 types of responding: "bout-initiation" and "within-bout" responding. Responding on RI schedules was related to the interval value rates, being higher on an RI-30s than on an RI-60s or RI-120s schedule, which impacted bout-initiation responding to the greater degree (Experiments 1 and 3). Experiment 2 found similar overall response rates on random ratio (RR) and random interval with a linear feedback loop (RI+) schedules, with both higher than on an RI schedule. Bout-initiation rates were similar across all schedules, but within-bout responding differed. Experiments 3 and 4 examined the impact of a response cost and noted greater bout-initiation responding but not greater within-bout rates with low costs. Overall, these experiments suggest that bout-initiation responding may be subject to control by factors that increase the strength of conditioning to the context, whereas within-bout responding is less sensitive to these influences. (PsycINFO Database Record

The effect of lever height on the microstructure of operant behavior

The effect of lever height on the temporal organization of reinforced lever pressing was examined. Lever pressing was reinforced on a variable-interval 30-s schedule in rats, with lever height manipulated across six successive conditions. Parameters of the organization of responses in bouts (bout length distribution, bout-initiation rate, within-bout rate, and sequential dependency) were estimated. These estimates revealed (1) a qualitative change in the distribution of IRTs and their sequential dependency when the lever was too high, (2) a mixture of geometrically-distributed bout lengths at all lever heights, and (3) longer bouts at lower and intermediate lever heights. In accordance with previous data, these findings suggest that lower and intermediate lever heights favored lever pressing with longer bout lengths, faster bout initiation, faster within-bout responding, and more sequentially dependent timing. These results underscore the disociability of motoric capacity in operant performance, and may reflect the influence of the body size on the temporal organization of the operant.

A bout analysis of operant response disruption

Operant behavior appears to be organized in bouts of responses, whose parameters are differentially sensitive to various manipulations. This study investigated potential differential effects of three forms of operant response disruption—extinction (EXT), non-contingent reinforcement (NCR), and prefeeding (PRE)—on response bouts. In Experiment 1, Wistar Kyoto rats (WKY) were trained on a tandem variable-time (VT) 120s fixed-ratio (FR) 5 schedule of reinforcement; after stability was established, their responding was disrupted for three sessions with one of the three disrupters (EXT, NCR, or PRE). In Experiment 2, Long Evans (LE) rats were trained on a tandem VT 240s FR 5 to stability, and their responding disrupted with EXT or NCR. In EXT and NCR, response rates declined significantly and progressively over the course of the session, primarily due to a declining bout-initiation rate in EXT, and to fewer responses per bout in NCR. In contrast, a session-wide drop in response rate was observed in PRE, primarily due to a reduction in bout-initiation rate at the start of the session. These findings suggest that each form of disruption differentially impacts dissociable aspects of behavior. Theories of behavioral persistence should account for these functional relations, which appear to be obscured in response rate measures.

Aging, motor function, and sensitivity to calcium channel blockers: An investigation using chronic methylmercury exposure

Methylmercury (MeHg) neurotoxicity is thought to be mediated, in part, by dysregulation of calcium (Ca2+) homeostasis, a mechanism that may also slowly and progressively degrade neuronal function during normal aging. Longitudinal studies of MeHg exposure provide a powerful approach to studying neural and behavioral mechanisms by which both MeHg toxicity and aging affect motor function. Wheel-running and rotarod performance were assessed in two age groups of BALB/c mice chronically exposed to 0 or 1.2mg/kg/day MeHg and 0 or 20mg/kg/day nimodipine, a 1,4-dihyrdopyridine L-type calcium channel blocker (CCB), for approximately 8.5 months. Adults began exposure on postnatal day (PND) 72 and retired breeders on PND 296. A log-survivor bout analysis partitioned wheel-running into bouts that identified motor (within-bout rates) and motivational (bout-initiation rates) influences. Retired breeders ran farther, because of a higher bout-initiation rates, but performed more poorly on the rotarod than younger adults, a difference unaffected by nimodipine. MeHg produced relatively age-independent deficits in wheel-running and rotarod performance, whereas nimodipine afforded greater protection to adult mice than to retired breeders. Rotarod performance and within-bout response rate were more sensitive to and more reliable predictors of MeHg toxicity than bout-initiation rate, which was least affected by MeHg exposure. Thus the motivation to run was unimpaired as the ability to do so declined. While chronic MeHg exposure produced functionally similar behavior deficits between age groups, the age-dependent neuroprotection by nimodipine supports the notion that underlying neurobiological systems mediated by Ca2+ signaling, are differentially affected in older adults.

A bout analysis reveals age-related methylmercury neurotoxicity and nimodipine neuroprotection

Age-related deficits in motor and cognitive functioning may be driven by perturbations in calcium (Ca2+) homeostasis in nerve terminals, mechanisms that are also thought to mediate the neurotoxicity of methylmercury (MeHg). Calcium-channel blockers (CCBs) protect against MeHg toxicity in adult mice, but little is known about their efficacy in other age groups. Two age groups of BALB/c mice were exposed to 0 or 1.2mg/kg/day MeHg and 0 or 20mg/kg/day of the CCB nimodipine for approximately 8.5 months. Adults began exposure on postnatal day (PND) 72 and the retired breeders on PND 296. High-rate operant behavior was maintained under a percentile schedule, which helped to decouple response rate from reinforcer rate. Responding was analyzed using a log-survivor bout analysis approach that partitioned behavior into high-rate bouts separated by pauses. MeHg-induced mortality did not depend on age but nimodipine neuroprotection was age-dependent, with poorer protection occurring in older mice. Within-bout response rate (a marker of sensorimotor function) was more sensitive to MeHg toxicity than bout-initiation rate (a marker of motivation). Within-bout rate declined almost 2 months prior to overt signs of toxicity for the MeHg-only retired breeders but not adults, suggesting greater delay to toxicity in younger animals. Motor-based decrements also appeared in relatively healthy adult MeHg+NIM animals. Aging appeared to alter the processes underlying Ca2+ homeostasis thereby diminishing protection by nimodipine, even in mice that have not reached senescence. The study of MeHg exposure presents an experimental model by which to study potential mechanisms of aging.

Extinction under a behavioral microscope: Isolating the sources of decline in operant response rate

Extinction performance is often used to assess underlying psychological processes without the interference of reinforcement. For example, in the extinction/reinstatement paradigm, motivation to seek drug is assessed by measuring responding elicited by drug-associated cues without drug reinforcement. However, extinction performance is governed by several psychological processes that involve motivation, memory, learning, and motoric functions. These processes are confounded when overall response rate is used to measure performance. Based on evidence that operant responding occurs in bouts, this paper proposes an analytic procedure that separates extinction performance into several behavioral components: (1-3) the baseline bout initiation rate, within-bout response rate, and bout length at the onset of extinction; (4-6) their rates of decay during extinction; (7) the time between extinction onset and the decline of responding; (8) the asymptotic response rate at the end of extinction; (9) the refractory period after each response. Data that illustrate the goodness of fit of this analytic model are presented. This paper also describes procedures to isolate behavioral components contributing to extinction performance and make inferences about experimental effects on these components. This microscopic behavioral analysis allows the mapping of different psychological processes to distinct behavioral components implicated in extinction performance, which may further our understanding of the psychological effects of neurobiological treatments.

Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat

BackgroundOperant hyperactivity, the emission of reinforced responses at an inordinately high rate, has been reported in children with ADHD and in the Spontaneously Hypertensive Rat (SHR), the most widely studied animal model of ADHD. The SHR emits behavior at hyperactive levels, relative to a normoactive strain, only when such behavior is seldom reinforced. Because of its dependence on rate of reinforcement, operant hyperactivity appears to be driven primarily by incentive motivation, not motoric capacity. This claim was evaluated in the present study using a novel strategy, based on the organization of behavior in bouts of reinforced responses separated by pauses.MethodMale SHR, Wistar-Kyoto (WKY) and Wistar rats (WIS) were exposed each to a multiple variable-interval schedule of sucrose reinforcement (12, 24, 48, 96, and 192 s) between post-natal days (PND) 48 and 93. Responding in each schedule was examined in two epochs, PND 58-62 and 89-93. Parameters of response-reinforcement functions (Herrnstein's hyperbola) and bout-organized behavior were estimated in each epoch.ResultsSHR emitted higher response rates than WKY and WIS, but only when rate of reinforcement was low (fewer than 2 reinforcers per minute), and particularly in the second epoch. Estimates of Herrnstein's hyperbola parameters suggested the primacy of motivational over motoric factors driving the response-rate differential. Across epochs and schedules, a more detailed analysis of response bouts by SHR revealed that these were shorter than those by WKY, but more frequent than those by WKY and WIS. Differences in bout length subsided between epochs, but differences in bout-initiation rate were exacerbated. These results were interpreted in light of robust evidence linking changes in bout-organization parameters and experimental manipulations of motivation and response-reinforcement contingency.ConclusionsOperant hyperactivity in SHR was confirmed. Although incentive motivation appears to play an important role in operant hyperactivity and motoric capacity cannot be ruled out as a factor, response-bout patterns suggest that operant hyperactivity is primarily driven by steeper delay-of-reinforcement gradients. Convergence of this conclusion with theoretical accounts of ADHD and with free-operant performance in children with ADHD supports the use of SHR as an animal model of ADHD.

THE ISOLATION OF MOTIVATIONAL, MOTORIC, AND SCHEDULE EFFECTS ON OPERANT PERFORMANCE: A MODELING APPROACH

Dissociating motoric and motivational effects of pharmacological manipulations on operant behavior is a substantial challenge. To address this problem, we applied a response-bout analysis to data from rats trained to lever press for sucrose on variable-interval (VI) schedules of reinforcement. Motoric, motivational, and schedule factors (effort requirement, deprivation level, and schedule requirements, respectively) were manipulated. Bout analysis found that interresponse times (IRTs) were described by a mixture of two exponential distributions, one characterizing IRTs within response bouts, another characterizing intervals between bouts. Increasing effort requirement lengthened the shortest IRT (the refractory period between responses). Adding a ratio requirement increased the length and density of response bouts. Both manipulations also decreased the bout-initiation rate. In contrast, food deprivation only increased the bout-initiation rate. Changes in the distribution of IRTs over time showed that responses during extinction were also emitted in bouts, and that the decrease in response rate was primarily due to progressively longer intervals between bouts. Taken together, these results suggest that changes in the refractory period indicate motoric effects, whereas selective alterations in bout initiation rate indicate incentive-motivational effects. These findings support the use of response-bout analyses to identify the influence of pharmacological manipulations on processes underlying operant performance.

Using pentobarbital to assess the sensitivity and independence of response-bout parameters in two mouse strains

A recently developed model posits that a bout of operant responding comprises three different components: bout initiation rate, within-bout response rate and bout length. Each parameter is thought to be affected by different classes of variables. Pentobarbital was used to assess the independence and sensitivity of these parameters in two mouse strains, BALB/c and C57BL/6, selected because of their different behavioral characteristics. With or without a running wheel present, BALB/c and C57BL/6 mice nose-poked under a Percentile 10:0.5 schedule designed to select high response rates while holding reinforcement rate constant. Baseline rates of nose-poking were higher for BALB/c mice than for C57BL/6 mice, but no strain difference occurred in baseline distance run. Nose-poking occurred at a higher rate when the wheel was absent from the chamber for both strains, and this was due to longer bout lengths and higher bout initiation rates. Nose-poke rates were increased by the 5.6–17mg/kg doses of pentobarbital, especially in C57BL/6 mice. Pentobarbital only decreased running. No strain differences in pentobarbital sensitivity were observed for running. Whether reinforcement was extrinsic or intrinsic to the response was hypothesized to influence pentobarbital's effects. The different bout parameters helped dissect pentobarbital's effects and were selectively affected by pentobarbital.

High-rate operant behavior in two mouse strains: A response-bout analysis

Operant behavior sometimes occurs in bouts characterized by an initiation rate, within-bout response rate, and bout length. The generality of this structure was tested using high-rate nose-poking in mice. Reinforcement of short interresponse times produced high response rates while a random-interval schedule held reinforcement rates constant. BALB/c mice produced bouts that were more frequent, longer, and contained a higher within-bout rate of responding (nine nose-pokes/s) than did the C57BL/6 mice (five nose-pokes/s). Adding a running wheel decreased total nose-pokes and bout length, and increased bout-initiation rate. Free-feeding reduced nose-poking by decreasing bout-initiation rate. Photoperiod reversal decreased bout-initiation rate but not total nose-poke rate. Despite strain differences in bout structure, both strains responded similarly to the interventions. The three bout measures were correlated with overall rate but not with each other. Log-survival analyses provided independent descriptors of the structure of high-rate responding in these two strains.

RESISTANCE TO CHANGE OF RESPONDING MAINTAINED BY UNSIGNALED DELAYS TO REINFORCEMENT: A RESPONSE‐BOUT ANALYSIS

Previous experiments have shown that unsignaled delayed reinforcement decreases response rates and resistance to change. However, the effects of different delays to reinforcement on underlying response structure have not been investigated in conjunction with tests of resistance to change. In the present experiment, pigeons responded on a three-component multiple variable-interval schedule for food presented immediately, following brief (0.5 s), or following long (3 s) unsignaled delays of reinforcement. Baseline response rates were lowest in the component with the longest delay; they were about equal with immediate and briefly delayed reinforcers. Resistance to disruption by presession feeding, response-independent food during the intercomponent interval, and extinction was slightly but consistently lower as delays increased. Because log survivor functions of interresponse times (IRTs) deviated from simple modes of bout initiations and within-bout responding, an IRT-cutoff method was used to examine underlying response structure. These analyses suggested that baseline rates of initiating bouts of responding decreased as scheduled delays increased, and within-bout response rates tended to be lower in the component with immediate reinforcers. The number of responses per bout was not reliably affected by reinforcer delay, but tended to be highest with brief delays when total response rates were higher in that component. Consistent with previous findings, resistance to change of overall response rate was highly correlated with resistance to change of bout-initiation rates but not with within-bout responding. These results suggest that unsignaled delays to reinforcement affect resistance to change through changes in the probability of initiating a response bout rather than through changes in the underlying response structure.

Response rate viewed as engagement bouts: resistance to extinction.

Rats obtained food pellets by nose poking a lighted key, the illumination of which alternated every 50 s during a session between blinking and steady, signaling either a relatively rich (60 per hour) or relatively lean (15 per hour) rate of reinforcement. During one training condition, all the reinforcers in the presence of the rich-reinforcement signal were response dependent (i.e., a variable-interval schedule); during another condition only 25% were response dependent (i.e., a variable-time schedule operated concurrently with a variable-interval schedule). An extinction session followed each training block. For both kinds of training schedule, and consistent with prior results, response rate was more resistant to extinction in the presence of the rich-reinforcement signal than in the presence of the lean-reinforcement signal. Analysis of interresponse-time distributions from baseline showed that differential resistance to extinction was not related to baseline differences in the rate of initiating response bouts or in the length of bouts. Also, bout-initiation rate (like response rate) was most resistant to extinction in the presence of the rich-reinforcement signal. These results support the proposal of behavioral momentum theory (e.g., Nevin & Grace, 2000) that resistance to extinction in the presence of a discriminative stimulus is determined more by the stimulus-reinforcer (Pavlovian) than by the stimulus-response-reinforcer (operant) contingency.