Interresponse Times Research Articles

The active time model of concurrent choice.

The following paper describes a steady-state model of concurrent choice, termed the active time model (ATM). ATM is derived from maximization principles and is characterized by a semi-Markov process. The model proposes that the controlling stimulus in concurrent variable-interval (VI) VI schedules of reinforcement is the time interval since the most recent response, termed here "the active interresponse time" or simply "active time." In the model after a response is generated, it is categorized by a function that relates active times to switch/stay probabilities. In the paper the output of ATM is compared with predictions made by three other models of operant conditioning: melioration, a version of scalar expectancy theory (SET), and momentary maximization. Data sets considered include preferences in multiple-concurrent VI VI schedules, molecular choice patterns, correlations between switching and perseveration, and molar choice proportions. It is shown that ATM can account for all of these data sets, while the other models produce more limited fits. However, rather than argue that ATM is the singular model for concurrent VI VI choice, a consideration of its concept space leads to the conclusion that operant choice is multiply-determined, and that an adaptive viewpoint-one that considers experimental procedures both as selecting mechanisms for animal choice as well as tests of the controlling variables of that choice-is warranted.

Behavioral signatures of the rapid recruitment of long-term memory to overcome working memory capacity limits.

Working- and long-term memory are often studied in isolation. To better understand the specific limitations of working memory, effort is made to reduce the potential influence of long-term memory on performance in working memory tasks (e.g., asking participants to remember artificial, abstract items rather than familiar real-world objects). However, in everyday life we use working- and long-term memory in tandem. Here, our goal was to characterize how long-term memory can be recruited to circumvent capacity limits in a typical visual working memory task (i.e., remembering colored squares). Prior work has shown that incidental repetitions of working memory arrays often do not improve visual working memory performance - even after dozens of incidental repetitions, working memory performance often shows no improvement for repeated arrays. Here, we used a whole-report working memory task with explicit rather than incidental repetitions of arrays. In contrast to prior work with incidental repetitions, in two behavioral experiments we found that explicit repetitions of arrays yielded robust improvement to working memory performance, even after a single repetition. Participants performed above chance at recognizing repeated arrays in a later long-term memory test, consistent with the idea that long-term memory was used to rapidly improve performance across array repetitions. Finally, we analyzed inter-item response times and we found a response time signature of chunk formation that only emerged after the array was repeated (inter-response time slowing after two to three items); thus, inter-item response times may be useful for examining the coordinated interaction of visual working and long-term memory in future work.

The temporal structure of goal-directed and habitual operant behavior.

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Neural evidence of switch processes during semantic and phonetic foraging in human memory

Humans may retrieve words from memory by exploring and exploiting in "semantic space" similar to how nonhuman animals forage for resources in physical space. This has been studied using the verbal fluency test (VFT), in which participants generate words belonging to a semantic or phonetic category in a limited time. People produce bursts of related items during VFT, referred to as "clustering" and "switching." The strategic foraging model posits that cognitive search behavior is guided by a monitoring process which detects relevant declines in performance and then triggers the searcher to seek a new patch or cluster in memory after the current patch has been depleted. An alternative body of research proposes that this behavior can be explained by an undirected rather than strategic search process, such as random walks with or without random jumps to new parts of semantic space. This study contributes to this theoretical debate by testing for neural evidence of strategically timed switches during memory search. Thirty participants performed category and letter VFT during functional MRI. Responses were classified as cluster or switch events based on computational metrics of similarity and participant evaluations. Results showed greater hippocampal and posterior cerebellar activation during switching than clustering, even while controlling for interresponse times and linguistic distance. Furthermore, these regions exhibited ramping activity which increased during within-patch search leading up to switches. Findings support the strategic foraging model, clarifying how neural switch processes may guide memory search in a manner akin to foraging in patchy spatial environments.

Stimulus control of a social operant.

Three pigeon dyads were exposed to a two-component multiple schedule comprised of two tandem variable-interval 30-s interresponse time (IRT) > 3-s schedules in the presence of different stimuli. Pecks to keys by both pigeons of a dyad occurring within 500 ms of one another were required for reinforcement under one tandem schedule (the coordination component), and such coordinated responses were not required under the other (the control component). The terminal link of each schedule ensured that the reinforced coordination response was an IRT > 3 s. Rates of coordinated IRTs > 3 s and total rates of coordinated responses (composed of IRTs > 3 s and IRTs ≤ 3 s) were higher in the coordination components than in either of two different control components in which coordination was not required for reinforcement. This difference in coordinated responses in the presence and absence of the coordination requirement under stimulus control transitorily deteriorated and then was reestablished when the relation between the stimulus and the coordination contingency or its absence was reversed. The results show coordinated responding to function as a discriminated social operant.

Comparison of progressive hold and progressive response schedules of reinforcement

Progressive ratio (PR) schedules have been widely used to study motivation to work for a reinforcer. After a post-reinforcer pause, subjects engage in pressing a lever until a reinforcer is obtained. However, the discrete nature of lever presses allows alternative behaviors during inter-response time and has led to several behavioral explanations of pauses and work time. A progressive hold-down (PH) is incompatible with alternative responses and may allow a precise estimation of work time. Performance of rats trained in both PR and PH that received sucrose or intracranial self-stimulation (ICSS) as reinforcer were compared. We observed that rats mastered the PR and PH schedules. Post-reinforcer pauses (PSRP), work time and inter-reinforcer time increased as a function of the response or hold requirement. However, rats’ performance suggested that the PH progression may be experienced by the rats as easier that the PR progression. Elimination of consummatory behavior with ICSS reduced post-reinforcer pause in accordance with predictions of explanatory models of fixed and variable schedules of reinforcement. In the case of PH performance, pauses showed little variation across intermediate requirements but increased rapidly on later requirements; since rats controlled their pause length and work time was close to the hold requirement, time allocation between PR and PH schedules diverged. Finally, the Mathematical Principles of Reinforcement model of Bradshaw and Killeen (Psychopharmacology 2012, 222: 549) rendered a good description of the performance in both PR and PH schedules.

Use of a Preassessment to Inform Treatment of Rapid Eating.

Rapid eating is a common and potentially dangerous behavior among individuals diagnosed with autism spectrum disorder (ASD; Favell et al. Behavior Modification, 4, 481-492,1980). Although limited research has shown efficacy in treating rapid eating using procedures that increase interresponse time between bites, the literature on preassessment methods to inform treatment remains limited. Therefore, the purpose of the present study was to replicate and extend procedures used by Page et al.Behavior Analysis in Practice, 10, 87-91 (2016) to effectively reduce the rapid eating of an adolescent male diagnosed with ASD through the incorporation of a preassessment and treatment package including a vibrating pager, vocal rule, and response blocking. Overall, results of the study demonstrated that the preassessment was effective in determining foods to be included in treatment, and the treatment package was effective in increasing average interresponse time between bites. Additions to the current literature as well as limitations to be addressed in future research are discussed.

Task parameters influence operant response variability in mice.

During operant conditioning, animals associate actions with outcomes. However, patterns and rates of operant responding change over learning, which makes it difficult to distinguish changes in learning from general changes in performance or movement. Thus, understanding how task parameters influence movement execution is essential. To understand how specific operant task parameters influenced the repetition of future operant responses, we investigated the ability of operant conditioning schedules and contingencies to promote reproducible bouts of five lever presses in mice. Mice were trained on one of the four operant tasks to test three distinct hypotheses: (1) whether a cue presented concurrently with sucrose delivery influenced the pattern of lever pressing; (2) whether requiring animals to collect earned sucrose promoted the organization of responses into bouts; and (3) whether only reinforcing bouts where interresponse time (IRT) variances were below a target promoted reproducible patterns of operant behavior. (1) Signaling reinforcer delivery with a cue increased learning rates but resulted in mice pressing the lever in fast succession until the cue turned on, rather than executing discrete bouts. (2) Requiring mice to collect the reinforcer between bouts had little effect on behavior. (3) A training strategy that directly reinforced bouts with low variance IRTs was not more effective than a traditional fixed ratio schedule at promoting reproducible action execution. Together, our findings provide insights into the parameters of behavioral training that promote reproducible actions and that should be carefully selected when designing operant conditioning experiments.

Inter-response time shaping by percentile schedule with discrete trial procedure

The percentile schedule has been used as a convenient procedure for the automatic shaping of operant responses. Experiments 1, 2, and 4 of this study examined whether two variables related to the percentile schedule—“update the reinforcement criterion for each response” and “retreat of the reinforcement criterion”—are critical factors of the percentile schedule on shaping. Experiment 3 examined the effect of combining one of the shaping rule “reinforce movement, not position” with the percentile schedule. Eight pigeons were exposed to a discrete trial procedure. In each trial, pigeons were required to peck one key and then to the other, and the interval between pecks was defined as the inter-response time (IRT). A percentile schedule was applied to this IRT aiming to lengthen it. The results showed that (1) IRTs were lengthened by the percentile schedule; (2) "update the reinforcement criterion for each response," "retreat of the reinforcement criterion," and "reinforce movement, not position" had no significant effect on lengthening IRT; and (3) the frequency of IRT occurrence decreased when the "retreat of the reinforcement criterion" was removed. These results indicate that the aforementioned percentile schedule variables are not critical for the percentile schedule on IRT shaping. Because the discrete trial procedure can dissociate the effects of variables on the length and frequency of IRT, this procedure seems effective for future shaping studies.

Burstiness and Stochasticity in the Malleability of Physical Activity.

This study examined whether patterns of self-organization in physical activity (PA) predicted long-term success in a yearlong PA intervention. Increased moderate to vigorous PA (MVPA) was targeted in insufficiently active adults (N = 512) via goal setting and financial reinforcement. The degree to which inverse power law distributions, which are reflective of self-organization, summarized (a)daily MVPA and (b)time elapsed between meeting daily goals (goal attainment interresponse times) was calculated. Goal attainment interresponse times were also used to calculate burstiness, the degree to which meeting daily goals clustered in time. Inverse power laws accurately summarized interresponse times, but not daily MVPA. For participants with higher levels of MVPA early in the study, burstiness in reaching goals was associated with long-term resistance to intervention, while stochasticity in meeting goals predicted receptiveness to intervention. These results suggest that burstiness may measure self-organizing resistance to change, while PA stochasticity could be a precondition for behavioral malleability.

Assessment of antidepressant-like effects of dextromethorphan on differential reinforcement of low-rate 72-s performance in rats.

The effectiveness of ketamine for treatment-resistant depression along with several other clinical advantages, such as rapid onset and reduced adverse effects associated with serotonin transporter inhibition, has garnered interest in other similar acting psychedelics as novel antidepressant drugs. The antitussive dextromethorphan exhibits glutamate N-methyl-d-aspartate receptor antagonism, sigma-1 receptor agonism, and serotonin reuptake inhibition, which has exhibited antidepressant effects in limited human studies and animal models. The present study sought to further examine dextromethorphan using a differential reinforcement of low-rate 72-s schedule, which can be used to screen antidepressant drugs, in male and female rats. The tricyclic antidepressant drug imipramine and the psychostimulant d-amphetamine also were examined. Sex differences were not shown for baseline performance or for the drugs tested. Further, performance did not differ between the estrus and diestrus stages. Dextromethorphan alone and with quinidine produced an antidepressant-like effect by reducing the number of responses emitted, increasing the number of reinforcers earned, and shifting inter-response times to the right, although significant response suppression occurred at these doses. An antidepressant-like effect was shown with imipramine, but d-amphetamine increased the number of responses emitted and did not affect the number of reinforcers earned. The present findings provide additional support for antidepressant effects produced by dextromethorphan.

Reducing rapid eating in adults with Down syndrome: Using token reinforcement to increase interresponse time between bites.

Reducing rapid eating in adults with Down syndrome: Using token reinforcement to increase interresponse time between bites.

Versatile Fusion of Renown Pediculicidal Agents: An Innovative Thought

Two Macaca mulatta and one macaca java monkey were trained to press a key for food pellets on a differential reinforcement of low rate (DRL) schedule which required consecutive key-press responses to be spaced at least 20 sec apart in order to produce reinforcement. When behavior had stabilized, the monkeys were administered either 0.0,0.05,0.10,0.25, or 0.50 mg/kg of cocaine i.v. during daily sessions. Cocaine produced a bi-component, dose-dependent effect. Immediately following cocaine infusion there was a dose-related complete cessation of responding which began and ended abruptly. Following this postinfusion pause, a dose-related behavioral disruption in the form of shortened inter-response times was observed.

Longer operant lever-press duration requirements induce fewer but longer response bouts in rats.

Operant behavior is organized in bouts that are particularly visible under variable-interval (VI) schedules of reinforcement. Previous research showed that increasing the work required to produce a response decreases the rate at which bouts are emitted and increases the minimum interresponse time (IRT). In the current study, the minimum effective IRT was directly manipulated by changing the minimum duration of effective lever presses reinforced on a VI 40-s schedule. Contrary to assumptions of previous models, response durations were variable. Response durations were typically 0.5 s greater than the minimum duration threshold; durations that exceeded this threshold were approximately log-normally distributed. As the required duration threshold increased, rats emitted fewer but longer bouts. This effect may reflect an effort-induced reduction in motivation and a duration-induced facilitation of a response-outcome association.

Lack of effect of dopamine receptor blockade on SKF83959-altered operant behavior in male rats.

Dopamine (DA) is important for the performance of operant behavior as revealed by psychopharmacological studies that manipulate the activity of DA subtype receptors. However, the effects of SKF83959, an atypical DA D1 receptor agonist, on operant behavior and the underlying pharmacological mechanisms remain unclear. The present study sought to determine whether blockade of DA D1- and D2-subtyped receptors would reverse the operant behavior altered by SKF83959. Male rats were trained to respond on either a fixed-interval 30 s (FI30) schedule or a differential reinforcement of low-rate response 10 s (DRL10) schedule, two timing-relevant tasks but with distinct reinforcement contingencies. Pharmacological evaluation was conducted with injection of a selective D1 (or D2) receptor antagonist alone or in combined with SKF83959 (1.0 mg/kg) following a stable baseline of behavioral performance. The results showed that SKF83959 treatment alone significantly disrupted the performance of FI30 and DRL10 behaviors mainly by showing the decreases of the response-related measures, and the distinct profiles of the behavior altered by the drug were manifested by the qualitative analysis of inter-response time data on both tasks. The effects of SKF83959 were not significantly affected/reversed by the pretreatment of either SCH23390 or eticlopride injected at the doses of 0.02 and 0.06 mg/kg; however, a subtle reversal effect was observed in the treatment of low-dose eticlopride. Despite that these results confirm the FI30 and DRL10 behaviors changed by SKF83959, the absence of pharmacological reversal effect by DA receptor antagonist suggests that either D1- or D2-subtyped receptors may not play a critical role in the alteration of timing-relevant operant behavior produced by SKF83959.

Tactical contingencies in the experimental analysis of reinforcement and operant classes

In his "Tactics of Scientific Research" (1960), his work on avoidance, his discovery of equivalence classes and his cautions on applications of coercion, Murray Sidman created high standards for behavior analytic research. I illustrate his influence in the context of three examples he might have characterized as pilot studies. Each examined trial N+1 response probabilities depending on whether trial N responding had produced a reinforcer. Differentially reinforced interresponse times, keys pecked in arbitrary matching, and two-key response sequences provided no robust evidence that reinforcing some response property on trial N raises the probability of responding with that property on trial N+1. These negative findings shed light on the nature of operant classes and on the relation of reinforcers to the responses that produced them. Through selection, reinforcers create operant classes and engender variations of the responses within those classes; operant classes are held together by common contingencies. Sidman extended our understanding of operant classes by expanding them to include equivalence relations.

Traumatic brain injury substantially reduces the conditioned reinforcing effects of environmental cues in rats

Traumatic brain injury affects millions of people each year and is an established risk factor for addiction. Recent animal studies have causally demonstrated that injuries can increase drug self-administration across a variety of substances. One potential behavioral mediator for this finding is an increased responsivity to drug-associated cues. This endophenotype can be identified by profiling non-drug-related behaviors. The current study evaluated several paradigms (conditioned approach, conditioned reinforcement, extinction from variable interval responding, conditioned facilitation) to determine how rats with a frontal TBI differed in their response to Pavlovian conditioning in response to food-paired cues. Surprisingly, rats with a TBI demonstrated increased goal-tracking in a conditioned approach paradigm and exerted less effort for a conditioned reinforcer. Moreover, they had slightly facilitated extinction (as demonstrated by significantly larger interresponse times) in the face of reinforcer-associated cues. Despite these effects, TBI rats still demonstrated conditioned facilitation to an auditory stimulus. Together, these effects suggest a phenotype in the opposite direction of what might be anticipated. Cues still served a strong discriminative function and altered behavior; however, they did not function as strong conditioned reinforcers for TBI animals. One potential reason for this is that substantial changes to the dopamine system after TBI may reduce the conditioned reinforcing effects of cues, but sensitize the brain to potent drugs of abuse. More research will be needed to determine whether this is the case.

The effects of force requirements on microstructure: Subcriterion responses and the mismeasurement of bouts.

The effects of response force on microstructure were evaluated. A strain-gauge operandum permitted the manipulation of the force required to produce reinforcers (criterion responses) independently from the force defining response threshold. Thus, we could detect subcriterion forces that fell short of the force criterion. Eight rats earned food according to variable-interval (VI) 30- and 120-s schedules. The force requirements were set to 5.6 or 32.0 g; the response threshold was fixed at 5.6 g. Interresponse times were computed when subcriterion responses were both included and omitted from the analysis. Log-survivor functions of interresponse times showed that increasing force requirements elevated the mean between-bout interval of the VI 120-s schedule, but only if subcriterion behavior was excluded. Omitting subcriterion responses thus leads to overestimation of intervals separating response bouts. Increasing force requirements also increased the skewness of the between-bout distribution. A subsequent analysis found that subcriterion responses are most plentiful following reinforcer delivery, which helps to explain why their omission might inflate between-bout intervals, as this period is an important transition from reinforcer consumption to engagement in operant activity. The data suggest caution interpreting the effects of force on microstructure when subcriterion behavior is not or cannot be measured.

Activation of cannabinoid type 1 receptors decreases the synchronization of local field potential oscillations in the hippocampus and entorhinal cortex and prolongs the interresponse time during a differential-reinforcement-of-low-rate task.

Marijuana intoxication impairs neurocognitive functions. Common side effects of consuming cannabis include time distortion and memory loss. However, the underlying neurophysiological mechanisms involved in these effects remain unclear. We hypothesized that communication between the hippocampal CA1 region and medial entorhinal cortex (MEC) is essential for the transmission of temporal-associated information. We used a differential-reinforcement-of-low-rate (DRL) task, which requires subjects to press a lever at an optimal time point, to correlate the distributions of interresponse time (IRT) with local field potentials (LFPs) recorded in the CA1 and MEC under the effects of a cannabinoid type 1 (CB1) receptor agonist. We used a DRL 10-s schedule and trained the rats to withhold for 10s before pressing a lever. Our data showed that the percentage of 12.4- to 14-s IRT events rose after activation of CB1 receptors in the MEC. In addition, gamma amplitude synchronization and CA1 theta phase-MEC gamma amplitude coupling decreased during the 6- to 14-s IRT events. These results suggest that activation of CB1 receptors in the MEC disrupt the functional connectivity between the CA1 and the MEC. This inefficient communication may result in increased IRT during a DRL schedule. Overall, we postulate that marijuana intoxication impairs the communication between the CA1 and MEC and influences behavioral performances that require precise timing ability.

Dissociating central and auditory attention: Is there a shared bottleneck for response selection and auditory search?

Central and auditory attention are limited in capacity. In dual-tasks, central attention is required to select the appropriate response, but because central attention is limited in capacity, response selection can only be carried out for one task at a time. In auditory search tasks, search time to detect the target sound increases with the number of distractor sounds added to the auditory scene (set sizes), indicating that auditory attention is limited in capacity. Here, we investigated whether central and auditory attention relied on common or distinct capacity limitations using a dual-task paradigm. In two experiments, participants completed a visual choice discrimination task (task 1) together with an auditory search task (task 2), and the two tasks were separated by an experimentally modulated stimulus onset asynchrony (SOA). Analysing auditory search time as a function of SOA and set sizes (locus-of-slack method) revealed that the auditory search process in task 2 was performed after response selection in a visual two-choice discrimination task 1 (Experiment 1), but concurrently with response selection in a visual four-choice discrimination task 1 (Experiment 2). Hence, although response selection in the visual four-choice discrimination task demanded more central attention as compared with response selection in the two-choice discrimination task, the auditory search process was performed in parallel. Distribution analyses of inter-response time further indicated that parallel processing of response selection and auditory search was not influenced by response grouping. Taken together, the two experiments provided evidence that central and auditory attention relied on distinct capacity limitations.