Interresponse Times Research Articles

Effects of a Reinforcement Schedule Controlling Energy of Pigeons’ Pecking Response

Operant behavior involves many physical dimensions, including time, force, location, and distance. Energy is an important dimension in foraging behavior or motor response. Kono (Learning & Behavior, 41, 425–432, 2013) developed a schedule in which the controlling variable was the energy of responses, calculated by dividing the squared value of the distance between two consecutive responses by the squared value of the interresponse time, or the elapsed time between the two responses. This energy schedule demonstrated the possibility of differential reinforcement of the energy of responses, but the effect was relatively weak. Kono (Learning & Behavior, 41, 425–432, 2013) proposed that this weak effect could have been caused by two features of the adjusting procedure: the use of two or more schedules and variation in the reinforced energy of responses across trials depending on the subjects’ previous choice. The present study employed a single energy schedule whose requirement was fixed throughout the session. The results showed that the energy of responses increased as the requirement of the energy schedule increased. In addition, the degree of increase was greater for distance of responses than for time of responses. These results suggest that the energy schedule is a suitable method for controlling the energy of behavior, and the distance of responses has a greater effect than time.

Effects of effort and difficulty on human preference for a stimulus: Investigation of the within-trial contrast.

The within-trial contrast hypothesis (WTC) provides a more parsimonious explanation for the phenomenon that humans and animals prefer outcomes that follow more effortful events to outcomes that follow less effortful events (Zentall, 2013). We conducted two WTC experiments with human adults. In Experiment 1, we manipulated the difficulty of a preceding event by varying the interresponse time and the limited-hold interval during differential reinforcement with a low response rate schedule, to examine the effect of effort on the preference for the subsequent stimuli. In Experiment 2, we attempted to identify the variables that had affected the results of Experiment 1, by manipulating time as the delay of reinforcement. The results showed preferences based on WTC only when participants made a high rate of incorrect responses in the preceding event, which was used as an index of the strength of individual effort. These results extend the findings of previous human WTC studies and suggest that the difficulty of a task could serve as an aversive event that affects the WTC effect. It is possible that an index based on performance in the preceding event would provide useful information for predicting the contrast effect.

Response-bout analysis of interresponse times in variable-ratio and variable-interval schedules

The purpose of this study was to analyze the response pattern difference between variable-ratio (VR) and variable-interval (VI) schedules of reinforcement by modeling interresponse time distributions of rats’ lever presses. All eight rats showed higher response rates under VR 30 than under inter-reinforcement intervals yoked VI. The 30 models consisting of single Exponential (with and without the lower limit on interresponse times), Weibull, Normal, Log-Normal or Gamma distributions, all possible two component combinations of those, and 3 and 4 component models consisting of Weibull, Normal, Log-Normal, or Gamma distribution combinations were compared. The 4 component Log-Normal model was the best in terms of the Akaike information criterion and visual inspection of fitting outcome. Parameter estimates for the L4 model showed that the VR-VI response rate difference is due to a difference in short interresponse times or within bout responses. This results suggests that the VR-VI response rate difference is not an indication of a difference in the overall tendency to respond but it is rather a difference in terms of what types of response patterns are engendered between the two schedules.

Reduction of Rapid Eating in an Adolescent Female with Autism.

Rapid eating, a potentially dangerous and socially inappropriate behavior, has received relatively little attention in the literature. This study sought to extend the research in this area by further evaluating the effectiveness of a vibrating pager combined with a rule for increasing inter-response time between bites in one adolescent female diagnosed with autism. Results indicated that inter-response time increased from baseline only after a vocal prompt to "wait" was introduced across clinic and home settings. Implications for promoting autonomy in individuals with developmental disabilities are discussed.This antecedent-based intervention can easily be generalized to caregiversThe unobtrusive nature of the intervention allows for implementation in inclusive settingsThere are implications for promoting social skills in naturalistic environmentsThe intervention can promote independence through teaching self-management.

Interval timing under a behavioral microscope: Dissociating motivational and timing processes in fixed-interval performance.

The distribution of latencies and interresponse times (IRTs) of rats was compared between two fixed-interval (FI) schedules of food reinforcement (FI 30s and FI 90s), and between two levels of food deprivation. Computational modeling revealed that latencies and IRTs were well described by mixture probability distributions embodying two-state Markov chains. Analysis of these models revealed that only a subset of latencies is sensitive to the periodicity of reinforcement, and prefeeding only reduces the size of this subset. The distribution of IRTs suggests that behavior in FI schedules is organized in bouts that lengthen and ramp up in frequency with proximity to reinforcement. Prefeeding slowed down the lengthening of bouts and increased the time between bouts. When concatenated, latency and IRT models adequately reproduced sigmoidal FI response functions. These findings suggest that behavior in FI schedules fluctuates in and out of schedule control; an account of such fluctuation suggests that timing and motivation are dissociable components of FI performance. These mixture-distribution models also provide novel insights on the motivational, associative, and timing processes expressed in FI performance. These processes may be obscured, however, when performance in timing tasks is analyzed in terms of mean response rates.

Modeling Symptoms of Attention-Deficit Hyperactivity Disorder in a Rat Model of Fetal Alcohol Syndrome.

Several studies indicate the similarity between the symptoms of fetal alcohol syndrome and attention-deficit hyperactivity disorder (ADHD). This study hypothesized that prenatal exposure to ethanol (EtOH) can be used as an animal model of ADHD in Wistar rats. At the first stage of the study, alcohol was delivered to the pregnant dams (237-252 g) by intra-gastric route throughout Gestation Days 8-20 at a dose of 6 g/kg/day. Untreated control group with isocaloric sucrose intubation was also included. Of the 16 male pups (174-180 g), 8 were in the fetal alcohol effects (FAE) group and 8 were in the untreated control group. Subjects went through behavior shaping, discrimination learning and reversal learning. Number of sessions to learn the tasks, response frequency to inhibitory (S-) and excitatory (S+) stimulus features, response latency and inter-response time (IRT) were measured. Significant differences were obtained on only the reversal task. Rats with FAE needed greater number of sessions to learn the reversal task, and they had a higher frequency of incorrect responses in specifically the latter part of the sessions. Our results suggest that reversal learning of FAE rats exhibits deficit in the inhibition of pre-learned responses. Responses behaviorally mimicked attention deficit and impulsivity symptoms of human ADHD. However, the experimental design of the study was not conducive to hyperactivity. Accordingly, rats with FEA can be an alternative to other models since it is not, for example, based on a symptom that is atypical (such as hypertension) to ADHD. Significant difference was obtained in a reversal task between male rats prenatally exposed to ethanol and matched controls. The greater number of sessions for learning and higher frequency of incorrect responses behaviorally mimicked symptoms of ADHD, suggesting that rats with fetal ethanol effects can serve as a useful animal model.

Minor Functional Deficits in Basic Response Patterns for Reinforcement after Frontal Traumatic Brain Injury in Rats.

Traumatic brain injury (TBI) is a major contributor to numerous psychiatric conditions and chronic behavioral dysfunction. Recent studies in experimental brain injury have begun to adopt operant methodologies to assess these deficits, all of which rely on the process of reinforcement. No studies have directly examined how reinforced behaviors are affected by TBI, however. The current study assessed performance under the four most common schedules of reinforcement (fixed ratio, variable ratio, fixed interval, variable interval) and one higher order schedule assessing motivation (progressive ratio) after bilateral, pre-frontal controlled cortical impact injury. TBI-induced differences on the basic schedules were minor, with the exception of the variable ratio, where increased efficacy (more reinforcers, higher response rates, lower interresponse times) at higher requirements was observed as a result of brain injury. Performance on the progressive ratio schedule showed some gross differences between the groups, in that sham rats became more efficient under this schedule while injured rats perseverated in lever pressing. Further, injured rats were specifically impaired at lower response requirements on the progressive ratio. Taken together, these findings indicate that simple reinforced behaviors are mostly unaffected after TBI, except in the case of variable ratio schedules, but the altered performance on the higher-order progressive ratio schedule suggests changes involving motivation or potentially perseveration. These findings validate operant measures of more complex behaviors for brain injury, all of which rely on reinforcement and can be taken into consideration when adapting and developing novel functional assessments.

Distributed Frequency Control via Randomized Response of Electric Vehicles in Power Grid

In this paper, we propose a new distributed frequency control scheme for electric vehicles (EVs) to help restore the power grid frequency upon a contingency of supply-demand imbalance. Under our scheme, each EV independently monitors the grid frequency at discrete times and responds by switching among its charging, idle, and discharging operational modes according to a simple threshold-based switching algorithm. To recover the grid frequency smoothly and prevent an undesired frequency overshoot/undershoot due to simultaneous response of EVs, we design the inter-response times of any EV to follow an exponentially distributed random variable with a certain mean value at each operational mode. To draw insights into the performance of our scheme, we characterize its impacts on the grid frequency in various aspects, including the mean and variance of the resulting grid frequency over time, the mean frequency recovery time, the average number of EV switching their modes, and the probability of frequency overshoot/undershoot. Accordingly, we formulate an optimization problem for the grid operator to minimize the expected cost of implementing our frequency control scheme by designing EVs response rates subject to their requested incentive prices and the given grid performance guarantees. Finally, we validate our analysis via simulations on the IEEE 9-Bus test system and the Ireland power system, where it is observed that our frequency control scheme can be used as a reliable and cost-efficient alternative for the conventional primary reserve service.

The Utility of Impulsive Bias and Altered Decision Making as Predictors of Drug Efficacy and Target Selection: Rethinking Behavioral Screening for Antidepressant Drugs.

Cognitive dysfunction may be a core feature of major depressive disorder, including affective processing bias, abnormal response to negative feedback, changes in decision making, and increased impulsivity. Accordingly, a translational medicine paradigm predicts clinical action of novel antidepressants by examining drug-induced changes in affective processing bias. With some exceptions, these concepts have not been systematically applied to preclinical models to test new chemical entities. The purpose of this review is to examine whether an empirically derived behavioral screen for antidepressant drugs may screen for compounds, at least in part, by modulating an impulsive biasing of responding and altered decision making. The differential-reinforcement-of-low-rate (DRL) 72-second schedule is an operant schedule with a documented fidelity for discriminating antidepressant drugs from nonantidepressant drugs. However, a theoretical basis for this empirical relationship has been lacking. Therefore, this review will discuss whether response bias toward impulsive behavior may be a critical screening characteristic of DRL behavior requiring long inter-response times to obtain rewards. This review will compare and contrast DRL behavior with the five-choice serial reaction time task, a test specifically designed for assessing motoric impulsivity, with respect to psychopharmacological testing and the neural basis of distributed macrocircuits underlying these tasks. This comparison suggests that the existing empirical basis for the DRL 72-second schedule as a pharmacological screen for antidepressant drugs is complemented by a novel hypothesis that altering impulsive response bias for rodents trained on this operant schedule is a previously unrecognized theoretical cornerstone for this screening paradigm.

Rats show molar sensitivity to different aspects of random-interval-with-linear-feedback-functions and random-ratio schedules.

Three experiments examined the impact of various aspects of reinforcement contingencies on responding maintained by free-operant schedules by food-deprived rats. Experiment 1 demonstrated that random interval (RI) and random-interval-with-positive-response-reinforcer-feedback (RI+) schedules maintained similar rates of responding at a variety of reinforcer frequencies. Experiment 2 demonstrated that a random ratio (RR) schedule maintained higher rates than RI or RI+ schedules, except at high rates of reinforcement, where response rates were similar on all schedules. Experiment 3 again demonstrated that RR schedules produced higher response rates than either RI or RI+ schedules, but modification of the RI+ schedule to prevent ratio strain enhanced response rates relative to an RI schedule. Together these results reveal a pattern of interacting factors in schedule controlled behavior: at high rates of reinforcement, this factor overrides the impact of other controlling factors, but as reinforcement rate decreases the joint impact of interresponse times reinforcement, response-reinforcer feedback functions, and ratio strain are observed.

Effects of Intranasal Oxytocin on Aggressive Responding in Antisocial Personality Disorder

The oxytocin receptor is important in several domains of social behavior, and administration of oxytocin modulates social responding in several mammalian species, including humans. Oxytocin has both therapeutic and scientific potential for elucidating the neural and behavioral mechanisms governing social behavior. In the present study, operationally-defined aggressive behavior of six males with Antisocial Personality Disorder (ASPD) was measured following acute intranasal oxytocin dosing (12, 24, and 48 international units) and placebo, using a well-validated laboratory task of human aggression (Point-Subtraction Aggression Paradigm, or PSAP). The PSAP provides participants with concurrently available monetary-earning and operationally-defined aggressive response options, maintained by fixed ratio schedules of consequences. Shifts in response rates and inter-response time (IRT) distributions were observed on the aggressive response option following oxytocin doses, relative to placebo. Few changes were observed in monetary-reinforced responding. However, across participants the direction and magnitude of changes in aggressive responding were not systematically related to dose. No trends were observed between psychometric or physiological data and oxytocin dosing or aggressive behavior. While this report is to our knowledge the first to examine the acute effects of oxytocin in this population at high risk for violence and other forms of antisocial behavior, several limitations in the experimental design and the results cast the study as a preliminary report. Strategies for more extensive future projects are discussed.

The distribution of response bout lengths and its sensitivity to differential reinforcement.

Response bouts are clusters of responses that occur in rapid succession and are punctuated by pauses during which the response does not occur. Under variable interval schedules of reinforcement, the number of responses in each bout (the bout length) varies among bouts. This experiment was aimed at determining whether the relative rate of reinforcement influenced the relative frequency of bouts of different lengths. Lever pressing in rats was reinforced under a tandem variable time (VT) 150-s fixed ratio (FR) X, where X could be 1 or 5 and varied randomly after each reinforcer. Two conditions were included: majority FR1 (mFR1) and majority FR5 (mFR5). In mFR1, 75% of reinforcers had a tandem FR requirement of 1 and 25% had a tandem FR requirement of 5; this distribution was reversed in mFR5. The dynamic bi-exponential refractory model of response bouts was fitted to the interresponse times (IRTs) in each condition. Model parameter estimates and IRTs were then used to simulate probable distributions of bout lengths. These distributions comprised a mixture of short geometrically-distributed bout lengths and long negative-binomially-distributed bout lengths. Long bouts were significantly longer in the mFR5 condition than in the mFR1 condition. In conjunction with previous data, the present study suggests that the prevalence of long bouts increases with the proportion of reinforcers with FR5 requirement. These results suggest that bouts of different lengths are sensitive to the rate at which they are reinforced.

Procedure for preventing response strain on random interval schedules with a linear feedback loop.

An experiment examined the impact of a procedure designed to prevent response or extinction strain occurring on random interval schedules with a linear feedback loop (i.e., an RI+ schedule). Rats lever-pressed for food reinforcement on either a RI+ or a random interval (RI) schedule that was matched to the RI+ schedule in terms of reinforcement rate. Two groups of rats responded on an RI+ and two on an RI schedule matched for rate of reinforcement. One group on each schedule also received response-independent food if there had been no response for 60s, and response-independent food continued to be delivered on an RT-60 schedule until a response was made. Rats on the RI and RI+ obtained similar rates of reinforcement and had similar reinforced inter-response times to one another. On the schedules without response-independent food, rats had similar rates of response to one another. However, while the delivery of response-independent food reduced rates of response on an RI schedule, they enhanced response rates on an RI+ schedule. These results suggest that rats can display sensitivity to the molar aspects of the free-operant contingency, when procedures are implemented to reduce the impact of factors such as extinction-strain.

A Need for Individual Data Analyses for Assessments of Temporal Control: Invertebrate Fixed Interval Performance

We outline several experimental variables and addressed the inconsistencies of these variables within the invertebrate fixed interval literature. We posit previous inconsistencies within the invertebrate fixed interval literature may be due to the utilization of aggregate versus individual analyses and contend individual analyses are critical in order for conclusions to be made about species’ abilities to emit responses that can come under temporal control. To exemplify these statements, we exposed honey bees Apis mellifera lingustica (N=13) to either an FI 15-sec or FI 30-sec schedule of reinforcement and analyzed subjects’ cumulative response records, response bin levels, quarter lives, inter-response time patterns, response duration patterns, and trial durations. No measures clearly indicated individual subjects’ responding came under temporal control of the fixed interval schedules; however, pooled group analyses did produce seemingly clear evidence of temporal control.

Concurrent performance as bouts of behavior.

Log-survivor analyses of interresponse times suggest that the behavior of rats responding under single variable-interval schedules is organized into bouts (i.e., periods of engagement and disengagement). Attempts to generalize this analysis to the key pecking in pigeons, however, have failed to produce the characteristic broken-stick appearance typically obtained with rats. This failure may be due to a relatively low rate of reinforcement for engaging in alternative behavior experienced by pigeons. The present study tested this hypothesis by exposing four pigeons to concurrent schedules of reinforcement for key pecking, first without a changeover delay (COD) and then with a COD. In this arrangement, one of the concurrent options was treated as the target response and the rate of reinforcement for that option was manipulated across conditions. The other option provided explicit reinforcement for engaging in an alternative response (i.e., explicit reinforcement for disengaging from the target response). In the absence of a COD, log-survivor plots for three of the pigeons were approximately linear, thus providing no evidence that responding was organized into bouts. When a COD was present, plots were broken stick in appearance, indicating a bout structure had been generated in the pigeons' behavior. Both bout length and the rate of bout initiations were a function of differences in rate of reinforcement. These data suggest that behavior may become organized into bouts when contingencies create sufficiently long visits to both the target behavior and the extraneous behavior. Fits of a double-exponential model deviated systematically from the actual plots due to the presence of a plateau between the two limbs. An alternative, double-gamma, model was explored, and it provided a considerably better fit than did the double-exponential.

Association between increased behavioral persistence and stereotypy in the pet dog

The aim of the present study was to evaluate whether dogs that exhibit stereotypy also show higher behavioral persistence in an extinction-learning task. Thirteen pet dogs with stereotypy and 13 breed-matched control dogs were assessed on a resistance to extinction test. Each dog was trained for 40 trials using a food reinforcer to nose-touch the experimenter's hand on a continuous reinforcement schedule. After acquisition, the dogs entered an extinction phase, during which food was no longer delivered. The numbers of nose-touches as well as inter-response times during this phase were recorded. A linear regression found that stereotypy status (t=−2.46, P=.027) and breed type (t=2.44, P=.023) were significant predictors of the number of responses in extinction. Dogs with stereotypy responded more in extinction than control dogs. The mean number of responses was 13.4 (SD=14.7) in the control group and 26.0 (SD=15.3) in the stereotypy group. These results suggest a link between previous laboratory and zoo animal findings on the neurophysiology of stereotypy and the pet dog population. They also have implications for the use of extinction procedures to reduce stereotypic behaviors in pet dogs, as these dogs show enhanced resistance to extinction.

Training Tolerance to Delay Using the Escalating Interest Task.

The present study examined the lasting effects of exposure to reinforcement that increased in magnitude as a function of time between responses in a first-person shooter video game preparation of the escalating interest task. When reinforcement density increased as a function of time, it encouraged participants to wait longer between responses (shots of a weapon). Participants exposed to such contingencies waited significantly longer to fire their weapons than participants who were exposed to linear growth, where long inter-response times were not differentially reinforced. Those with experience in conditions where reinforcement density increased as a function of time showed persistently longer wait times when the contingencies changed in the latter portion of the game where the disincentive to fire quickly was removed. The potential utility of such contingencies for training tolerance to delay of reinforcement and the broader implications of training self-control are discussed.

Preliminary efficacy of prize-based contingency management to increase activity levels in healthy adults.

An estimated 30% of Americans meet the criteria for obesity. Effective, low-cost interventions to increase physical activity are needed to prevent and treat obesity. In this study, 11 healthy adults wore Fitbit accelerometers for 3 weeks. During the initial baseline, subjects earned prize draws for wearing the Fitbit. During intervention, percentile schedules were used to calculate individual prize-draw criteria. The final week was a return to baseline. Four subjects increased step counts as a result of the intervention. A bout analysis of interresponse times revealed that subjects increased overall step counts by increasing daily minutes active and within-bout response rates and decreasing pauses between bouts of activity. Strategies to improve effectiveness are suggested, such as modification of reinforcement probability and amount and identification of the function of periods of inactivity.

Time-based reward maximization

Humans and animals time intervals from seconds to minutes with high accuracy but limited precision. Consequently, time-based decisions are inevitably subjected to our endogenous timing uncertainty, and thus require temporal risk assessment. In this study, we tested temporal risk assessment ability of humans when participants had to withhold each subsequent response for a minimum duration to earn reward and each response reset the trial time. Premature responses were not penalized in Experiment 1 but were penalized in Experiment 2. Participants tried to maximize reward within a fixed session time (over eight sessions) by pressing a key. No instructions were provided regarding the task rules/parameters. We evaluated empirical performance within the framework of optimality that was based on the level of endogenous timing uncertainty and the payoff structure. Participants nearly tracked the optimal target inter-response times (IRTs) that changed as a function of the level of timing uncertainty and maximized the reward rate in both experiments. Acquisition of optimal target IRT was rapid and abrupt without any further improvement or worsening. These results constitute an example of optimal temporal risk assessment performance in a task that required finding the optimal trade-off between the 'speed' (timing) and 'accuracy' (reward probability) of timed responses for reward maximization.

Ketamine, but not MK-801, produces antidepressant-like effects in rats responding on a differential-reinforcement-of-low-rate operant schedule

The noncompetitive N-methyl-D-aspartate (NMDA) receptor antagonist ketamine has been shown to produce rapid and lasting antidepressant effects in treatment-resistant patients with major depressive disorder and in preclinical behavioral assays. The present study sought to extend the preclinical antidepressant-like effects of ketamine using the differential-reinforcement-of-low-rate 72 s operant task in rats, as well as to determine whether the more selective and higher affinity NMDA receptor antagonist MK-801 produced antidepressant-like effects similar to those of ketamine. Ketamine, the NMDA receptor agonist N-methyl-D-aspartic acid, the tricyclic antidepressant imipramine, and the selective serotonin reuptake inhibitor fluoxetine all produced antidepressant-like effects by increasing the number of reinforcers, decreasing the number of responses, and producing a rightward shift in the peak location of the inter-response time distributions. Conversely, MK-801 and the dopamine receptor agonist D-amphetamine produced a psychostimulant-like effect by decreasing the number of reinforcers, increasing the number of responses, and producing a leftward shift in the peak location of the inter-response time distributions. Although a subeffective dose of ketamine attenuated the antidepressant-like effects of NMDA, a subeffective dose of NMDA did not alter the antidepressant-like effects of ketamine. These data indicate that ketamine and MK-801 produced dissociable effects in the differential-reinforcement-of-low-rate 72 s task, and further suggest that the underlying mechanisms responsible for the antidepressant effects of ketamine may be unique to ketamine and not shared by all NMDA receptor antagonists.