Variable-interval Schedules Research Articles

Repeated, within-session resurgence.

Resurgence is a reliable, transient effect that only occasionally is replicated more than once within a single experiment or subject. In the present experiments, within-session resurgence was generated repeatedly by dividing individual sessions into three phases (Training, Alternative-Reinforcement, and Resurgence-Test). In Experiments 1 and 2, resurgence reliably occurred in most of the 22-30 daily sessions when responding was reinforced on, respectively, fixed- and variable-interval schedules. Resurgence magnitude and duration did decrease across replications for some subjects, but not for others. To examine the utility of the procedure in studying the effects of an independent variable on resurgence, in Experiment 3 the effects of rich and lean baseline and alternative reinforcement rates on resurgence were compared. The target response was eliminated more rapidly, resurgence occurred more often, and usually was greater following rich alternative reinforcement rates. Resurgence was of greater magnitude when the baseline reinforcement rate was relatively lean compared to the alternative reinforcement rate. These experiments provide a reliable method for generating resurgence within individual sessions, instead of across multiple-session conditions, that can be repeated over many successive sessions.

Predator videos and electric shock function as punishers for zebrafish (Danio rerio).

Zebrafish (Danio rerio) are a promising animal model for studying the effects of gene-environment interactions on behavior. Two experiments were conducted to assess punishment effects of presenting predator videos (Indian leaf fish; Nandus nandus) and electric shock on operant approach responses in zebrafish. In Experiment 1, the predator video and shock stimuli were presented upon a response maintained by a single variable-interval schedule of food reinforcement in different groups of fish. In Experiment 2, the predator video and shock stimuli were presented upon one of two response alternatives maintain by concurrently available variable-interval schedules of food reinforcement in different groups of fish. Responding decreased when the predator video and shock stimuli were presented relative to their absence in both experiments. Moreover, responding on an unpunished alternative did not reliably decrease in Experiment 2. These results indicate that the decrease in responding resulted from the punishment contingency rather than from elicited species-specific defense responses or conditioned avoidance. Thus, the predator video and electric shock functioned as punishers of operant behavior for zebrafish. Identifying punishers for this species could lead to research on how gene-environment interactions influence individual differences in sensitivity to punishment.

Reinforcer magnitude and resurgence.

One experiment each was conducted with pigeons and rats to assess the effects of changes in reinforcer magnitude on resurgence. Each experiment involved three phases. In the Training phase, key pecking (Experiment 1) or lever pressing (Experiment 2) on two concurrently available operanda was reinforced according to variable-interval schedules. In the Alternative Reinforcement phase, responding to one operandum was extinguished while that to the other was reinforced with greater duration of food access (Experiment 1), greater number of pellets (Experiment 2a), or a similar number of pellets (Experiment 2b) than occurred in the Training phase. In the Resurgence Test phase, the reinforcer magnitude associated with the Alternative response was either reduced (Experiments 1 & 2a) or increased (Experiment 2b) relative to the preceding condition. Resurgence generally occurred when the reinforcer magnitude maintaining the Alternative response was reduced, but not when it was increased relative to the preceding condition. The results further support the suggestion that resurgence results from an overall "worsening" of reinforcement conditions, but not simply from a change in conditions.

Reforzamiento Concurrente de Secuencias de Respuestas

Decidir si una secuencia de respuestas es una unidad conductual diferente a los elementos que la constituyen y que obedece a las mismas reglas de reforzamiento que respuestas discretas ha representado un problema difícil de resolver. Una forma de mostrar que una secuencia es una unidad de conducta, es observar si dicha secuencia cumple la relación propuesta por la ley de igualación. Con este propósito, se entrenó a cuatro ratas privadas de alimento y experimentalmente ingenuas a apretar en secuencia dos operandos para recibir una gota de leche. Se usaron diferentes programas concurrentes de intervalo variable (VI) para reforzar cada una de cuatro posibles secuencias. Cuando las ratas produjeron una secuencia no reforzada, las luces de la cámara experimental se apagaron durante 2 s. En tres diferentes fases se varió la tasa de reforzamiento para cada secuencia. Los resultados mostraron que las ratas fueron altamente sensibles a las tasas relativas de reforzamiento. Sin embargo, las pendientes de las curvas de ajuste fueron menores a uno, mostrando sub-igualación. Se concluyó que una secuencia de respuestas puede comportarse como un patrón integrado pero no representa una unidad conductual estrictamente equivalente a las operantes discretas.

A multivariate assessment of the rapidly changing procedure with McDowell's Evolutionary Theory of Behavior Dynamics.

A multivariate analysis is concerned with more than one dependent variable simultaneously. Models that generate event records have a privileged status in a multivariate analysis. From a model that generates event records, we may compute predictions for any dependent variable associated with those event records. However, because of the generality that is afforded to us by these kinds of models, we must carefully consider the selection of dependent variables. Thus, we present a conditional compromise heuristic for the selection of dependent variables from a large group of variables. The heuristic is applied to McDowell's Evolutionary Theory of Behavior Dynamics (ETBD) for fitting to a concurrent variable-interval schedule in-transition dataset. From the parameters obtained from fitting ETBD, we generated predictions for a wide range of dependent variables. Overall, we found that our ETBD implementation accounted well for various flavors of the log response ratio, but had difficulty accounting for the overall response rates and cumulative reinforcer effects. Based on these results, we argue that the predictions of our ETBD implementation could be improved by decreasing the base response probabilities, either by increasing the response latencies or by decreasing the sizes of the operant classes.

Modeling bout-pause response patterns in variable-ratio and variable-interval schedules

Modeling bout-pause response patterns in variable-ratio and variable-interval schedules

Effect of Response Effort on Choice Behavior of Pigeons in Reinforcement Schedules Manipulating Distance between Operanda

Previous studies have shown that choice behavior in pigeons is systematically affected by the response effort, such as force or locomotion between response keys, as well as a delay in reinforcement or the amount of reinforcement. The present study aimed to investigate choice behavior in reinforcement schedules in which distance between operanda was manipulated as a response effort from the point of view of generalized matching law. To do so, a distance schedule was developed to manipulate the distance between response keys. In this schedule, pigeons were required to produce either 4 or 10 responses. Interresponse distances, a measure of the distance as a response effort, were calculated by summing the distances between the locations of successive responses. The present study employed a concurrent-chain design in which variable-interval schedules and distance schedules formed the initial and terminal links, respectively, of the concurrent chain. The results showed a matching relation between the initial-links response ratio and the interresponse distance ratio in a condition where 10 responses were required in terminal-links, but not in the condition with 4 responses. This implies that response effort is an important factor in determining choice behavior, as well as other factors including rate or amount of reinforcement or delay in reinforcement. However, the present results could be confounded by the effects of the elapsed time before reinforcement. Thus, further research will be needed using a modified version of the present distance-choice procedure to isolate the effect of distance itself.

Performance on the rapidly changing procedure, according to an associative learner

From a model that generates event records, we may generate predictions for any dependent variable computable from those event records. In the present paper, we examined an implementation of Shimp’s (1978) Associative Learner for two-alternative concurrent variable-interval variable-interval schedules in transition. In general, our Associative Learner implementation had difficulty accounting for the inter-response times as a function of the reinforcer ratio, the general frequency of inter-response times, the acquisition of consecutive reinforcers from the same alternative, and the response rates to the not-just-reinforced alternative after a reinforcer. An extended Associative Learner implementation was designed with provisions for increasing responding to the just-reinforced alternative after a reinforcer and organising bout-like responses. The extended implementation rectified many of the weaknesses of the base implementation.

Modeling bout–pause response patterns in variable-ratio and variable-interval schedules using hierarchical Bayesian methodology

Streams of operant responses are arranged in bouts separated by pauses and differences in performance in reinforcement schedules with identical inter-reinforcement intervals (IRIs) are primarily due to differences in within–bout response rate, not in bout–initiation rate. The present study used hierarchical Bayesian modeling as a new method to quantify the properties of the response bout. A Bernoulli distribution was utilized to express the probability to stay in bout/pause, while a Poisson distribution was utilized to quantify the within–bout response rates. We compared bout/pause patterns between variable–ratio (VR) and variable–interval (VI) schedules across IRIs. The model estimation revealed no difference in within-bout staying probability between schedules. However, response rates of within–bout responses were higher in VR than VI across IRIs. These results are consistent with previous analyses using a log–survivor plot to describe within–bout responses and bouts–initiation responses. In addition, a simulation study was performed to examine how sensitively the model estimate the parameters according to different bout initiation rates. These result showed that the within–bout staying probability was affected by changes in between–bout while within–bout response rate parameters were not. This suggests model estimation robustness of the model estimation to dissociate within–bout and between–bout parameters during different reinforcement schedules.

A new variable interval schedule with constant hazard rate and finite time range.

We propose a new variable interval (VI) schedule that achieves constant probability of reinforcement in time while using a bounded range of intervals. By sampling each trial duration from a uniform distribution ranging from 0 to 2 T seconds, and then applying a reinforcement rule that depends linearly on trial duration, the schedule alternates reinforced and unreinforced trials, each less than 2 T seconds, while preserving a constant hazard function.

Acute drug effects on habitual and non-habitual responding in crossed high alcohol preferring mice.

Drug reward plays a central role in acquiring drug-seeking behavior. However, subjects may continue using drugs despite negative consequences because self-administration becomes habitual, and divorced from outcome values. Although a history of drug and alcohol use expedite habit acquisition, and in spite of the fact that self-administration leads to intoxication, the acute effects of drugs on habitual responding are not well understood. We sought to observe how acute ethanol and amphetamine affect the balance between habitual and goal-directed behavior, as measured by a fluid-reinforced operant conditioning task. Selectively bred crossed high-alcohol-preferring (cHAP) mice were trained on an operant conditioning task reinforced on a variable interval schedule with 1% banana solution, which was subsequently devalued via LiCl pairing in half the animals. Ethanol (1.0g/kg), amphetamine (2.0mg/kg), or saline was administered prior to a post-devaluation test. Overall, mice showed habitual behavior, but when divided into high- or low-responding groups based on training response rates, saline-treated, low-responding animals devalued, while saline-treated high-responding animals did not. Furthermore, amphetamine elicited devaluation even in high-responding animals, while ethanol prevented devaluation even in low-responding animals. These data show that ethanol shifts animals toward behaving habitually. This may illuminate why alcohol-intoxicated individuals display impaired judgment about the relative merits of drinking, and potentially serve as a mechanism by which intoxicated subjects resume previously devalued behaviors, such as comorbid drug use. These findings also show that high variable interval response rates facilitate a shift from goal-directed to habitual behavior.

Punishment and the potential for negative reinforcement with histamine injection.

The present study examined punishment of responding with histamine injection, and its potential to generate avoidance of punishment. Sprague-Dawley rats were trained under concurrent schedules in which responses on one lever (the punishment lever) produced food under a variable-interval schedule, and under some conditions intermittent injections of histamine, which suppressed behavior. Responses on a second (avoidance) lever prevented histamine injections scheduled on the punishment lever. After stabilization of punished responding, a variable-interval 15-s schedule of cancellation of histamine (avoidance) was added for responding on the second/avoidance lever, without subsequent acquisition of responding on that lever. Progressive decreases in the length of the punishment variable-interval schedule increased suppression on the punishment lever without increases in response rates on the avoidance lever. Exchanging contingencies on the levers ensured that response rates on the avoidance lever were sufficiently high to decrease the histamine injection frequency; nonetheless response rates on the avoidance lever decreased over subsequent sessions. Under no condition was responding maintained on the avoidance lever despite continued punishing effectiveness of histamine throughout. The present results suggest that avoidance conditioning is not a necessary condition for effective punishment, and confirm the importance of empirical rather than presumed categorization of behavioral effects of stimulus events.

Rat choice in rapidly changing concurrent schedules.

In two experiments, experimentally naïve rats were trained in concurrent variable-interval schedules in which the reinforcer ratios changed daily according to a pseudorandom binary sequence. In Experiment 1, relative response rates showed clear sensitivity to current-session reinforcer ratios, but not to previous sessions' reinforcer ratios. Within sessions, sensitivity to the current session's reinforcement rates increased steadily, and by session end, response ratios approached matching to the current-session reinforcer ratios. Across sessions, sensitivity to the current session's reinforcer ratio decreased with continued exposure to the pseudorandom binary sequence, contrary to expectations based on previous studies demonstrating learning sets. Using a second group of naïve rats, Experiment 2 replicated the main results from Experiment 1 and showed that although there were increases over sessions in both changeover rate and response rate during the changeover delay, neither could explain the accompanying reductions in sensitivity. We consider the role of reinforcement history, showing that our results can be simulated using two separate representations, one local and one nonlocal, but a more complex approach will be needed to bring together these results and other history effects such as learning sets and spontaneous recovery.

Adaptive learning and forgetting in an unconventional experimental routine.

Forgetting is often thought of as the inability to remember, but remembering and forgetting allow behavior to adapt to a changing environment in distinct and separable ways. Learning and forgetting were assessed concurrently in two pigeon experiments that involved the same unconventional routine where the schedule of reinforcement changed every session. Sessions were run back-to-back with a 23-h mid-session break such that in a single visit to the testing chamber, a pigeon completed the second half of one session and the first half of the next. The beginning of a new session was either signaled or unsignaled. Experiment 1 involved concurrent variable-interval variable-interval schedules with four possible reinforcer ratios. Response allocation was sensitive to the richer schedule and was retained through the mid-session break. Experiment 2 involved peak interval schedules of varying durations. Temporal discrimination was rapidly acquired before and after the mid-session break, but not retained. Signaling the session change decreased control by past contingencies in both experiments, demonstrating that learning and forgetting can be investigated separately. These results suggest that the temporal structure of training, such as multiple short daily sessions instead of one long session, can meaningfully impact measurement of animals' capacity to forget and remember.

Relative effects of reinforcement and punishment on human choice

ABSTRACTAn experiment with humans was conducted to assess the relative effects of reinforcement and punishment on choice. Point gains and losses were programmed as reinforcers and punishers, respectively. Participants used a mouse to click two buttons on a computer screen while the ratio of point gains was varied according to a concurrent variable-interval schedule. Responding was sensitive to relative point gains in baselines. When a schedule of point loss was superimposed on one button, relative to baseline, bias away from the point-loss button was observed, indicating the punishment effects of losses. Finally, a schedule arranging equal rates of both point gains and losses was superimposed on one button. One participant showed no change in bias and two showed shifts in bias away from the alternative with the superimposed schedule relative to baseline. These results suggest that punishment effects are at least equal and sometimes greater than reinforcement effects..

Human instrumental performance in ratio and interval contingencies: A challenge for associative theory.

Associative learning theories regard the probability of reinforcement as the critical factor determining responding. However, the role of this factor in instrumental conditioning is not completely clear. In fact, free-operant experiments show that participants respond at a higher rate on variable ratio than on variable interval schedules even though the reinforcement probability is matched between the schedules. This difference has been attributed to the differential reinforcement of long inter-response times (IRTs) by interval schedules, which acts to slow responding. In the present study, we used a novel experimental design to investigate human responding under random ratio (RR) and regulated probability interval (RPI) schedules, a type of interval schedule that sets a reinforcement probability independently of the IRT duration. Participants responded on each type of schedule before a final choice test in which they distributed responding between two schedules similar to those experienced during training. Although response rates did not differ during training, the participants responded at a lower rate on the RPI schedule than on the matched RR schedule during the choice test. This preference cannot be attributed to a higher probability of reinforcement for long IRTs and questions the idea that similar associative processes underlie classical and instrumental conditioning.

Concurrent variable‐interval variable‐ratio schedules in a dynamic choice environment

Most studies of operant choice have focused on presenting subjects with a fixed pair of schedules across many experimental sessions. Using these methods, studies of concurrent variable- interval variable-ratio schedules helped to evaluate theories of choice. More recently, a growing literature has focused on dynamic choice behavior. Those dynamic choice studies have analyzed behavior on a number of different time scales using concurrent variable-interval schedules. Following the dynamic choice approach, the present experiment examined performance on concurrent variable-interval variable-ratio schedules in a rapidly changing environment. Our objectives were to compare performance on concurrent variable-interval variable-ratio schedules with extant data on concurrent variable-interval variable-interval schedules using a dynamic choice procedure and to extend earlier work on concurrent variable-interval variable-ratio schedules. We analyzed performances at different time scales, finding strong similarities between concurrent variable-interval variable-interval and concurrent variable-interval variable- ratio performance within dynamic choice procedures. Time-based measures revealed almost identical performance in the two procedures compared with response-based measures, supporting the view that choice is best understood as time allocation. Performance at the smaller time scale of visits accorded with the tendency seen in earlier research toward developing a pattern of strong preference for and long visits to the richer alternative paired with brief "samples" at the leaner alternative ("fix and sample").

Baseline response rates affect resistance to change.

The effect of response rates on resistance to change, measured as resistance to extinction, was examined in two experiments. In Experiment 1, responding in transition from a variable-ratio schedule and its yoked-interval counterpart to extinction was compared with pigeons. Following training on a multiple variable-ratio yoked-interval schedule of reinforcement, in which response rates were higher in the former component, reinforcement was removed from both components during a single extended extinction session. Resistance to extinction in the yoked-interval component was always either greater or equal to that in the variable-ratio component. In Experiment 2, resistance to extinction was compared for two groups of rats that exhibited either high or low response rates when maintained on identical variable-interval schedules. Resistance to extinction was greater for the lower-response-rate group. These results suggest that baseline response rate can contribute to resistance to change. Such effects, however, can only be revealed when baseline response rate and reinforcement rate are disentangled (Experiments 1 and 2) from the more usual circumstance where the two covary. Furthermore, they are more cleanly revealed when the programmed contingencies controlling high and low response rates are identical, as in Experiment 2.

The effect of scopolamine on matching behavior and the estimation of relative reward magnitude.

We investigated the behavioral effects of scopolamine on rats that bar pressed for trains of electrically stimulating pulses under concurrent variable interval schedules of reward. For the first half of the session (30 min) a 1:4 ratio in the programmed number of stimulation trains delivered at each option was in effect. At the start of the second half of the session, an unsignaled reversal in the relative train number (4:1) occurred. We tracked the relative magnitude of reward estimated for each contiguous pair of reinforced visits to competing options. Scopolamine hydrobromide led to a reduction in the relative magnitude of reward. A similar result was obtained in a follow-up test in which relative magnitude was manipulated by varying the pulse frequency of stimulation, while equating the train number at each option. The effect of scopolamine hydrobromide could not be attributed to undermatching, side bias, nor to an effect of scopolamine on the reward integration process. When the same rats were treated with scopolamine methylbromide, no effects on matching behavior were observed. Our results suggest a cholinergic basis for the computation of choice variables related to matching behavior. (PsycINFO Database Record

Behavioral intoxication following voluntary oral ingestion of tetramethylenedisulfotetramine: Dose-dependent onset, severity, survival, and recovery

Tetramethylenedisulfotetramine (tetramine, or TETS) is a highly toxic rodenticide that has been responsible for over 14,000 accidental and intentional poisonings worldwide. Although the vast majority of TETS poisonings involved tainted food or drink, the laboratory in vivo studies of TETS intoxication used intraperitoneal injection or gavage for TETS exposure. Seeking to develop and characterize a more realistic model of TETS intoxication in the present study, rats were trained to rapidly and voluntarily consume a poisoned food morsel. Initially, the overt toxic effects of TETS consumption across a large range of doses were characterized, then a focused range of doses was selected for more intensive behavioral evaluation (in operant test chambers providing a variable-interval schedule of food reinforcement). The onset of intoxication following voluntary oral consumption of TETS was rapid, and clear dose-dependent response-rate suppression was observed across multiple performance measures within the operant-chamber environment. At most doses, recovery of operant performance did not occur within 30h. Food consumption and body weight changes were also dose dependent and corroborated the behavioral measures of intoxication. This voluntary oral-poisoning method with concomitant operant-behavioral assessment shows promise for future studies of TETS (and other toxic chemicals of interest) and may be extremely valuable in characterizing treatment outcomes.