Two-component Multiple Schedule Research Articles

Toward a Procedure to Study Rule-Governed Choice: Preliminary Data

AbstractThe aim of this study was to model a situation that induced choice between following two incompatible rules, each associated with a different rate of reinforcement. In Experiment 1, eight undergraduate students were exposed to a two-component multiple schedule (training). In each component, there was a concurrent variable interval (VI)–extinction (EXT) schedule. Participants were given two rules that instructed them to respond to the VI alternative in the presence of different discriminative stimuli. The side of the VI schedule changed in each component and offered a different reinforcer rate according to the discriminative stimuli in the operation. When both discriminative stimuli were concurrently presented (test), participants favored the alternative previously instructed by the rule, which was associated with the greatest reinforcer rate, whereas indifference was observed in the absence of discriminative stimuli. Experiment 2 tested the effects of reinforcement rate using the same procedure without providing rules. During training, participants gradually developed a preference for the VI alternatives. In the choice test phase, participants favored the alternative associated with the stimuli with the highest reinforcer rate when both discriminative stimuli were present. Unsystematic preference was observed in the absence of discriminative stimuli. Two alternative explanations were provided for the findings.

Stimulus control of a social operant.

Three pigeon dyads were exposed to a two-component multiple schedule comprised of two tandem variable-interval 30-s interresponse time (IRT) > 3-s schedules in the presence of different stimuli. Pecks to keys by both pigeons of a dyad occurring within 500 ms of one another were required for reinforcement under one tandem schedule (the coordination component), and such coordinated responses were not required under the other (the control component). The terminal link of each schedule ensured that the reinforced coordination response was an IRT > 3 s. Rates of coordinated IRTs > 3 s and total rates of coordinated responses (composed of IRTs > 3 s and IRTs ≤ 3 s) were higher in the coordination components than in either of two different control components in which coordination was not required for reinforcement. This difference in coordinated responses in the presence and absence of the coordination requirement under stimulus control transitorily deteriorated and then was reestablished when the relation between the stimulus and the coordination contingency or its absence was reversed. The results show coordinated responding to function as a discriminated social operant.

Rich-lean transitions produced by stimuli associated with accurate and inaccurate responding by pigeons.

Discriminable transitions from relatively favorable schedules of reinforcement to unfavorable schedules (rich-lean transitions) can produce disruptions in operant behavior. A prior evaluation in our laboratory (Toegel et al., 2021) found that placing a border around a key displayed on a resistive touchscreen increased pigeons' response accuracy relative to conditions without the border. We sought to evaluate (a) whether effects of the key border on accuracy could be replicated in a within-session comparison and (b) whether transitions from a response alternative associated with accurate responding to one associated with inaccurate responding functioned as rich-lean transitions. Pigeons' key pecks were reinforced according to a two-component multiple schedule with identical fixed-ratio (FR) requirements and reinforcer magnitudes. The components differed based on whether the virtual key was displayed behind a border or with no border. In line with prior research, within-session comparisons yielded higher response accuracy in components with the key border than in components without the border. Furthermore, transitions from the border component to the no border component functioned as rich-lean transitions for pigeons whose obtained FRs in no border components were substantially larger than the FR programmed in that component.

Nondrug reinforcers contingent on alternative behavior or abstinence increase resistance to extinction and reinstatement of ethanol-maintained behavior.

The effects of delivering nondrug alternative reinforcement on resistance to extinction and reinstatement of rats' ethanol-maintained lever pressing were evaluated in two experiments. In both, rats self-administered ethanol by lever pressing in a two-component multiple schedule during baseline. In the Rich component, alternative food reinforcement was made available for performing an alternative response (Experiment 1) or according to a differential-reinforcement-of-other-behavior schedule for lever pressing (Experiment 2). In the Lean component, only ethanol was available. Moreover, the frequency of alternative reinforcement was manipulated across conditions in Experiment 1. Following baseline, lever pressing was extinguished in both components by suspending ethanol reinforcement, and alternative food reinforcers were discontinued. Finally, to test for reinstatement, ethanol reinforcers were delivered independently of lever pressing in both components. In both experiments, proportion-of-baseline response rates were higher during extinction and reinstatement testing in the Rich component than in the Lean component (although differentiation was not observed at the lowest frequency of alternative reinforcement in Experiment 1). Thus, alternative nondrug reinforcers increased resistance to extinction and reinstatement of rats' ethanol-maintained lever pressing, even when those reinforcers were delivered contingently on an alternative response or on abstinence from lever pressing.

Examination of alternative-response discrimination training and resurgence in rats.

Resurgence is an increase in a previously reinforced behavior following a worsening of conditions for a more recently reinforced behavior. Discrimination training is incorporated into treatment for problem behavior to prevent treatment adherence failures that may result in resurgence. There is evidence that resurgence may be reduced when a stimulus that signals alternative-response extinction is present compared with absent; however, the generality of this effect is unknown given the limited testing conditions. The goal of the present experiments was to further examine the effects of such stimuli in a reverse-translational evaluation using rats. Target responding was reinforced in baseline and then placed on extinction in the following discrimination-training phase. An alternative response was differentially reinforced in a two-component multiple schedule where one stimulus (i.e., SD) signaled alternative-response reinforcement and the other (i.e., SΔ) signaled extinction. Experiment 1 assessed resurgence in both the SΔ and SD when alternative reinforcement was removed. Experiment 2 evaluated resurgence under conditions that better approximated those used in the clinic in which the alternative-response SΔ was present or absent. The SΔ failed to suppress target responding during resurgence testing in both experiments. These findings suggest that the conditions under which an alternative-response SΔ will successfully mitigate resurgence may be limited and require further research.

Behavioral momentum and resistance to extinction across repeated extinction tests.

The present experiments assessed whether resistance to extinction of pigeons' key pecking decreased across repeated extinction tests. An additional impetus for this research was to determine how the quantitative framework provided by behavioral momentum theory might be used to describe any such changes across tests. Pigeons pecked keys in two-component multiple schedules (one component associated with a higher reinforcer rate and the other with a lower rate) in which baseline and extinction conditions alternated. In Experiment 1, baseline and extinction conditions alternated every session, and, in Experiment 2, these conditions lasted for 10 and 7 sessions, respectively. Resistance to extinction decreased across successive extinction conditions in both experiments. Fits of the behavioral-momentum based model of extinction to the data returned uncertain results in Experiment 1 but implicated both generalization decrement and response-reinforcer contingency termination as the possible mechanisms responsible for behavior change in Experiment 2. Thus, these data suggest that experimental manipulations that affect discrimination of changes in reinforcement contingencies may influence resistance to extinction by modulating the disruptive impacts of removing reinforcers from the experimental context and of suspending response-reinforcer contingencies.

Effects of the Type of Reinforcer on Renewal of Operant Responding

Some studies have identified that ABA renewal seems to depend on how response-reinforcer contingency is established. Using rats as subjects, the present study assessed ABA and ABB renewal using a two-component multiple schedule (VI30 s - VI30 s) each with a different reinforcer (pellets or sucrose). 16 subjects were trained to lever-press during 20 sessions in Context A; lever-pressing was extinguished during 10 sessions in Context B. And for the renewal test, 8 subjects were tested in Context A (Group ABA); whereas, the rest were tested in Context B (Group ABB). During acquisition, response rates were higher on the pellets component than the sucrose component; during extinction, response rates decreased to near-zero responses. A renewal effect was observed only for Group ABA during test, showing no differences between components. Our results suggest that different type of reinforcers do not seem to affect ABA renewal, using different contexts allows for renewal to be observed regardless of the differences in response rates during acquisition.

Response-reinforcer dependency and resistance to change.

The effects of the response-reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two-component multiple schedule. Across conditions, in the fixed component, all reinforcers were response-dependent; in the alternative component, the percentage of response-dependent reinforcers was 100, 50 (i.e., 50% response-dependent and 50% response-independent) or 10% (i.e., 10% response-dependent and 90% response-independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three-component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response-dependent or independent reinforcers. In the other component, response-dependent and -independent reinforcers were programmed by superimposing a variable-time schedule on an independent variable-interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple-schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response-reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.

Omission Training Results in More Resurgence than Alternative Reinforcement

Resurgence refers to the reemergence of a previously reinforced response following the extinction of a more recently reinforced response. In a published study, resurgence occurred to a lesser extent following differential reinforcement of alternative behavior (DRA) than differential reinforcement of other behavior (DRO) with pigeons, but this effect has not been replicated with humans. We conducted a within-subject comparison of resurgence following DRA and DRO, using a human-operant preparation. Six college students earned points by clicking a mouse button across two-component multiple schedules. During both baseline components, points were delivered for the first click to a black circle after 2 s. Responding on the black circle decreased during the second phase, using a DRA in one component and a DRO in the other component. We tested for resurgence by terminating point deliveries (extinction) during both components in the third phase. For three of the participants, more resurgence occurred in the component previously associated with DRO than the component previously associated with DRA. The other three participants showed more resurgence in the first component experienced during extinction, regardless of whether that component was associated with DRA or DRO. However, resurgence was exacerbated when the first component during extinction was DRO rather than DRA. Although sequence influenced resurgence, DRA may be preferable to DRO as an intervention when resurgence is a concern.

Effect of a single free food presentation on extinction responding in a multiple schedule

The present study was designed to assess the effect of a single, response-independent food presentation on responding during extinction. Using a two-component multiple schedule, we examined differences in pigeons’ extinction responding resulting from a single response-independent food presentation occurring at the beginning of the experimental session (30-s prior to the beginning of the first component). One component presented reinforcement according to a variable interval 45-s schedule and the second presented reinforcement according to a variable interval 180-s schedule. After establishing stable baseline responding we extinguished responding. We systematically manipulated the presence or absence of a single 3-s free food presentation using the food hopper that occurred 30-s prior to the presentation of the first component. We found the single free food presentation increased persistence of responding in extinction. This finding is inconsistent with behavioral momentum theory inasmuch as it assigns a response disruptive role to food presentations occurring outside of the context of the target operant.

Early social isolation increases persistence of alcohol-seeking behavior in alcohol-related contexts.

Social conditions during rearing are well known to affect adult alcohol consumption, but few experiments have explored the effects of social conditions on behaviors that are related to alcohol dependence, such as the persistence of alcohol seeking. This study compared the effects of isolation (ISO) and interaction (INT) rearing on the persistence of alcohol-seeking behavior. Rats were trained to lever press for a solution of 10% alcohol diluted in water. They were then exposed to a two-component multiple schedule of reinforcement (baseline). Responses in one component were reinforced by a higher rate of alcohol delivery (rich component, variable interval 15 s) and responses in the other component were reinforced by a lower rate of delivery (lean component, variable interval 45 s). The persistence of lever pressing in the presence of each stimulus was then assessed during extinction. The results from baseline showed that response rates in rats in both groups were higher in the rich component than in the lean component, but ISO rats responded significantly more than INT rats in both components. The persistence of responding during extinction in ISO rats in both components was also higher than that in INT rats. The results show that effects of ISO are not restricted to alcohol consumption, but also affect persistence of alcohol-seeking behavior, which may reflect differences in the value of drug-related stimuli.

A FURTHER LOOK AT REINFORCEMENT RATE AND RESURGENCE

Resurgence is the recurrence of a previously reinforced and then extinguished response when a more recently reinforced response is extinguished. The purpose of the present experiments was to examine the relation between alternative reinforcement rates and resurgence with pigeons. Each experiment consisted of a three-phase procedure and each phase of three experiments was a two-component multiple schedule, except for Experiments 2a and 2b, which employed single schedules. In each experiment, the target response was reinforced according to a variable–interval (VI) schedule in the Acquisition phase. In the Elimination phase, the target response was eliminated using either extinction or a differential–reinforcement–of–other–behavior (DRO) schedule and the alternative response was reinforced according to a VI schedule. The rate of alternative reinforcement differed between components, but the number of reinforcers per hour (and thus the reinforcement ratio) was 60 and 180 in the Lean and Rich components, respectively. In the Resurgence phase, all reinforcers were withheld and resurgence was compared between components. Across all experiments, results were inconsistent. In some instances there was greater resurgence in the Rich component, but the opposite was observed in others. These results do not offer evidence supporting the prediction of behavioral momentum and related models of resurgence.

Effect of sucrose availability on wheel-running as an operant and as a reinforcing consequence on a multiple schedule: Additive effects of extrinsic and automatic reinforcement

As a follow up to Belke and Pierce’s (2014) study, we assessed the effects of repeated presentation and removal of sucrose solution on the behavior of rats responding on a two-component multiple schedule. Rats completed 15 wheel turns (FR 15) for either 15% or 0% sucrose solution in the manipulated component and lever pressed 10 times on average (VR 10) for an opportunity to complete 15 wheel turns (FR 15) in the other component. In contrast to our earlier study, the components advanced based on time (every 8min) rather than completed responses. Results showed that in the manipulated component wheel-running rates were higher and the latency to initiate running longer when sucrose was present (15%) compared to absent (0% or water); the number of obtained outcomes (sucrose/water), however, did not differ with the presentation and withdrawal of sucrose. For the wheel-running as reinforcement component, rates of wheel turns, overall lever-pressing rates, and obtained wheel-running reinforcements were higher, and postreinforcement pauses shorter, when sucrose was present (15%) than absent (0%) in manipulated component. Overall, our findings suggest that wheel-running rate regardless of its function (operant or reinforcement) is maintained by automatically generated consequences (automatic reinforcement) and is increased as an operant by adding experimentally arranged sucrose reinforcement (extrinsic reinforcement). This additive effect on operant wheel-running generalizes through induction or arousal to the wheel-running as reinforcement component, increasing the rate of responding for opportunities to run and the rate of wheel-running per opportunity.

The several roles of stimuli in token reinforcement

Three experiments were conducted with pigeons to identify the stimulus functions of tokens in second-order token-reinforcement schedules. All experiments employed two-component multiple schedules with a token-reinforcement schedule in one component and a schedule with equivalent response requirements and/or reinforcer density in the other. In Experiment 1, response rates were lower under a token-reinforcement schedule than under a tandem schedule with the same response requirements, suggesting a discriminative role for the tokens. In Experiment 2, response rates varied systematically with signaling functions of the tokens in a series of conditions designed to explore other aspects of the temporal-correlative relations between tokens and food. In Experiment 3, response rates were reduced but not eliminated by presenting tokens independent of responding, yoked to their temporal occurrence in a preceding token component, suggesting both a reinforcing function and eliciting/evocative functions based on stimulus-food relations. Only when tokens were removed entirely was responding eliminated. On the whole, the results suggest that tokens, as stimuli temporally correlated with food, may serve multiple stimulus functions in token-reinforcement procedures--reinforcing, discriminative, or eliciting--depending on the precise arrangement of the contingencies in which they are embedded.

Inter-response-time reinforcement and relative reinforcer frequency control choice.

In Conditions 1 and 3 of our Experiment 1, rats pressed levers for food in a two-component multiple schedule. The first component was concurrent variable-ratio (VR) 20 variable-interval (VI) 90 s, and the second was concurrent yoked VI (its reinforcement rate equaled that of the prior component's VR) VI 90 s. In Condition 2, the VR was changed to tandem VR 20, differential reinforcement of low rates (DRL) 0.8 s. Local response rates were higher in the VR than in the yoked VI schedule, and this difference disappeared between tandem VR DRL and yoked VI. The relative time allocations to VR and yoked VI, as well as to tandem VR DRL and yoked VI, were approximately the same across conditions. In Experiment 2, rats chose in a single session between five different VI pairs, each lasting for 12 reinforcer presentations (variable-environment procedure). The across-schedule hourly reinforcement rates were 120 and 40, respectively, in Conditions 1-3 and 4-6. During Conditions 2 and 5, one lever's VI was changed to tandem VI, DRL 2 s. High covariation between relative time allocations and relative reinforcer frequencies, as well as invariance in local response rates to the schedules, was evident in all conditions. In addition, the relative local response rates were biased toward the unchanged VI in Conditions 2 and 5. These results demonstrate two-process control of choice: Inter-response-time reinforcement controls the local response rate, and relative reinforcer frequency controls relative time allocations.

Effects of baseline reinforcement rate on operant ABA and ABC renewal

Renewal is a relapse phenomenon that occurs when the contextual stimuli present during extinction change, and consequently, an extinguished response increases in rate. Two experiments assessed extinction and renewal of key-pecking in pigeons in a two-component multiple schedule wherein baseline reinforcer rates were delivered at relatively rich or lean rates. In Experiment 1, an ABA design was used in which baseline stimuli were steady key lights (Context A). Food was then removed during extinction, and simultaneously, the context was changed by flashing the key lights (Context B). Following extinction, steady key lights were reintroduced, but food remained unavailable. Key-pecking was more resistant to extinction and recovered to a greater degree in the rich relative to the lean component. In Experiment 2, we introduced novel stimuli following extinction (ABC renewal) rather than reintroducing baseline stimuli. Similar to Experiment 1, in Experiment 2 resistance to change and renewal remained greater in the component associated with higher reinforcer rates during baseline. These findings provide additional support for the context-specificity of operant extinction, and support the prediction of behavioral momentum theory that stimuli associated with higher rates of reinforcement engender greater persistence and relapse than those associated with lower rates of reinforcement.

Effect of sucrose availability and pre-running on the intrinsic value of wheel running as an operant and a reinforcing consequence

The current study investigated the effect of motivational manipulations on operant wheel running for sucrose reinforcement and on wheel running as a behavioral consequence for lever pressing, within the same experimental context. Specifically, rats responded on a two-component multiple schedule of reinforcement in which lever pressing produced the opportunity to run in a wheel in one component of the schedule (reinforcer component) and wheel running produced the opportunity to consume sucrose solution in the other component (operant component). Motivational manipulations involved removal of sucrose contingent on wheel running and providing 1h of pre-session wheel running. Results showed that, in opposition to a response strengthening view, sucrose did not maintain operant wheel running. The motivational operations of withdrawing sucrose or providing pre-session wheel running, however, resulted in different wheel-running rates in the operant and reinforcer components of the multiple schedule; this rate discrepancy revealed the extrinsic reinforcing effects of sucrose on operant wheel running, but also indicated the intrinsic reinforcement value of wheel running across components. Differences in wheel-running rates between components were discussed in terms of arousal, undermining of intrinsic motivation, and behavioral contrast.

Effects of reinforcer magnitude on reinforced behavioral variability

Eight pigeons were exposed to a two-component multiple schedule. In each component, four-peck sequences across left and right keys were reinforced according to a variability threshold contingency. In one condition, only infrequently occurring response sequences were reinforced in each component, thereby generating highly variable sequences. In a separate condition, when the variability threshold contingency was lenient in each component, sequences were much less variable. In each condition, reinforcer magnitude was manipulated across components, and the larger reinforcer magnitude produced less variability than the smaller reinforcer magnitude. These results suggest that larger reinforcers hinder the reinforcement of behavioral variability. The results are interpretable in terms of the larger reinforcer inducing a greater level of behavioral repetition, particularly as the time to reinforcement was approached. This effect may have implications for reinforcing behavioral variability in humans.

Effects of changeover delay on response allocation during probe tests

The present study assessed whether a pattern of responding that develops when choosing between two alternatives generalizes to novel choice tests when alternatives are presented in new combinations. Pigeons were trained on a two-component multiple schedule. In both components, a concurrent variable-interval (VI) 40-s VI 80-s schedule was used. The COD was 1 s in one component and 10 s in the other. The long COD produced consistently longer dwell times than the short COD did. Following training, subjects were presented with four types of probe-test components in which one alternative was drawn from the component with the short COD and one alternative was drawn from the component with the long COD. When the schedule values of the two alternatives were identical (VI 40 vs. VI 40 and VI 80 vs. VI 80), subjects preferred the alternative trained with the long COD (Ms = .78 and .61, respectively). Additionally, subjects preferred the VI 40-s alternative trained with the long COD to the VI 80-s alternative that was trained with the short COD (M = .85). Systematic preference was not observed when subjects were given a choice between the VI 40-s alternative that was trained with the short COD and the VI 80-s alternative that was trained with the long COD. These results demonstrate that a stimulus associated with a longer COD, and thus longer dwell times in baseline training, may be more preferred during probe tests than expected on the basis of the rate of primary reinforcement associated with that stimulus.

Briefly delayed reinforcement effects on variable‐ratio and yoked‐interval schedule performance

Most investigations of briefly delayed reinforcement have involved schedules that arrange a time-plus-response requirement. The present experiment examined whether briefly delaying reinforcement on schedules that have a ratio requirement differs from results with schedules that have a time-plus-response requirement. Four pigeons responded on a two-component multiple schedule. One component arranged a variable-ratio (VR) 50 and the other a variable-interval (VI) schedule in which the distribution of reinforcers was yoked to the preceding VR schedule. Across a series of conditions, delays were imposed in both schedules. These delays were brief (0.25- or 0.5-s) unsignaled delays and, as control conditions, a 5-s unsignaled delay and a 0.5-s delay signaled by a blackout of the chamber. In the yoked-VI component, the brief unsignaled delay increased response rates in six of nine opportunities and increased the proportion of short interresponse times (IRTs) (<0.4 s) in eight of nine opportunities. In the VR component, the brief unsignaled delay increased response rates and the proportion of short IRTs in only two of nine opportunities. For two of the three pigeons that were exposed to the 5-s unsignaled delay, response rates and the proportion of short IRTs decreased in both of the components. The 0.5-s signaled delay did not systematically change response rates nor did it change the distribution of short IRTs relative to the immediate reinforcement condition. The results replicate effects reported with time-based schedules and extend these observations by showing that changes commonly observed in VI performance with briefly delayed reinforcement are not characteristic of VR responding.