Overall Rate Of Reinforcement Research Articles

Factors controlling the micro-structure of human free-operant behaviour: Bout-initiation and within-bout responses are effected by different aspects of the schedule

Two experiments examined factors controlling human free-operant performance in relation to predictions based on the nature of bout-initiation and within-bout responding. Overall, responding was higher for a random ratio (RR) than a random interval (RI) schedule, with equal rates of reinforcement. Bout-initiation rates were not different across the two schedules, but within-bout rates were higher on the RR schedule. Response cost reduced overall rates of responding, but tended to suppress bout-initiation responding more than within-bout responding (Experiments 1 & 2). In contrast, reinforcement magnitude increased all forms of responding (Experiment 2). One explanation consistent with these effects is that bout-initiation responses are controlled by overall rates of reinforcement through their impact on the context (i.e. are stimulus-driven), but that within-bout responses are controlled by response reinforcement (i.e. are goal-directed). These current findings are discussed in the light of these theoretical suggestions.

The partial reinforcement extinction effect: The proportion of trials reinforced during conditioning predicts the number of trials to extinction.

Four experiments compared the extinction of responding to a continuously reinforced (CRf) conditioned stimulus (conditional stimulus [CS]) consistently reinforced on every trial, with extinction of responding to a partially reinforced (PRf) CS that had been inconsistently reinforced. To equate the acquisition of responding between the two CSs, the average duration of the CRf CS was extended so that it scheduled the same overall rate of reinforcement per unit time as the PRf CS. Experiment 1 used a within-subjects design to compare the rates of extinction for a 10-s PRf CS reinforced on 33% of trials versus a 30-s CRf CS. Experiment 2 made the same comparison but using a between-subjects design. Experiment 3 compared extinction in a group trained with a 10-s PRf CS reinforced on 20% of trials and a group trained with a 50-s CRf CS. Experiment 4 compared the rates of extinction following two partial reinforcement schedules: a 10-s PRf CS reinforced on 33% of trial versus a 20-s CRf CS reinforced on 66% of trials. In each experiment, responding took longer to extinguish for the CS that scheduled a lower per-trial probability of reinforcement. Modeling of individual extinction curves using Weibull functions indicated that the latency to initiate extinction was directly related to the per-trial probability of reinforcement learned during acquisition. For example, compared with training with a CRf CS, rats reinforced on 33% of trials took approximately 3 times as many trials to initiate extinction, and rats reinforced on 20% of trials took 5 times as many trials to initiate extinction. These results provide support for trial-based accounts of extinction (e.g., Capaldi & Deutsch, 1967), whereby rats learn about the expected number of trials per reinforcer, and extinction depends on the number of expected reinforcers that have been omitted rather than on the number of extinction trials per se. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Melioration revisited: a systematic replication of Vaughan (1981).

Organisms that behave so as to forfeit a relatively higher overall rate of reinforcement in favor of a relatively lower rate are said to engage in suboptimal choice. Suboptimal choice has been linked with maladaptive behavior in humans. Melioration theory offers one explanatory framework for suboptimal choice. Melioration theory suggests behavior is controlled by differences in local reinforcer rates between alternatives. Vaughan (1981) arranged two experimental conditions in which maximizing the overall rate of reinforcement required behavior that was compatible, or incompatible, with melioration. Vaughan found pigeons allocated more time to a locally richer alternative even when doing so resulted in suboptimal choice. However, Vaughan did not show whether these effects could systematically reverse and did not provide within-session data to show that choice across short time spans remains under the control of differences in local reinforcer rates. The present study used pigeons to replicate and extend Vaughan's findings. We investigated shifts in overall- and within-session choice across repeated conditions, according to arranged local contingencies. Behavior systematically followed changes in local contingencies for most pigeons. Within-session data suggests that, providing differences in local reinforcer rates are discriminated, pigeons will allocate more time to a locally richer alternative, even if this leads to suboptimal choice. These findings facilitate the more confident use of similar procedures that investigate how melioration contributes to suboptimal choice.

Choosing among multiple alternatives: Relative and overall reinforcer rates.

Choice behavior among two alternatives has been widely researched, but fewer studies have examined the effect of multiple (more than two) alternatives on choice. Two experiments investigated whether changing the overall reinforcer rate affected preference among three and four concurrently scheduled alternatives. Experiment 1 trained six pigeons on concurrent schedules with three alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 9:3:1 with the configuration counterbalanced across pigeons. The overall rate of reinforcement was varied across conditions. Preference between the pair of keys arranging the 9:3 reinforcer ratio was less extreme than the pair arranging the 3:1 reinforcer ratio regardless of overall reinforcer rate. This difference was attributable to the richer alternative receiving fewer responses per reinforcer than the other alternatives. Experiment 2 trained pigeons on concurrent schedules with four alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 8:4:2:1, and the overall reinforcer rate was varied. Next, two of the alternatives were put into extinction and the random interval duration was changed from 60 s to 5 s. The ratio of absolute response rates was independent of interval length across all conditions. In both experiments, an analysis of sequences of visits following each reinforcer showed that the pigeons typically made their first response to the richer alternative irrespective of which alternative was just reinforced. Performance on these three- and four-alternative concurrent schedules is not easily extrapolated from corresponding research using two-alternative concurrent schedules.

Persistence during and resurgence following noncontingent reinforcement implemented with and without extinction.

Noncontingent reinforcement (NCR) is typically implemented with extinction (EXT) for destructive behavior reinforced by social consequences and without EXT for destructive behavior reinforced by sensory consequences. Behavioral momentum theory (BMT) predicts that responding will be more persistent, and treatment relapse in the form of response resurgence more likely, when NCR is implemented without EXT due to the greater overall rate of reinforcement associated with this intervention. We used an analogue arrangement to test these predictions of BMT by comparing NCR implemented with and without EXT. For two of three participants, we observed more immediate reductions in responding during NCR without EXT. However, for all participants, NCR without EXT produced greater resurgence than NCR with EXT when we discontinued all reinforcers during an EXT Only phase, although there was variability in response patterns across and within participants. Implications for treatment of destructive behavior using NCR are discussed.

A COMPONENT ANALYSIS OF SCHEDULE THINNING DURING FUNCTIONAL COMMUNICATION TRAINING

One limitation of functional communication training (FCT) is that individuals may request reinforcement via the functional communication response (FCR) at exceedingly high rates. Multiple schedules with alternating periods of reinforcement and extinction of the FCR combined with gradually lengthening the extinction-component interval can effectively address this limitation. However, the extent to which each of these components contributes to the effectiveness of the overall approach remains uncertain. In the current investigation, we evaluated the first component by comparing rates of the FCR and problem behavior under mixed and multiple schedules and evaluated the second component by rapidly switching from dense mixed and multiple schedules to lean multiple schedules without gradually thinning the density of reinforcement. Results indicated that multiple schedules decreased the overall rate of reinforcement for the FCR and maintained the strength of the FCR and low rates of problem behavior without gradually thinning the reinforcement schedule.

Differential reinforcement and resistance to change of divided-attention performance

Behavioral momentum theory provides a framework for understanding how conditions of reinforcement influence instrumental response strength under conditions of disruption (i.e., resistance to change). The present experiment examined resistance to change of divided-attention performance when different overall probabilities of reinforcement were arranged across two components of a multiple schedule. Pigeons responded in a delayed-matching-to-sample procedure with compound samples (color + line orientation) and element comparisons (two colors or two line orientations). Reinforcement ratios of 1:9, 1:1, and 9:1 for accurate matches on the two types of comparison trials were examined across conditions using reinforcement probabilities (color/lines) of .9/.1, .5/.5, and .1/.9 in the rich component and .18/.02, .1/.1, and .02/.18 in the lean component. Relative accuracy with color and line comparisons was an orderly function of relative reinforcement, but this relation did not depend on the overall rate of reinforcement between components. The resistance to change of divided-attention performance was greater for both trial types in the rich component with presession feeding and extinction, but not with decreases in sample duration. These findings suggest promise for the applicability of quantitative models of operant behavior to divided-attention performance, but they highlight the need to further explore conditions impacting the resistance to change of attending.

Effects of reinforcer delay and variability in a successive-encounters procedure.

Pigeons responded in a successive-encounters procedure that consisted of a search period, a choice period, and a handling period. The search period was either a fixed-interval or a mixed-interval schedule presented on the center key of a three-key chamber. Upon completion of the search period, the center key was turned off and the two side keys were lit. A pigeon could either accept a delay followed by food (by pecking the right key) or reject this option and return to the search period (by pecking the left key). During the choice period, a red right key represented the long alternative (a long handling delay followed by food), and a green right key represented the short alternative (a short handling delay followed by food). The experiment consisted of a series of comparisons for which optimal diet theory predicted no changes in preference for the long alternative (because the overall rates of reinforcement were unchanged), whereas the hyperbolic-decay model predicted changes in preference (because the delays to the next possible reinforcer were varied). In all comparisons, the results supported the predictions of the hyperbolic-decay model, which states that the value of a reinforcer is inversely related to the delay between a choice response and reinforcer delivery.

Choice and the Initial Delay to a Reinforcer

Pigeons were trained in two experiments that used the concurrent-chains procedure. These experiments sought to identify the variables controlling the preference of pigeons for a constant duration over a variable duration of exposure to an aperiodic, time-based, terminal-link schedule. The results indicated that two variables correlated with the constant-duration terminal link combined to control preference: (a) a shorter initial delay to a reinforcer; and (b) the probabilistic occurrence of multiple reinforcers. Grace and Nevin (2000) trained pigeons on a concurrent-chains procedure with equal variable-interval (VI) schedules in the initial links and equal VI schedules in the terminal links. The terminal links differed in that one ended after a single reinforcer, which they called “variable-duration” terminal link, whereas the other ended after a fixed period of exposure equal to the average interreinforcement interval (IRI) of the schedule, which they called “constantduration” terminal link. As Grace and Nevin identified, and as discussed at some length below, an important feature of the constant-duration terminal link is that it probabilistically yielded 0, 1, or multiple reinforcers per entry, although it provided the same average rate of reinforcement overall as the variable-duration terminal link. Grace and Nevin (2000) found that three of four pigeons clearly preferred the constant-duration terminal link. In their words, the data of a fourth pigeon “demonstrated a consistent right-key bias” (p. 178), and the present conclusion is that its data are more difficult to interpret. In any case, an important question is what variables caused the preference. Ordinarily, one would have expected the pigeons to be indifferent, since the schedules in effect during the alternatives were identical, and each alternative yielded the same overall rate of reinforcement. Grace and Nevin (2000) initially pondered the role of multiple reinforcers in the constant-duration terminal link, because research has shown that subjects may well prefer a choice alternative associated with multiple reinforcers rather than a single reinforcer per terminal-link entry (e.g., Fantino & Herrnstein, 1968; Mazur, 1986; McDiarmid & Rilling, 1965; Moore, 1979; Poniewaz, 1984; Shull, Mellon, & Sharp, 1990; Shull, Spear, & Bryson, 1981). In particular, Grace and Nevin discussed their findings from the view Correspondence concerning this article may be addressed to Dr. J. Moore, Dept. of Psychology,

Performance on ratio and interval schedules with matched reinforcement rates.

In the first study, rats were trained to pull a chain on a schedule (RPI) that regulates the probability of reinforcement to maintain a constant average reinforcement rate without differentially reinforcing long inter-response times (IRTs). Although the response rate was sensitive to the overall rate of reinforcement, performance was unaffected by variations between 1 and 50 in the IRT memory size used in programming the schedule. In the second study, two groups of animals performed on either a random-interval (RI) schedule or a RPI schedule, with reinforcement rates determined by those generated by a third group performing on a random ratio (RR) 20 schedule. The RI group responded at a lower rate than the RPI group, which, in turn, responded at a lower rate than the RR group, even though the three groups experienced comparable rates of reinforcement. The fact that the RPI group responded at a lower rate than the RR group suggests that the standard response rate difference observed between ratio and interval schedules, which have been matched for reinforcement rate, cannot be attributed solely to the fact that conventional interval schedules differentially reinforce long IRTs.

Participants’ Sensitivity to Percentage Payback and Credit Value When Playing a Slot-Machine Simulation

Legalized gambling has become both a major industry and concern in the United States, but little research from the behavior-analytic perspective has been done on the topic. The present study consisted of two experiments that had participants play a computer-simulated slot machine. The variables manipulated were the percentage payback rate (i.e., overall rate of reinforcement) and the amount of money the credits being wagered were worth (i.e., reinforcer magnitude). Experiment 1 investigated these variables using a between-groups design. Experiment 2 investigated them using a within-subjects design. Results from both experiments demonstrated that participants' gambling behavior did not vary as a function of payback percentage. Their behavior was, however, sensitive to credit value; overall, participants bet less when the credits were worth more. These findings have potential implications for why some people display problem gambling. They will also hopefully promote research on a topic that has been largely ignored by the field of behavior analysis.

Does the terminal-link effect depend on duration or reinforcement rate?

Preference in concurrent chains for the richer terminal-link schedule becomes more extreme as the schedule values increase with their ratio held constant, a result known as the terminal-link effect. We report two experiments that attempt to determine whether this effect is related to terminal-link duration or the overall rate of reinforcement. These variables have been confounded in prior studies, but can be separated by comparing variable-duration schedules that end after a single reinforcer has been earned, with constant-duration schedules during which a variable number of reinforcers may be earned. In Experiment 1, the terminal-link effect was obtained with variable-duration schedules when duration and overall reinforcement rate were manipulated, but not with constant-duration schedules when overall reinforcement rate was changed with duration held constant. In Experiment 2, the terminal-link effect was obtained with constant-duration schedules when duration was manipulated with overall reinforcement rate held constant. Taken together, these results show that the terminal-link effect depends on changes in terminal-link duration, not overall reinforcement rate (or equivalently, average time to reinforcement). This accords with the account of the terminal-link effect provided by the contextual choice model [J. Exp. Anal. Behav. 61 (1994) 113] but not delay-reduction theory [J. Exp. Anal. Behav. 12 (1969) 723].

Concurrent schedules: reinforcer magnitude effects.

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key procedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each condition remained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed. At a molar level of analysis, preference was described well by a version of the generalized matching law, consistent with previous reports. More local analyses showed that recently obtained reinforcers had small measurable effects on current preference, with the most recently obtained reinforcer having a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses, and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choice have both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitude is exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcer magnitude is varied are likely to be due to equal frequencies of different sized preference pulses, whereas higher sensitivities when reinforcer rates are varied might result from changes in the frequencies of different sized preference pulses.

Dynamic changes in the size of behavioral contrast.

McSweeney and Weatherly (1998) suggested that multiple-schedule behavioral contrast may be explained, at least partly, by differences in the amount of habituation to the reinforcer produced during the baseline and contrast phases of a contrast experiment. The present experiment studied behavioral contrast when pigeons pecked keys for food reinforcers delivered on multiple VI VI schedules. The results confirmed two predictions made by McSweeney and Weatherly. Both positive and negative contrast were larger for longer (i.e. 80 min) than for shorter sessions (i.e. 20 min). Positive and negative contrast were also larger later, rather than earlier, in the session. These results support the conclusion that factors that weaken the effectiveness of the reinforcer with its repeated delivery (e.g., habituation) may be among the factors contributing to multiple-schedule behavioral contrast. Key words: behavioral contrast, dynamic changes, session length, habituation Behavioral contrast refers to an inverse relationship between the rate of reinforcement in one component of a multiple schedule and the rate of responding in the other component. Positive contrast occurs when decreasing the rate of reinforcement in one component is associated with increases in the rate of responding in the other component. Negative contrast occurs when increasing the rate of reinforcement in one component is associated with decreases in the rate of responding in the other component. ********** Understanding behavioral contrast is important for several reasons. Contrast is important theoretically because it shows that the effects of reinforcers are relative rather than absolute. For this reason, it plays a pivotal role in several operant theories (e.g. Herrnstein, 1970). Contrast is important practically because it can occur in applied settings (e.g., Gross & Drabman, 1981). In some cases, behavioral contrast may work against the goals of therapy. Koegal, Egel and Williams (1980) found that reinforcing a target behavior by autistic children in a therapy setting was accompanied by decreases in this behavior in another, non-therapy setting (e.g., home, a classroom). If the goal of therapy is to change the frequency of a target behavior across various contexts, such instances of contrast interfere with this aim. In other cases, contrast may facilitate the goals of therapy. Charlop, Kurtz and Milstein (1992) showed that putting maintenance tasks (i.e. tasks that their subjects had been performing at approximately 80% accuracy for 3 months prior to the study) on extinction was associated with increases in the frequency of a designated acquisition task (i.e. a task the child had never been asked to do prior to the experiment). All six autistic children reached performance criteria for the acquisition task using this procedure. Although behavioral contrast has been extensively studied, the theoretical factors that contribute(s) to this phenomenon remain the topic of debate (see Williams, 1983, 2002, for a review). Most theories of contrast argue that contrast arises from a comparison of the rates of reinforcement in the two components of the multiple schedule when those components provide different rates of reinforcement. For example, the additive theories (e.g., Gamzu & Schwartz, 1973; Hearst & Jenkins, 1974; Rachlin, 1973) argue that contrast occurs when subjects make transitions between components that provide higher or lower valued reinforcers. Response interference theories (e.g., Hinson & Staddon, 1978) argue that the difference in the rates of reinforcement between the components leads to the reallocation of responding across those components. McSweeney and Weatherly (1998) attribute behavioral contrast to a different comparison (e.g., different overall rates of reinforcement received during different phases of the experiment) Specifically, they suggest that behavioral contrast occurs, at least partly, because the procedure commonly used to study contrast (e. …

Concurrent schedules: short- and long-term effects of reinforcers.

Five pigeons were trained on concurrent variable-interval schedules in a switching-key procedure. The overall rate of reinforcement was constant in all conditions, and the ratios of reinforcers obtainable on the two alternatives were varied over seven levels. Each condition remained in effect for 65 sessions, and the last 50 sessions of data from each condition were analyzed. The most recently obtained reinforcer had the largest effect on current preference, but each of the eight previously obtained reinforcers had a small measurable effect. These effects were larger when the reinforcer ratio was more extreme. A longer term effect of reinforcement was also evident, which changed as a function of the reinforcer ratio arranged. More local analyses showed regularities at a reinforcer-by-reinforcer level and large transient movements in preference toward the just-reinforced alternative immediately following reinforcers, followed by a return to stable levels that were related to the reinforcer ratio in effect. The present data suggest that the variables that control choice have both short- and long-term effects and that the short-term effects increased when the reinforcer ratios arranged were more extreme.

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration.

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.

Temporal context and conditioned reinforcement value.

The effectiveness of a stimulus as a conditioned reinforcer depends on the temporal context of reinforcement, that is, the overall rate of reinforcement in the situation. The dominant view has been that context determines the learned value of a stimulus directly, according to delay-reduction theory. By contrast, the contextual choice model (CCM) maintains that value is independent of context and incorporates the effects of context on choice in the framework of the matching law. The authors report 2 experiments with pigeons as subjects that use transfer tests to assess the value of stimuli in the concurrent-chains procedure. Results strongly support the assumption of CCM that pigeons learn the temporal relations between events independently of context but that context modulates the expression of that learning as choice.

WITHIN‐SESSION RESPONSE PATTERNS ON CONJOINT VARIABLE‐INTERVAL VARIABLE‐TIME SCHEDULES

Operant responding often changes within sessions, even when factors such as rate of reinforcement remain constant. The present study was designed to determine whether within‐session response patterns are determined by the total number of reinforcers delivered during the session or only by the reinforcers earned by the operant response. Four rats pressed a lever and 3 pigeons pecked a key for food reinforcers delivered by a conjoint variable‐interval variable‐time schedule. The overall rate of reinforcement of the conjoint schedule varied across conditions from 15 to 480 reinforcers per hour. When fewer than 120 reinforcers were delivered per hour, the within‐session patterns of responding on conjoint schedules were similar to those previously observed when subjects received the same total number of reinforcers by responding on simple variable‐interval schedules. Response patterns were less similar to those observed on simple variable‐interval schedules when the overall rate of reinforcement exceeded 120 reinforcers per hour. These results suggest that response‐independent reinforcers can affect the within‐session pattern of responding on a response‐dependent schedule. The results are incompatible with a response‐based explanation of within‐session changes in responding (e.g., fatigue), but are consistent with both reinforcer‐based (e.g., satiation) and stimulus‐based (e.g., habituation) explanations.

HUMAN SIGNAL‐DETECTION PERFORMANCE: EFFECTS OF SIGNAL PRESENTATION PROBABILITIES AND REINFORCER DISTRIBUTIONS

University students participated in one of four standard two‐choice signal‐detection experiments in which signal presentation probability was varied and the reinforcement distribution was held constant and equal. In Experiments 1, 3 and 4, subjects' performance showed a systematic response bias for reporting the stimulus presented least often. Experiments 1 and 4 showed that this effect was reliable with extended training and monetary, rather than point, reinforcement. In Experiment 2, all correct responses were signaled in some way, and this produced the opposite relationship between signal presentation probability and response bias. Experiments 1 and 3 found that explicitly deducting money (intended as punishment) for equal numbers of incorrect responses on each alternative, or varying the obtained overall rate of reinforcement, produced no clear change in response bias. The bias, shown by humans, for reporting the stimulus presented least often remains a challenge for theories of stimulus detection.

BIASING THE PACEMAKER IN THE BEHAVIORAL THEORY OF TIMING

In the behavioral theory of timing, pacemaker rate is determined by overall rate of reinforcement. A two-alternative free-operant psychophysical procedure was employed to investigate whether pacemaker period was also sensitive to the differential rate of reinforcement. Responding on a left key during the first 25 s and on a right key during the second 25 s of a 50-s trial was reinforced at variable intervals, and variable-interval schedule values during the two halves of the trials were varied systematically. Responding on the right key during the first 25 s and on the left key during the second 25 s was not reinforced. Estimates of pacemaker period were derived from fits of a function predicted by the behavioral theory of timing to right-key response proportions in consecutive 5-s bins of the 50-s trial. Estimates of pacemaker period were shortest when the differential reinforcer rate most strongly favored right-key responses, and were longest when the differential reinforcer rate most strongly favored left-key responses. The results were consistent with the conclusion that pacemaker rate is influenced by relative reinforcer rate.