Variable-ratio Schedules Research Articles

The effects of reinforcer choice on rates of challenging behavior and free operant responding in individuals with severe disabilities

A multielement design was used to evaluate the eAects of three reinforcement conditions on the free-operant and challenging responses of two individuals with severe disabilities. Preference assessments identified three stimuli that could function as reinforcers for each participant. Reinforcers were delivered on a variable ratio schedule for a free-operant response. In the participant-selected condition, each reinforcer delivery consisted of placing an array of three diAerent stimuli in front of the participant, who was allowed to select one. In the constant condition, each reinforcer delivery consisted of placing an array of three identical stimuli in front of the participant. Within each session the three stimuli were held constant, but varied across sessions. In the experimenter-selected condition, each reinforcer delivery consisted of an array of three identical stimuli being placed in front of the participant. Within each session, the three stimuli were presented in an order predetermined by the experimenter such that, on average, each stimulus was available every third reinforcer delivery. For both individuals, the participantselected condition resulted in lower average rates of challenging responses and slightly higher average free-operant response rates. No diAerences in responding were noted between the constant condition and the experimenter-selected condition. #1998 John Wiley & Sons, Ltd. Individuals with severe disabilities frequently require dense reinforcement for both skill acquisition and the reduction of challenging behaviors. Finding and maintaining eAective reinforcers are essential for eAective intervention. Most reinforcement procedures used in applied settings are derived from basic animal research in which establishing operations, such as deprivation, were manipulated (Michael, 1982). Although there is some evidence that manipulating establishing operations can aAect responding in individuals with developmental disabilities

Cautiousness, Stereotypy, and Variability in Older and Younger Adults

Older adults and college students were tested with two procedures that measure stereotypy and response variability In Experiment 1 subjects guided a marker through a 6 by 6 matrix by pressing two computer keys. Points were awarded on either a continuous or variable ratio 3 schedule of reinforcement. Points were exchanged for money Continuous reinforcement produced significantly higher stereotypy than intermittent reinforcement did in both age groups. A difference was found in stereotypy between age groups during the variable ratio schedule. This difference may have been caused by the greater task anxiety of the older adults. In Experiment 2 subjects were tested using the matrix procedure with the consequence that response variability was differentially reinforced. Four levels of variability were tested in the two age groups. Both older and younger subjects learned to vary response patterns to obtain points. The results are discussed in the context of cautiousness as an age-related explanation for behavior.

Reinforcer frequency and restricted stimulus control.

Stimulus control was evaluated in 3 individuals with moderate to severe mental retardation by delayed identity matching-to-sample procedures that presented either one or two discrete forms as sample stimuli on each trial. On pretests, accuracy scores on one-sample trials were uniformly high. On two-sample trials, the correct stimulus (i.e., the one that subsequently appeared in the comparison array) varied unpredictably, and accuracy scores were substantially lower, suggesting that both sample stimuli did not exert stimulus control on every trial. Subjects were then given training sessions with the one-sample task and with a new set of four stimuli. For two of the stimuli, correct matching responses were followed by reinforcers on a variable-ratio schedule that led to a high reinforcer rate. For the other two stimuli, correct responses were followed by reinforcers on a variable-ratio schedule that led to a substantially lower reinforcer rate. Results on two-sample tests that followed showed that (a) on trials in which comparison arrays consisted of one high reinforcer-rate and one low reinforcer-rate stimulus, subjects most often selected the high-rate stimulus; and (b) on trials in which the comparison arrays were either two high reinforcer-rate stimuli or two low reinforcer-rate stimuli and the samples were one high reinforcer- and one low reinforcer-rate stimulus, accuracy was higher on trials with the high-rate comparisons. These results indicate that the frequency of stimulus control by high reinforcer-rate samples was greater than that by low reinforcer-rate samples. Following more training with the one-sample task and reversed reinforcement schedules for all stimuli, the differences in stimulus control frequencies on two-sample tests also reversed. These results demonstrate experimental control by reinforcement contingencies of which of two sample stimuli controlled selections in the two-sample task. The procedures and results may prove to be relevant for understanding restricted stimulus control and stimulus overselectivity.

Effects of 5-HT 1A receptor ligands in a modified Geller-Seifter conflict model in the rat

In a modified Geller-Seifter conflict procedure, rats were trained to lever-press for food under a multiple variable interval-fixed ratio (VI30: food; FR10: food+shock) schedule of reinforcement. The ability to antagonize response suppression in the punished period is considered a good predictor for anxiolytic activity. Chlordiazepoxide and alprazolam increased punished responding. The 5-HT 1A receptor agonists flesinoxan ( R(+)- N-[2[4-(2,3-dihydro-2-2-hydroxymethyl-1,4-benzodioxin-5-yl)-1-piperazinyl]ethyl]-4-fluorobenzoamide; 0.1–10.0 mg/kg) and 8-OH-DPAT (8-hydroxy-2-(di- n-propyl-amino)tetralin; 0.03–0.5 mg/kg) significantly increased punished responding, supporting a role of the 5-HT 1A receptor in anxiety. 8-OH-DPAT and flesinoxan also reduced unpunished responding. The anxiolytic effects of 8-OH-DPAT and flesinoxan could only be antagonized with a high dose (1.0 and 3.0 mg/kg respectively) of the 5-HT 1A receptor antagonist WAY-100635 ( N-[2-[4-(2-methoxyphenyl)-1-piperazinyl]ethyl]- N-(2-pyridinyl) cyclohexanecarboxamide trihydrochloride). All doses of WAY-100635 antagonized the 5-HT 1A-induced effects on unpunished responding. The dissimilarity in dose-response curve of WAY-100635 on punished and unpunished behaviour poses questions about the mediation of these effects. © 1997 Elsevier Science B.V.

Key pecking during extinction after intermittent or continuous reinforcement as a function of the number of reinforcers delivered during training.

Key pecking by 7 pigeons was established and maintained on a multiple variable-ratio variable-ratio (VR) schedule of food presentation. The schedule in one of the components was then changed to fixed-ratio (FR) 1 for a predetermined number of reinforcers. Both components were then changed to extinction (i.e., multiple extinction, extinction). This sequence was repeated a different number of times for each pigeon to determine the relation between the number of reinforcers delivered during each component of the multiple VR FR 1 schedule and the number of responses during extinction. For most pigeons, there were fewer responses during extinction in the presence of a stimulus recently correlated with FR 1, regardless of the number of reinforcers received. The ratio of the total responses in extinction in the former VR component to the total responses in the former FR 1 component increased as the number of reinforcers delivered during each component of the multiple schedule increased. Within-subject replications of the partial-reinforcement extinction effect generally occurred, and there were no overall reductions in the number of responses in extinction with repeated exposures to extinction.

BEHAVIORAL ECONOMICS AND BEHAVIORAL MOMENTUM

Some relations between elasticity of demand and the conditions of reinforcement are reanalyzed in terms of resistance to change, in ways suggested by the metaphor of behavioral momentum; some relations between resistance to change and the conditions of reinforcement are reanalyzed in terms of elasticity of demand, in ways suggested by behavioral economics. In addition, some data on labor supply in relation to variable-ratio schedules and alternative reinforcement are reanalyzed in terms of resistance to change and compared with steady-state resistance data for performance on multiple and concurrent interval schedules. The results of these studies can be summarized by two functions based on the behavioral momentum approach, relating relative behavioral mass to relative reinforcement per response or per unit time. The former is a relation between relative unit price and relative behavioral mass, suggesting the possibility of convergent measurement of a theoretical construct common to both approaches. However, the momentum and economic approaches differ fundamentally on whether it is preferable to construe discriminated operant behavior as selected and strengthened by its consequences or as part of a behavior-consequence bundle that maximizes utility.

Effects of a variable-ratio conditioning history on sensitivity to fixed-interval contingencies in rats.

We investigated the possibility that human-like fixed-interval performances would appear in rats given a variable-ratio history (Wanchisen, Tatham, & Mooney, 1989). Nine rats were trained under single or compound variable-ratio schedules and then under a fixed-interval 30-s schedule. The histories produced high fixed-interval rates that declined slowly over 90 sessions; differences as a function of the particular history were absent. Nine control animals given only fixed-interval training responded at lower levels initially, but rates increased with training. Despite differences in absolute rates, rates within the intervals and postreinforcement pauses indicated equivalent development of the accelerated response patterns suggestive of sensitivity to fixed-interval contingencies. The finding that the histories elevated rates without retarding development of differentiated patterns suggests that the effective response unit was a burst of several lever presses and that the fixed-interval contingencies acted on these units in the same way as for single responses. Regardless of history, the rats did not manifest the persistent, undifferentiated responding reported for humans under comparable schedules. We concluded that the shortcomings of animal models of human fixed-interval performances cannot be easily remedied by including a variable-ratio conditioning history within the model.

WITHIN‐SESSION CHANGES IN RESPONDING DURING SEVERAL SIMPLE SCHEDULES

Pigeons' key pecking was reinforced by food delivered by several fixed-interval, variable-ratio, and differential-reinforcement-of-low-rate schedules. Rate of responding, number of responses per reinforcer, length of postreinforcement pause, running response rate, and the time required to collect an available reinforcer changed systematically within sessions when the schedules provided high rates of reinforcement, but usually not when they provided low rates. These results suggest that the factors that produce within-session changes in responding are generally similar for different schedules of reinforcement. However, a separate factor may also contribute during variable-ratio schedules. The results question explanations for within-session changes that are related solely to the passage of time, to responding, and to one interpretation of attention. They support the idea that one or more factors related to reinforcement play a role.

RESPONSE‐RATE DIFFERENCES IN VARIABLE‐INTERVAL AND VARIABLE‐RATIO SCHEDULES: AN OLD PROBLEM REVISITED

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate-reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.

Delay reduction and optimal foraging: variable-ratio search in a foraging analogue.

The present study investigated conditions under which the conditioned reinforcement principles of delay-reduction theory and views based on simple maximization of reinforcement rate make ordinally opposing predictions with respect to foraging-related choice behavior. The use of variable-ratio schedules in the choice phase also represents an extension of delay-reduction theory to schedules that may better mimic the effort involved in searching. Pigeons responded on modified concurrent-chains schedules in which equal variable-ratio schedules led to unequal variable-interval outcomes and unequal reinforcer amounts. All 4 subjects completed a minimum of two replications of conditions for which the predictions of delay-reduction theory and a simple rate-maximizing theory were opposed. Results were consistent with delay reduction's ordinal predictions in 11 of 11 replications of the divergent predictions favoring the smaller, more immediate alternative. The predictions of rate maximization were upheld only when they were consistent with those of delay reduction. Results are discussed in terms of conditioned reinforcement, sensitivity to reductions in delay to food, and possible rules of thumb that may be useful in characterizing foraging.

Human choice in concurrent ratio-interval schedules of reinforcement.

Seven undergraduates participated in a concurrent-choice experiment with monetary reinforcers. Response-independent analogues of variable-interval and variable-ratio schedules were used to assess whether subjects would maximize reinforcement rate. The optimal pattern of behavior, in terms of maximizing reinforcement rate, involved a large bias toward the ratio alternative, with only occasional sampling of the interval schedule. Most experiments with pigeons, however, demonstrate matching of response rates to reinforcement rates, with only slight biases for the ratio schedule. Although subjects in the present experiment allocated more time to the ratio alternative than required by matching, the magnitude of the bias did not approximate that predicted by a maximizing account. After exposure to clock stimuli correlated with the operation of each schedule, 1 subject's behavior did show a substantial level of bias, increasing the total number of reinforcers obtained, and lay at a point between the predictions of matching and maximizing. The other subjects, however, continued to respond less optimally. The present results can be accounted for by a view of matching that incorporates the effects of delayed reinforcement.

BRIEF‐STIMULUS PRESENTATIONS ON MULTIFORM TANDEM SCHEDULES

Three experiments examined the influence of a brief stimulus (a light) on the behavior of food-deprived rats whose lever pressing on tandem schedules comprising components of different schedule types resulted in food presentation. In Experiment 1, either a tandem variable-ratio variable-interval or a tandem variable-interval variable-ratio schedule was used. The variable-interval requirement in the tandem variable-ratio variable-interval schedule was yoked to the time taken to complete the variable-ratio component in the tandem variable-interval variable-ratio schedule, and the length of the variable-interval component in the latter schedule was yoked to the variable-ratio component in the former schedule. If a brief stimulus occurred following completion of the first component, then behavior was differentiated in the two components; subjects responded more quickly in the variable-ratio than in the variable-interval component. If the stimulus was removed, then response rate was determined by the nature of the final component. Similar results were obtained in Experiments 2 and 3 with the use of a three-component tandem variable-ratio variable-interval variable-ratio schedule or tandem variable-interval variable-ratio variable-interval schedule. Thus, a brief stimulus that was not explicitly paired with reinforcement engendered behavior typical of the component schedule preceding its presentation.

Opioid drugs and timeout from avoidance.

The effects of µ agonists fentanyl, methadone and morphine and kappa agonist U50,488 on behavior maintained by negative reinforcement were determined. Rats were trained on concurrent schedules in which pressing one lever postponed shock on a Sidman avoidance schedule and pressing the other lever produced signaled periods of timeout from avoidance on variable-ratio schedules. All of the µ agonists decreased responding maintained by timeout from avoidance at doses that increased or did not affect avoidance rates. The kappa agonist U50,488 decreased response rates in some rats, but increased responding in others. In no case was a selective reduction in responding on the timeout lever produced by U50,488. Thus, the previously reported selective decreases in timeout responding by morphine are also produced by µ agonists fentanyl and methadone, but not by kappa agonist U50, 488.

The effect of food deprivation on the reinforcing value of food and smoking in bulimic and control women

The reinforcing value of both food and cigarettes was examined in female smokers with and without bulimia nervosa. Subjects underwent two food-deprivation (18 h) and two nondeprivation conditions and worked for either food vs. money or cigarettes vs. money on concurrent progressive variable ratio schedules. Schedules for food/cigarettes increased progressively while schedules for money remained the same. Results indicated no differences in the amount of time subjects spent working for food or the number of points earned for food in either bulimics or controls when examining the entire eight-game trial. Results of the initial two-game trial indicated marked differences, with the reinforcing value of food increasing after deprivation in control but not bulimic women and the total amount of time spent working for cigarettes increased after deprivation in controls. These results highlight that the effects of experimental food deprivation are most salient immediately after the deprivation and at low unit prices, and that bulimic and control women respond differently to a food-deprivation paradigm.

Influence of the cost of responding

In three experiments, the effect of costs associated with responding on judgments of the causal effectiveness of the response was examined. In Experiment 1, the temporal interval between outcomes was matched on a variable interval (VI) and a variable ratio (VR) schedule. When each response was made at some "cost," and outcomes represented some "gain" for the subject, the rating of causal effectiveness for responses was higher on the VR than on the VI schedule. This relationship was absent when the outcome was a triangle flash. In Experiment 2, the number of responses required per outcome on a VI and a VR schedule were matched, and responses on the VR schedule were rated as more causally effective. In Experiment 3, a VI-to-VR yoking procedure was used. With minimal response costs, judgments were similar on the VI and VR schedules, but with greater response costs, responses performed on the VR schedule were rated as more causally effective than those emitted on the VI schedule.

Differential sensitivity of operant behaviors to changes in the concentration of a sucrose reinforcer: Effects of pimozide

The sensitivity of operant response rates to changes in a sucrose reinforcer was examined in well-trained animals maintained on a variable ratio (VR) or variable interval (VI) schedule (experiment 1). Although VR performance showed greater resistance to small reductions in the concentration of the sucrose reinforcer than VI performance, VR performance was more sensitive to large reductions in the sucrose concentration. Despite this differential sensitivity only the smallest dose of pimozide (0.125 mg/kg) differentially affected these behaviors by reducing VI rates without affecting VR rates. These and other results support the view that low doses of pimozide reduce the hedonic impact of the reinforcer. The results also indicate that the attenuation of operant responding by higher doses (0.25, 0.5, and 1.0 mg/kg) cannot be solely a result of the blunting of reward. Experiment 2 demonstrated that when rats drink in daily, brief one-bottle tests they show greater resistance to reductions in the sucrose concentration than when they lever-press for sucrose, and require a higher dose of pimozide (2.0 mg/kg) to attenuate consumption. Together the results of both experiments suggest that the greater the resistance to reductions in the reinforcement value, the greater the dose of pimozide necessary to attenuate performance. We discuss the importance of attaining a more complete understanding of the factors in control of operant performance in order to better assess the effects of neuroleptics on reward.

In Utero Caffeine Exposure Affects Feeding Pattern and Variable Ratio Performance in Infant Monkeys

In Utero Caffeine Exposure Affects Feeding Pattern and Variable Ratio Performance in Infant Monkeys. Gilbert, S. G., and Rice, D. C. (1994). Fundam. Appl. Toxicol. 22, 41-50.Adult female monkeys (Macaca fascicularis) were exposed via their drinking water to 0.0, 0.15, or 0.35 mg/ml of caffeine prior to and throughout pregnancy. Caffeine exposure resulted in a dose-related increase in the number of infant deaths at parturition. Additional females were added to the control group and the original groups were rebred, which resulted in 16, 13, and 9 live infants in control, low-dose, and high-dose groups, respectively. During pregnancy the mean serum caffeine levels were 3.4 and 10.3 μg/ml and the mean serum theophylline levels were 6.6 and 12.9 μg/ml for the low-dose and high-dose groups, respectively. Infants were separated from their mothers at birth and reared in a primate nursery to facilitate evaluation of infant performance on a variety of behavioral tasks. Infant formula consumption was monitored by a computer-based system 19 hr per day until 30 days of age. At 30 days of age infants were trained to press a button for a formula reward, after which they performed on a variable ratio schedule for a 14-day period. Monitoring of feeding pattern revealed that the treated infants spent significantly more time feeding than controls. On the variable ratio schedule, the high-dose group had consistently longer pause times and longer interresponse times than the control group. The results from this study indicate that in utero exposure to caffeine and its metabolites results in altered behavioral patterns in infant monkeys.

In Utero Caffeine Exposure Affects Feeding Pattern and Variable Ratio Performance in Infant Monkeys

Adult female monkeys (Macacafascicularis) were exposed via their drinking water to 0.0, 0.15, or 0.35 mg/ml of caffeine prior to and throughout pregnancy. Caffeine exposure resulted in a dose-related increase in the number of infant deaths at parturition. Additional females were added to the control group and the original groups were rebred, which resulted in 16, 13, and 9 live infants in control, low-dose, and high-dose groups, respectively. During pregnancy the mean serum caffeine levels were 3.4 and 10.3 μg/ml and the mean serum theophylline levels were 6.6 and 12.9 μg/ml for the low-dose and high-dose groups, respectively. Infants were separated from their mothers at birth and reared in a primate nursery to facilitate evaluation of infant performance on a variety of behavioral tasks. Infant formula consumption was monitored by a computer-based system 19 hr per day until 30 days of age. At 30 days of age infants were trained to press a button for a formula reward, after which they performed on a variable ratio schedule for a 14-day period. Monitoring of feeding pattern revealed that the treated infants spent significantly more time feeding than controls. On the variable ratio schedule, the high-dose group had consistently longer pause times and longer inter-response times than the control group. The results from this study indicate that in utero exposure to caffeine and its metabolites results in altered behavioral patterns in infant monkeys.

SCHEDULE‐INDUCED DEFECATION BY RATS DURING RATIO AND INTERVAL SCHEDULES OF FOOD REINFORCEMENT

Lever pressing in rats was maintained by continuous and intermittent schedules of food while defecation was monitored. In Experiment 1, reinforcement densities were matched across variable-ratio and variable-interval schedules for three pairs of rats. Defecation occurred in all 3 rats on the variable-ratio schedule and in all 3 rats on the yoked variable-interval schedule. In Experiment 2, fixed-ratio and fixed-interval schedules with similar reinforcement densities maintained lever pressing. Defecation occurred in 3 of 4 rats on the fixed-ratio schedule and in 4 of 4 rats on the fixed-interval schedule. Almost no defecation occurred during continuous reinforcement in either experiment. These results demonstrate that defecation may occur during both ratio and interval schedules and that the inter-reinforcement interval is more important than the behavioral requirements of the schedule in generating schedule-induced defecation.

Variable-ratio schedules of timeout from avoidance

Concurrent performances in rats were studied under conditions where responses on one lever postponed shock on an unsignaled avoidance schedule, and responses on another level produced periods of signaled timeout from avoidance on a variable-ratio schedule. This procedure resulted in relatively high rates of responding on the timeout lever, and provided a baseline which permitted simultaneous evaluation of drug effects on two different types of negative reinforcement (shock postponement vs timeout). Chlordiazepoxide and ethanol selectively increased responding on the timeout lever at low doses, while higher doses decreased responding on both levers. Two 5-HT(1A) agonists, buspirone and 8-OH-DPAT, had different effects. Buspirone decreased responding across all effective doses, but 8-OH-DPAT increased responding on both the timeout and avoidance levers, with greater increases noted in responding maintained by timeout. These results replicate and extend previous findings, and support the notion that traditional anxiolytic drugs like chlordiazepoxide and ethanol may increase the reinforcing properties of escape from an avoidance schedule. Differences between the behavioral effects of buspirone and 8-OH-DPAT may reflect differential activity at the 5-HT(1A) receptor or the dopaminergic properties of buspirone.