Random-ratio Schedules Research Articles

The structure of random ratio responding in humans.

Four experiments explored the rate and structure of human responding on random ratio (RR) schedules of reinforcement, using 3 different methods of analysis. All experiments demonstrated that responding on RR schedules was higher with intermediate ratio values-rates being higher on an RR-30 schedule than on either a RR-10 or RR-60 schedule. All experiments identified 2 distinct types of responding generated by RR schedules: "response-initiation" responding and "within-burst" responding. In contrast to overall rates, response-initiation responding was greatest on RR-10 schedules, and less pronounced as the ratio value increased, but response rates did not vary greatly within the burst (being higher on longer ratio schedules when there was a difference). These findings are consistent with previously obtained data from nonhumans, and suggest human schedule performance should be considered as a joint product of 2 different aspects of responding: response-initiation responding which is sensitive to rates of reinforcement; and within-burst responding which is less sensitive to the molar characteristics of the schedule in operation. The presence of 2 distinct forms of responding may require reinterpretation of many human schedule-controlled phenomena that rely on assumptions about the similarity of all responses to one another.

Rats show molar sensitivity to different aspects of random-interval-with-linear-feedback-functions and random-ratio schedules.

Three experiments examined the impact of various aspects of reinforcement contingencies on responding maintained by free-operant schedules by food-deprived rats. Experiment 1 demonstrated that random interval (RI) and random-interval-with-positive-response-reinforcer-feedback (RI+) schedules maintained similar rates of responding at a variety of reinforcer frequencies. Experiment 2 demonstrated that a random ratio (RR) schedule maintained higher rates than RI or RI+ schedules, except at high rates of reinforcement, where response rates were similar on all schedules. Experiment 3 again demonstrated that RR schedules produced higher response rates than either RI or RI+ schedules, but modification of the RI+ schedule to prevent ratio strain enhanced response rates relative to an RI schedule. Together these results reveal a pattern of interacting factors in schedule controlled behavior: at high rates of reinforcement, this factor overrides the impact of other controlling factors, but as reinforcement rate decreases the joint impact of interresponse times reinforcement, response-reinforcer feedback functions, and ratio strain are observed.

A FUNCTIONAL ANALYSIS OF BEHAVIOR IN THE BALLOON ANALOGUE TASK

In the balloon analogue task, a continuous schedule of positive reinforcement and a random ratio schedule of punishment for “pump” responses operate simultaneously, along with the opportunity to terminate each trial and avoid an impending punishing stimulus. In this study, the functional properties of game points and point losses were analyzed in three groups of college students (N=92) by varying the density of balloon explosions throughout the 30-balloon session. The within-session changes in density of explosions produced immediate systematic adjustments in pump responses per balloon. The results showed that 1) the number of pumps per balloon is sensitive to the local probability of explosions; 2) the suppression brought on by explosions depends on the temporal distance to the last explosion and the number of consecutive explosions; 3) suppression appears to be non-linear, with stronger effects as the local probability of explosions increases; and, 4) within limits, points without assigned extrinsic value may serve as reinforcers in the balloon analogue environment for populations with a history of game playing. Given these findings, a valuable strategy for future research may be to study differential reactivity to wins and losses as a proximal determinant of total scores that is amenable to experimental analysis.

Human performance on random ratio and random interval schedules, performance awareness and verbal instructions.

Humans responded on multiple random-ratio (RR) random-interval (RI) schedules, and their verbalized performance awareness (PA; i.e., their ability to accurately describe what they did) was measured in three experiments. In Experiment 1, instructions informed participants that to earn points, either sometimes rapid responding and sometimes slow responding would work best (accurate instructions); rapid responding would work best (go fast instructions); spaced responding would work best (go slow instructions); or no advice was provided (minimal instructions). In Experiments 2 and 3, participants received either accurate or minimal instructions and were subject to extinction after a multiple RR-RI schedule. In all experiments, both performance awareness, and receiving accurate instructions, were related to schedule-sensitive responding, but were unrelated to one another - participants receiving accurate-rate instructions were not more likely to show performance awareness than those exposed to minimal instructions. Both higher performance awareness and exposure to accurate instructions predicted faster extinction in Experiment 2 but not in Experiment 3. The current results suggest that performance awareness rather than contingency awareness is more strongly related to humans displaying schedule-typical behavior and that this is not strongly related to any explicit verbal instructions that are given.

Random-ratio schedules produce greater demand for i.v. drug administration than fixed-ratio schedules in rhesus monkeys

Organisms emit more responses when food is provided according to random as compared with fixed schedules of reinforcement. Similarly, many human behaviors deemed compulsive are maintained on variable schedules (e.g., gambling). If greater amounts of behavior are maintained by drugs of abuse when earned according to variably reinforced schedules, this would suggest that excessive drug-taking behavior may be due in part to the nature of drug availability. The aim was to determine whether random schedules of contingent intravenous drug delivery would produce more responding than similarly priced fixed schedules. Six rhesus macaque subjects responded to produce cocaine (0.003-0.03mg/kg/inj), remifentanil (0.01-1.0μg/kg/inj), or ketamine (0.01-0.1mg/kg/inj) according to either fixed or random ratio requirements that increased systematically across sessions. Demand curves were generated with the most effective dose of each drug and compared across drug and schedule type. Cocaine and remifentanil maintained higher levels and rates of responding when earned according to random-ratio schedules as compared with fixed-ratio schedules. This difference was most pronounced when drugs were available at high unit prices. Differences in responding across the schedule types generated by ketamine-a lesser-valued reinforcer-were qualitatively similar but smaller in magnitude. The current study provides a systematic replication across reinforcer type demonstrating that drugs delivered after a random number of responses generate more behavior than those delivered according to a fixed schedule. The variable nature of the availability of drugs of abuse-particularly those that are scarce or expensive-may be a contributing factor to excessive drug intake by humans. This effect is most likely to be observed when more highly demanded (reinforcing) drugs are being consumed.

The Effect of Ratio and Interval Training on Pavlovian-Instrumental Transfer in Mice

Conditional stimuli (CS) that are paired with reward can be used to motivate instrumental responses. This process is called Pavlovian-instrumental transfer (PIT). A recent study in rats suggested that habitual responses are particularly sensitive to the motivational effects of reward cues. The current experiments examined this idea using ratio and interval training in mice. Two groups of animals were trained to lever press for food pellets that were delivered on random ratio or random interval schedules. Devaluation tests revealed that interval training led to habitual responding while ratio training produced goal-directed actions. The presentation of CSs paired with reward led to positive transfer in both groups, however, the size of this effect was much larger in mice that were trained on interval schedules. This result suggests that habitual responses are more sensitive to the motivational influence of reward cues than goal-directed actions. The implications for neurobiological models of motivation and drug seeking behaviors are discussed.

Some factors modulating the strength of resurgence after extinction of an instrumental behavior

In resurgence, an operant behavior that has undergone extinction can return (“resurge”) when a second operant that has replaced it itself undergoes extinction. The phenomenon may provide insight into relapse that may occur after incentive or contingency management therapies in humans. Three experiments with rats examined the impact of several variables on the strength of the resurgence effect. In each, pressing one lever (L1) was first reinforced and then extinguished while pressing a second, alternative, lever (L2) was now reinforced. When L2 responding was then itself extinguished, L1 responses resurged. Experiment 1 found that resurgence was especially strong after an extensive amount of L1 training (12 as opposed to 4 training sessions) and after L1 was reinforced on a Random Ratio schedule as opposed to a variable interval schedule that was matched on reinforcement rate. Experiment 2 found that after 12 initial sessions of L1 training, 4, 12, or 36 sessions of Phase 2 each allowed substantial (and apparently equivalent) resurgence. Experiment 3 found no effect of changing the identity of the reinforcer (from grain pellet to sucrose pellet or sucrose to grain) on the amount of resurgence. The results suggest that resurgence can be robust; in the natural world, an operant behavior with an extensive reinforcement history may still resurge after extensive incentive-based therapy. The results are discussed in terms of current explanations of the resurgence effect.

Relationship between contingency awareness and human performance on random ratio and random interval schedules

In three experiments, human participants pressed the space bar on a computer keyboard to earn points on random-ratio (RR) and random-interval (RI) schedules of reinforcement. Verbalized contingency awareness (CA) for each schedule was measured after the entire task (Experiments 1 and 2), or after each RR–RI trial (Experiment 3). In all three experiments, an ability to accurately verbalize the appropriate contingency for each schedule was associated with typical schedule responding. In addition, lagged correlational analysis of verbalized contingency awareness and schedule performance was undertaken in Experiment 3. Results of these correlations initially suggested that CA scores were better predictors of typical schedule performance on the following trial than typical schedule performance was of CA scores on the following trial; however, further analysis revealed no significant differences between the alternate lagged correlation types in each pair. Findings, therefore, suggest that operant performance on RR and RI schedules is associated with participant-derived rule governed behavior, and results are discussed in terms of the introduction of a new paradigm for measuring this association, and the potential for further research to manipulate these parameters further in order to investigate causality in this respect.

Influence of schizotypy on responding and contingency awareness on free-operant schedules of reinforcement

Schedules of reinforcement typically produce reliable patterns of behaviour, and one factor that can cause deviations from these normally reliable patterns is schizotypy. Low scorers on the unusual experiences subscale of the Oxford-Liverpool Inventory of Feelings and Experiences performed as expected on a yoked random-ratio (RR), random-interval (RI) schedule of reinforcement, with significantly higher rates of responding on the RR schedule than in the RI schedule. However, high scorers in UE showed no such differences between response rates between the RR and RI schedules. In addition, contingency awareness scores were high, and did not differ in low UE scorers for both types of schedule, whilst awareness scores differed significantly between the schedules in high UE scorers, with more awareness of the RR schedule than the RI one. These results suggest that, as well as being unable to differentiate between the RR and RI schedules in terms of response rates, high UE scorers are also unable to verbally describe the RI schedule parameters.

Impaired Formation of Stimulus–Response, But Not Action–Outcome, Associations in Rats with Methamphetamine-Induced Neurotoxicity

Methamphetamine (METH) induces neurotoxic changes, including partial striatal dopamine depletions, which are thought to contribute to cognitive dysfunction in rodents and humans. The dorsal striatum is implicated in action-outcome (A-O) and stimulus-response (S-R) associations underlying instrumental learning. Thus, the present study examined the long-term consequences of METH-induced neurotoxicity on A-O and S-R associations underlying appetitive instrumental behavior. Rats were pretreated with saline or a neurotoxic regimen of METH (4 × 7.5-10 mg/kg). Rats trained on random ratio (RR) or random interval (RI) schedules of reinforcement were then subjected to outcome devaluation or contingency degradation, followed by an extinction test. All rats then were killed, and brains removed for determination of striatal dopamine loss. The results show that: (1) METH pretreatment induced a partial 45-50% decrease in striatal dopamine tissue content in dorsomedial and dorsolateral striatum; (2) METH-induced neurotoxicity did not alter acquisition of instrumental behavior on either RR or RI schedules; (3) outcome devaluation and contingency degradation similarly decreased responding in saline- and METH-pretreated rats trained on the RR schedule, suggesting intact A-O associations guiding behavior; (4) outcome devaluation after training on the RI schedule decreased extinction responding only in METH-pretreated rats, suggesting impaired S-R associations. Overall, these data suggest that METH-induced neurotoxicity, possibly due to impairment of the function of dorsolateral striatal circuitry, may decrease cognitive flexibility by impairing the ability to automatize behavioral patterns.

Random-ratio schedules of reinforcement: The role of early wins and unreinforced trials

The distribution of rewards in both variable-ratio and random-ratio schedules is examined with specific reference to gambling behaviour. In particular, it is the number of early wins and unreinforced trials that is suggested to be of importance in these schedules, rather than the often-reported average frequency of wins. Gaming machine data are provided to demonstrate the importance of early wins and unreinforced trials. Additionally, the implication of these distributional properties for betting strategies and the gambler's fallacy is discussed. Finally, the role of early wins and unreinforced trials is considered for gambling research that utilises simulated gaming machines and research that compares concurrent schedules of reinforcement.

Effect on subsequent fixed-interval schedule performance of prior exposure to ratio and interval schedules of reinforcement

In three experiments, we examined the effect on the patterns of responding noted on fixed interval (FI) schedules of prior exposure to a range of interval and ratio schedules. Rats leverpressed for food reinforcement on random ratio (RR), random interval (RI), or variable interval (VI) schedules prior to transfer to FI schedules. In Experiment 1, prior exposure to an RR schedule retarded the development of typical FI patterns of responding. Exposure to a yoked RI schedule produced even greater retardation of typical FI performance. This effect was replicated in Experiment 2, using a within-subjects design. Rats responded on a multiple RR-RI schedule prior to a multiple FI-FI schedule. Typical FI performance emerged more slowly in the component previously associated with the RI than with that associated with the RR. In Experiment 3, exposure to an RR schedule retarded the development of FI performance to a greater extent than did exposure to a VR schedule. The latter schedule was programmed to allow the possibility that inhibitory control would develop after reinforcement. These results confirm that ratio schedules independently result in the disruption of FI responding. This effect was not long lasting and cannot be used plausibly to explain species differences in responding to FI schedules. However, it does suggest that temporal control--as manifested by the transfer of inhibitory control from one schedule to another--could facilitate movement between interval schedules.

Human choice among five alternatives when reinforcers decay

Human participants played a computer game in which choices among five alternatives were concurrently reinforced according to dependent random-ratio schedules. "Dependent" indicates that choices to any of the wedges activated the random-number generators governing reinforcers on all five alternatives. Two conditions were compared. In the hold condition, once scheduled, a reinforcer - worth a constant five points - remained available until it was collected. In the decay condition, point values decreased with intervening responses, i.e., rapid collection was differentially reinforced. Slopes of matching functions were higher in the decay than hold condition. However inter-subject variability was high in both conditions.

Augmented reinforcer value and accelerated habit formation after repeated amphetamine treatment

Various processes might explain the progression from casual to compulsive drug use underlying the development of drug addiction. Two of these, accelerated stimulus–response (S–R) habit learning and augmented assignment of motivational value to reinforcers, could be mediated via neuroadaptations associated with long-lasting sensitization to psychostimulant drugs, i.e. augmented dopaminergic neurotransmission in the striatum. Here, we tested the hypothesis that both processes, which are often regarded as mutually exclusive alternatives, are present in amphetamine-sensitized rats. Amphetamine-sensitized rats showed increased responding for food under a random ratio schedule of reinforcement, indicating increased incentive motivational value of food. In addition, satiety-specific devaluation experiments under a random interval schedule of reinforcement showed that amphetamine-sensitized animals exhibit accelerated development of S–R habits. These data show that both habit formation and motivational value of reinforcers are augmented in amphetamine-sensitized rats, and suggest that the task demands determine which behavioral alteration is most prominently expressed.

Endocannabinoid signaling is critical for habit formation

Extended training can induce a shift in behavioral control from goal-directed actions, which are governed by action-outcome contingencies and sensitive to changes in the expected value of the outcome, to habits which are less dependent on action-outcome relations and insensitive to changes in outcome value. Previous studies in rats have shown that interval schedules of reinforcement favor habit formation while ratio schedules favor goal-directed behavior. However, the molecular mechanisms underlying habit formation are not well understood. Endocannabinoids, which can function as retrograde messengers acting through presynaptic CB1 receptors, are highly expressed in the dorsolateral striatum, a key region involved in habit formation. Using a reversible devaluation paradigm, we confirmed that in mice random interval schedules also favor habit formation compared with random ratio schedules. We also found that training with interval schedules resulted in a preference for exploration of a novel lever, whereas training with ratio schedules resulted in less generalization and more exploitation of the reinforced lever. Furthermore, mice carrying either a heterozygous or a homozygous null mutation of the cannabinoid receptor type I (CB1) showed reduced habit formation and enhanced exploitation. The impaired habit formation in CB1 mutant mice cannot be attributed to chronic developmental or behavioral abnormalities because pharmacological blockade of CB1 receptors specifically during training also impairs habit formation. Taken together our data suggest that endocannabinoid signaling is critical for habit formation.

Modification of slot-machine preferences through the use of a conditional discrimination paradigm.

The present experiment investigated the impact of contextually trained discriminations on gambling behavior. Nine recreational slot-machine players were initially exposed to concurrently available computerized slot machines that were each programmed on random-ratio schedules of reinforcement and differed only in color. All participants distributed responding equally across the two slot machines. A conditional discrimination procedure was then used to teach the contextual cues representing the arbitrary relations of "greater than" and "less than." Following contextual cue training, participants were reexposed to the concurrent slot-machine task. After training of the contextual cues, a higher proportion of responses were made to the slot machine that shared formal properties (i.e., color) with the contextual cue representing "greater than."

Induction When Rats Respond for Liquid-Sucrose Reinforcement as a Function of Amount of Upcoming “Work”

Previous research has shown that rats increase their rate of responding for 1 % sucrose reinforcement in the first half of the session if food-pellet, rather than 1 % sucrose, reinforcement will be available in the second half (i.e., positive induction). Four experiments investigated whether this induction effect would be changed by altering the amount of “work” required to earn reinforcement in the second half of the session. In.Experiments 1, 2, and 3, rats pressed a lever for 1 % sucrose delivered by a random-interval 60-s schedule during the first 25 min of the session. In different conditions, the reward in the second half was either 1 % sucrose or a food pellet delivered on a lIixed-ratio (FR) 10, 20, or 40 schedule. The FR schedule was in eff,ect throughout the second half of the session (Experiment 1), until 25 reinforcers had been collected and then extinction was in effect (Experiment 2), or until 25 reinforcers had been collected and then the session ended (Experiment 3). Positive induction occurred in each experiment, but did not vary as a function of upcoming FR requirement. Experiment 4 yoked the rate of reinforcement on a random-interval schedule in the second half of the session to that obtained when subjects responded on a random-ratio schedule,. Similar induction effects were observed regardless of which schedule was in effect in the second half of the session. These results suggest induction is primarily controlled by the availability of upcoming differential reinforcement, not by upcoming “work.”

A steady-state test of the demand curve analysis of relative reinforcer efficacy.

The demand curve analysis of relative reinforcer efficacy predicts that preference for 1 reinforcer over another may be predicted from relative levels of consumption under single-schedule conditions. Prior experiments with human cigarette smokers have largely confirmed this prediction, but because subjects completed only 1 session under each arrangement, questions remain about the stability of single and concurrent-schedule (choice) performances. In addition, no prior tests of this prediction of the demand curve analysis have used nonhuman subjects, so questions remain about the effects of instructions and demand characteristics. In the present study, the authors examined pigeons' steady-state preferences in a closed economy across a wide range of concurrent fixed-ratio and random-ratio (RR) schedules. On the basis of the pigeons' single-schedule consumption, the demand curve analysis qualitatively predicted indifference at low ratio values and preference for the RR alternative at higher ratio values. These predictions proved correct in 88% of the cases in which the analysis predicted preferences in individual subjects at specific concurrent-schedule values. These findings offer preliminary support for the position that the predictions of the demand curve analysis hold true when steady-state performances are assessed. They also offer preliminary evidence that previous confirmations of the demand curve analysis were not due to the influence of instructions or to demand characteristics of the human laboratory setting. These findings supplement prior questioning of the coherence and utility of traditional measures of relative reinforcer efficacy.

Labor supply and consumption of food in a closed economy under a range of fixed- and random-ratio schedules: tests of unit price.

The behavioral economic concept of unit price predicts that consumption and response output (labor supply) are determined by the unit price at which a good is available regardless of the value of the cost and benefit components of the unit price ratio. Experiment 1 assessed 4 pigeons' consumption and response output at a range of unit prices. In one condition, food was available according to a range of fixed-ratio schedules, whereas in the other condition, food was available according to a range of random-ratio schedules. Consistent with unit price predictions, consumption and response output were approximately equivalent across schedule types within the lower range of unit prices. However, at Unit Prices 64 (ratio value = 192) and greater, considerably more consumption and response output were observed in the random-ratio condition. Experiment 2 replicated these findings with 4 pigeons using the rapid demand curve assay procedure that is commonly used in the behavioral economics literature. Findings are integrated with two mathematical models of behavior under variable reinforcer delays.

Control of Variation by Reward Probability.

Two bar-press experiments with rats tested the rule that reducing expectation of reward increases the variation from which reward selects. Experiment 1 used a discrete-trial random-interval schedule, with trials signaled by light or sound. One signal always ended with reward; the other signal ended with reward less often. The 2 signals were randomly mixed. Bar-press duration (how long the bar was held down) varied more during the signal with the lower probability of reward. Experiment 2 closely resembled Experiment 1 but used a random-ratio schedule rather than a random-interval schedule. Again, bar-press duration varied more during the signal with the lower probability of reward. The results support the rule--the first well-controlled comparisons to do so.