Random-ratio Schedules Research Articles

Sampling of schedule parameters by pigeons: tests of optimizing theory

In three experiments, pigeons (domesticated Columba livia) were exposed to concurrent randomratio schedules of reinforcement in a keypecking apparatus, as a simulation of foraging in an environment containing two patches of unknown food density. All reinforcements were given for pecks on a single key, but different patch densities were associated with different key colours, and the colour was changed following pecks to a second key. Patch densities varied from day to day, and no colour reliably signalled the better patch from session to session. In the first experiment, either 256 or 1024 pecks were allowed per session. As predicted by optimal sampling theory, the proportion of the first 256 pecks made in the denser patch, and the number of reinforcements gained during them, were higher in the short than in the long sessions. These results were simulated using principles derived from operant conditioning theory. In the second experiment, the number of different patch densities used was varied between conditions. Birds made fewer patch changes when the density range was more restricted. In the third experiment, the colour cues differentiating patch densities were dropped in one condition, and the birds were less successful at finding the better patch in this condition. These results show that pigeons are sensitive to aspects of a foraging situation that are predicted by optimal sampling theory (e.g. the session length), and also in ways that have not been considered so far (e.g. patch density range). But they also show that cues that are not logically necessary (e.g. patch key colour) may in practice be an essential part of an animal's mechanism for achieving efficient foraging.

Sex differences in response rates during random ratio acquisition: Effects of gonadectomy

Sex differences in response rates were observed during acquisition of behavior on different random ratio schedules of reinforcement. Intact male rats responded at higher rates than intact female rats. Response rates of castrated males were found to be lower than response rates of intact males. Differences were not observed between intact and ovariectomized females.

CONSUMPTION‐LEISURE TRADEOFFS IN PIGEONS: EFFECTS OF CHANGING MARGINAL WAGE RATES BY VARYING AMOUNT OF REINFORCEMENT

Pigeons' rates of responding and food reinforcement under simple random-ratio schedules were compared with those obtained under comparable ratio schedules in which free food deliveries were added, but the duration of each food delivery was halved. These ratio-with-free-food schedules were constructed so that, were the pigeon to maintain the same rate of responding as it had under the simple ratio schedule, total food obtained (earned plus free) would remain unchanged. However, any reduction in responding would reduce total food consumption below that under the simple ratio schedule. These "compensated wage decreases" led to decreases in responding and decreases in food consumption, as predicted by an economic model of labor supply. Moreover, the reductions in responding increased as the ratio value increased (i.e., as wage rates decreased). Pigeons, therefore, substituted leisure for consumption. The relationship between these procedures and negative-income-tax programs is noted.

Operant-Pavlovian interactions: Ratio-schedules and the effects of the duration and location of a stimulus preceding response-independent food

Two groups of four pigeons were exposed to either a fixed-ratio or a random-ratio schedule of reinforcement. Presentation of stimuli which preceded response-independent food delivery was then superimposed upon this operant baseline. The duration and the location of stimulus presentation were varied in different experimental conditions. Random-ratio schedules maintained higher response rates than fixed-ratio schedules. Response rates during presentation of the stimulus on the operant key exceeded response rates in the absence of stimulus presentation for the fixed-ratio subjects, but not for the random-ratio subjects. Similar observations were made when stimulus presentation occurred on a seperate stimulus key. During stimulus presentation on the stimulus key, response rates on the operant key were considerably higher for the random-ratio subjects than for the fixed-ratio subjects, especially during longer presentation of the stimulus (30 sec).

Effects of chronically administered d-amphetamine on spaced responding maintained under multiple and single-component schedules.

Key pressing by rats was maintained under spaced-responding and random-ratio schedules of food delivery, and rates of responding were reliably different for each schedule. When responding was maintained under a multiple schedule, appropriate doses of d-amphetamine (1.0 and 1.7 mg/kg) markedly increased low rates of spaced responding while markedly decreasing high rates of ratio responding. These drug-produced changes in response rate resulted in decreased food presentation during both schedule components. When 1.0 mg/kg d-amphetamine was given daily, tolerance developed to initially decreased ratio responding in six to nine sessions, but did not develop to initially increased spaced responding. However, when the ratio schedule was removed, tolerance developed very quickly to increases under the spaced-responding schedule, and associated food frequency returned to control levels. When the ratio schedule was reinstated, spaced responding was once again increased, and its associated frequency of food delivery was again decreased. The development of tolerance to the behavioral effects of d-amphetamine was influenced more by global changes in response consequences during entire experimental sessions than by local changes in response consequences in single components of those sessions. Whenever the concept of "response cost" is used, it should be understood in terms of total "cost".

Contingency Effects with Maintained Instrumental Reinforcement

In three experiments we investigated the effect on the performance of thirsty rats of varying the instrumental contingency between lever pressing and the delivery of a saccharin reinforcer. In Experiment 1, the subjects performed more slowly in a non-contingent condition, in which the momentary probability of reinforcement was unaffected by whether or not the animals pressed, than in a contingent condition in which the reinforcer was never presented except following a lever press. This was true of performance under both random ratio and interval schedules in which the function determining the probability of reinforcement following a lever press remained the same across the contingent and non-contingent conditions. Experiment 2 demonstrated that instrumental performance was less affected when the contingency was degraded by the introduction of free reinforcers if these reinforcers were signalled. In Experiment 3, lever pressing was reinstated to some degree after non-contingent training by giving non-reinforced exposure to the operant chamber in the absence of the lever. These results suggest that free reinforcers depress instrumental behaviour through a performance mechanism engaged by their ability to support conditioning of the contextual cues.

Properties of behavior under different random ratio and random interval schedules: A parametric study

Four pigeons were trained to peck a key under different values of a temporally defined independent variable (T) and different probabilities of reinforcement (p). Parameter T is a fixed repeating time cycle and p the probability of reinforcement for the first response of each cycle T. Two dependent variables were used: mean response rate and mean postreinforcement pause. For all values of p a critical value for the independent variable T was found (T=1 sec) in which marked changes took place in response rate and postreinforcement pauses. Behavior typical of random ratio schedules was obtained at T ▪ 1 sec and behavior typical of random interval schedules at T ▪ 1 sec.

Apple Picker: Computer software for studying human responding on concurrent and multiple schedules

Apple Picker, a software package for the Apple II microcomputer, is described. Subjects respond by searching for “apples” in two “orchards.” “Attempted pickings” may be reinforced according to random ratio or interval schedules. “Search” responses are recorded and may be examined either as rates of responding in the two orchards or visually as paths the subject has traced during the session.

The effects of d-amphetamine, chlordiazepoxide and alpha-flupenthixol on food-reinforced tracking of a visual stimulus by rats.

Rats were trained to respond to one of two levers under a random ratio schedule of food reinforcement. Which of the levers was correct was redetermined before each response and signalled by a light. The effects of d-amphetamine (0.2-3.2 mg/kg), chlordiazepoxide (1-8 mg/kg), and the neuroleptic alpha-flupenthixol (0.03-0.33 mg/kg) on the efficiency of rats tracking this visual cue were examined. d-Amphetamine increased the proportion of responses made on the correct lever at low and intermediate doses, but reduced the proportion at 3.2 mg/kg. At the highest dose, chlordiazepoxide produced a small increase in this measure, together with a reduction in response rate, but alpha-flupenthixol had no effect, even at a dose reducing response rate. Low doses of amphetamine also increased switching between the levers, producing a proportionately greater increase in switching from the correct lever to the incorrect lever than vice versa. The results are interpreted as showing that d-amphetamine facilitates tracking performance as a result of its action of enhancing response switching, and supporting the hypothesis that facilitation of performance by amphetamine-like drugs depends on the effect of the drug on response output coinciding with task requirements.

Observing Responses in Rats: Support for the Secondary Reinforcement Hypothesis

Rats’ lever presses on a retractable lever earned brief presentations of discriminative stimuli signalling periods in which responding on an alternative lever was either non-reinforced (extinction) or reinforced on a random ratio schedule. The predictions of two theoretical accounts of this behaviour were tested by studying the effects of omitting either the stimulus signalling the reinforced or that signalling the non-reinforced schedule component. Under these conditions rats’ behaviour is determined by the conditioned, affective properties of the stimuli rather than by their purely informational properties.

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.

STEADY‐STATE PERFORMANCE ON FIXED‐, MIXED‐, AND RANDOM‐RATIO SCHEDULES

Three groups of rats pressed a lever for milk reinforcers on various simple reinforcement schedules (one schedule per condition). In Group M, each pair of conditions included a mixed-ratio schedule and a fixed-ratio schedule with equal average response:reinforcer ratios. On mixed-ratio schedules, reinforcement occurred with equal probability after a small or a large response requirement was met. In Group R, fixed-ratio and random-ratio schedules were compared in each pair of conditions. For all subjects in these two groups, the frequency distributions of interresponse times of less than one second were very similar on all ratio schedules, exhibiting a peak at about .2 seconds. For comparison, subjects in Group V responded on variable-interval schedules, and few interresponse times as short as .2 seconds were recorded. The results suggest that the rate of continuous responding is the same on all ratio schedules, and what varies among ratio schedules is the frequency, location, and duration of pauses. Preratio pauses were longer on fixed-ratio schedules than on mixed-ratio or random-ratio schedules, but there was more within-ratio pausing on mixed-ratio and random-ratio schedules. Across a single trial, the probability of an interruption in responding decreased on fixed-ratio schedules, was roughly constant on random-ratio schedules, and often increased and then decreased on mixed-ratio schedules. These response patterns provided partial support for Mazur's (1982) theory that the probability of instrumental responding is directly related to the probability of reinforcement and the proximity of reinforcement.

INSTRUCTED VERSUS SHAPED HUMAN VERBAL BEHAVIOR: INTERACTIONS WITH NONVERBAL RESPONDING

Undergraduate students' presses on left and right buttons occasionally made available points exchangeable for money. Blue lights over the buttons were correlated with multiple random-ratio random-interval components; usually, the random-ratio schedule was assigned to the left button and the random-interval to the right. During interruptions on the multiple schedule, students filled out sentence-completion guess sheets (e.g., The way to earn points with the left button is to...). For different groups, guesses were shaped with differential points also worth money (e.g., successive approximations to "press fast" for the left button), or were instructed (e.g., Write "press slowly" for the left button), or were simply collected. Control of rate of pressing by guesses was examined in individual cases by reversing shaped or instructed guesses, by instructing pressing rates, and/or by reversing multiple-schedule contingencies. Shaped guesses produced guess-consistent pressing even when guessed rates opposed those characteristic of the contingencies (e.g., slow random-ratio and fast random-interval rates), whereas guesses and rates of pressing rarely corresponded after unsuccessful shaping of guesses or when guessing had no differential consequences. Instructed guesses and pressing were inconsistently related. In other words, when verbal responses were shaped (contingency-governed), they controlled nonverbal responding. When they were instructed (rule-governed), their control of nonverbal responding was inconsistent: the verbal behavior sometimes controlled, sometimes was controlled by, and sometimes was independent of the nonverbal behavior.

UNINSTRUCTED HUMAN RESPONDING: SENSITIVITY OF LOW‐RATE PERFORMANCE TO SCHEDULE CONTINGENCIES

College students' presses on a telegraph key occasionally turned on a light in the presence of which button presses produced points later exchangeable for money. Initially, responding was maintained by low-rate contingencies superimposed on either random-interval or random-ratio schedules. Later, the low-rate contingencies were relaxed. Low-rate key pressing had been established for some students by shaping and for others by demonstration and written instructions. After the low-rate contingencies were relaxed, higher response rates generally did not increase point earnings with random-interval scheduling, but did so with random-ratio scheduling. In both cases, shaped responding usually increased, and instructed responding usually continued at an unchanged low rate. The insensitivity of instructed responding typically occurred despite contact with the contingencies. The differential sensitivity to schedule contingencies of shaped responding relative to instructed responding is consistent with the different properties of contingency-governed and rule-governed behavior and is not rate-dependent.

Simulated Slot-Machine Play with Concurrent Variable Ratio and Random Ratio Schedules of Reinforcement

Variable and random ratio schedules of reinforcement are conceptually contrasted and experimentally compared in a computer simulated “slot machine” task. 20 subjects did not behave differentially in terms of either choice of game or employment of strategy.

Resistance to Extinction And Satiation Following Training on Random-Ratio Schedules of Reinforcement

Resistance to extinction and resistance to satiation were examined with rats after training on random-ratio 1,5, 10, 20, 30, and 40 schedules of food reinforcement. During training, response rate was an increasing function of the random ratio. Relative to baseline, extinction responding increased and responding under reduced deprivation decreased as functions of the random ratio. A formula based on reinforcement frequency and responses per reinforcer during training was devised to describe extinction responding. Further support for the formula was obtained from data on responding during extinction after training on fixed-ratio (Boren, 1961) and fixed-interval (Wilson, 1954) schedules of reinforcement.

FIXED‐RATIO DISCRIMINATION: EFFECTS OF INTERMITTENT REINFORCEMENT

Four pigeons had discrimination training that required the choice of a left side-key after completing a fixed-ratio 10 on the center key, and a right side-key choice after fixed-ratio 20. Correct choices were reinforced on various fixed-interval, fixed-ratio, random-interval, and random-ratio schedules. When performance was examined across successive 15-second intervals (fixed-interval and fixed-ratio schedules) accuracy was high in the first 15-second interval, decreased in one or several of the next 15-second intervals, and then increased again as reinforcement was approached. When performance was examined across correct trials on fixed-interval and fixed-ratio schedules, accuracy was lowest immediately after reinforcement, followed by a systematic increase in accuracy as the number of correct choices increased. These patterns were due primarily to errors on fixed-ratio 20 trials. Systematic accuracy patterns did not occur on random-interval or random-ratio schedules. The results indicate that when choice patterns differed on fixed-interval and fixed-ratio schedules, the differences were due to the method of data analysis.

Low-cost equipment for schedule generation I: Fixed- and random-ratio schedules

The use of complementary metal oxide semiconductor (CMOS) technology to produce fixed-ratio and random-ratio schedules is described. Such a technology implies several advantages over traditional methods of construction, especially with respect to cost and complexity. Suitable interfaces for linking with everyday laboratory equipment are also described.

EFFECTS OF RESPONSE RATE, REINFORCEMENT FREQUENCY, AND THE DURATION OF A STIMULUS PRECEDING RESPONSE‐INDEPENDENT FOOD1

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.

EFFECTS OF A PRE‐REWARD STIMULUS ON A TEMPORAL DISCRIMINATION

Pigeons responded in a situation in which a response on key B could be followed by reinforcement if a response on key A preceded it by 10 sec. A‐to‐B responses with intervals of greater than 10 sec. duration were reinforced on a random ratio schedule. The effects of a conditioned stimulus terminating in response‐independent reinforcement were assessed against this baseline. Two levels of response‐independent reinforcement (6 or 10 sec. hopper access) and two conditioned stimulus durations (1.5 min. and 3 min.) were used. Except for the combination of the 3 min. conditioned stimulus with the 6 sec. response‐independent reinforcement, all other combinations produced a suppression of A‐to‐B responding during the conditioned stimulus. Concomitantly, the number of correct (i.e. > 10 sec.) responses, and in some cases the percentage of total responses that were correct, were reduced. All of these effects were seen most clearly using the conditioned stimulus of shorter duration.