Variable-interval 30-s Schedule Research Articles

Humans' choice in a self-control choice situation: sensitivity to reinforcer amount, reinforcer delay, and overall reinforcement density.

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.

Assaying lakatos's philosophy of mathematics

Five pigeons were trained in a procedure in which, with a specified probability, food was either available on a fixed-interval schedule on the left key, or on a variable-interval schedule on the right key. In Phase 1, we arranged, with a probability of 0.5, either a left-key fixed-interval schedule or a right-key variable-interval 30 s, and varied the value of the fixed-interval schedule from 5 s to 50 s across 5 conditions. In Phase 2, we arranged either a left-key fixed-interval 20-s schedule or a right-key variable-interval 30-s schedule, and varied the probability of the fixed-interval schedule from 0.05 to 1.0 across 8 conditions. Phase 3 always arranged a fixed-interval schedule on the left key, and its value was varied over the same range as in Phase 1. In Phase 1, overall preference was generally toward the variable-interval schedule, preference following reinforcers was initially toward the variable-interval schedule, and maximum preference for the fixed-interval schedule generally occurred close to the arranged fixed-interval time, becoming relatively constant thereafter. In Phase 2, overall left-key preference followed the probability of the fixed-interval schedule, and maximum fixed-interval choice again occurred close to the fixed-interval time, except when the fixed-interval probability was 0.1 or less. The pattern of choice following reinforcers was similar to that in Phase 1, but the peak fixed-interval choice became more peaked with higher probabilities of the fixed interval. Phase 3 produced typical fixed-interval schedule responding. The results are discussed in terms of reinforcement effects, timing in the context of alternative reinforcers, and generalized matching. These results can be described by a quantitative model in which reinforcer rates obtained at times since the last reinforcer are distributed across time according to a Gaussian distribution with constant coefficient of variation before the fixed-interval schedule time, changing to extended choice controlled by extended reinforcer ratios beyond the fixed-interval time. The same model provides a good description of response rates on single fixed-interval schedules.This article is part of a Special Issue entitled: SQAB 2012.

Humans’ Choice between Different Reinforcer Amounts and Delays: Effects of Choice Procedures and Monetary Deduction

Human subjects were exposed to concurrent-chains schedules in which reinforcer amounts or delays were varied in the terminal links and consummatory responses were required to receive points later exchangeable for money. Two independent variable-interval 30-s schedules or a single variable-interval 15-s schedule were in effect during the initial links and delay periods were defined by fixed-time schedules used in the terminal links. In addition to varying reinforcer amount and delay, two different choice procedures were studied for two different groups and a monetary deduction procedure was studied, within subjects, under independent scheduling procedure. The human subjects preferred the larger of two different reinforcer amounts and the shorter of two different reinforcer delays. The degree of preference was higher in the independent scheduling procedure than in the nonindependent scheduling procedure and there was no substantial difference in preference between the absence and presence of the monetary deduction procedure.

Representing within-session response rates proportionally and entirely.

In this technical article, methods for collecting and representing response rates maintained by schedules of reinforcement are presented. First, the time in a session that each important event (e.g., responses, reinforcers) occurs is collected and stored by a computer. Another computer program is used, then, to convert each response to a percentage of the total responses in a session and to plot these percentages cumulatively as a function of the time in the session that they occurred. In this manner, response rates may be expressed proportionally (i.e., using the same y-axis scale regardless of absolute response rate) without requiring the arbitrary selection of an interval over which responses are aggregated and expressed relative to the entire-session rate. A property of these records is that deviations in the slope of the obtained record from the diagonal, which connects (x, y) = (start of session, 0%) to (x, y) = (end of session, 100%), occurring at any point and for any duration, represent changes in the local response rate from the entire-session rate. This method of representing ongoing responding is illustrated by several records of key pecking of a pigeon on a variable-interval 60-s schedule of food reinforcement. Relative local response rates were also computed from these data at several levels of resolution (i.e., the time over which responses were aggregated), including the level typically employed by those interested in within-session changes in response rates.

CHANGEOVER BEHAVIOR AND PREFERENCE IN CONCURRENT SCHEDULES

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.

Effects of cocaine on briefly signaled versus completely signaled delays to reinforcement.

Key pecking by 4 pigeons was maintained by a multiple schedule consisting of two variable-interval 60-s schedules wherein each food presentation followed a nonresetting 27-s delay that was either briefly signaled at its outset or completely signaled. Brief-signal duration was adjusted so that response rates maintained by the briefly and completely signaled delays of reinforcement were similar. In general, acute administration of small to intermediate doses (0.3 to 3.0 mg/kg) of cocaine produced either small increases in response rates in both components or no change, and larger doses (5.6 to 13.0 mg/kg) decreased response rates. Chronic (i.e., daily) cocaine administration (10.0 mg/kg) resulted in tolerance to the rate-decreasing effects in both components. Cocaine's effects were generally similar whether delays were completely or briefly signaled. Discontinuation of cocaine administration and subsequent removal of the delay signals also had similar effects in both components of the multiple schedule. Taken together, these results are consistent with the view that the two types of delay signals were equally effective in maintaining responding during the variable-interval schedules.

Effects of problem difficulty and reinforcer quality on time allocated to concurrent arithmetic problems.

Students with learning difficulties participated in two studies that analyzed the effects of problem difficulty and reinforcer quality upon time allocated to two sets of arithmetic problems reinforced according to a concurrent variable-interval 30-s variable-interval 120-s schedule. In Study 1, high- and low-difficulty arithmetic problems were systematically combined with rich and lean concurrent schedules (nickels used as reinforcers) across conditions using a single-subject design. The pairing of the high-difficulty problems with the richer schedule failed to offset time allocated to that alternative. Study 2 investigated the interactive effects of problem difficulty and reinforcer quality (nickels vs. program money) upon time allocation to arithmetic problems maintained by the concurrent schedules of reinforcement. Unlike problem difficulty, the pairing of the lesser quality reinforcer (program money) with the richer schedule reduced the time allocated to that alternative. The magnitude of this effect was greatest when combined with the low-difficulty problems. These studies have important implications for a matching law analysis of asymmetrical reinforcement variables that influence time allocation.

Sensitization–habituation may occur during operant conditioning.

Operant response rates often change within experimental sessions, sometimes increasing and then decreasing. The authors attribute these changes to sensitization and habituation to aspects of the experimental situation presented repeatedly (e.g., reinforcers) or for a prolonged time (e.g., the experimental enclosure). They describe several empirical similarities between sensitization-habituation and within-session changes in operant responding. They argue that many alternative explanations for within-session changes in operant responding can be dismissed. They also examine some implications of linking the literatures on habituation and operant responding. Because responding follows a similar pattern in several other cases (e.g., human vigilance, classical conditioning, and unconditioned responding), 2 relatively simple processes may be responsible for the temporal patterning of behavior in a wide variety of situations. We observed large changes in response rate within experimental sessions when subjects (e.g., rats) responded on operant conditioning procedures (e.g., McSweeney, Hatfield, & Allen, 1990). In many of our experiments, response rates increased to a peak and then decreased. In other experiments, response rates increased without decreasing or decreased without increasing. Figure 1 contains an example of each of these types of changes. The top represents results for rats pressing levers on multiple variable interval (VI) 60-s VI 60-s schedules; the middle, the results for rats pressing levers on a VI 15-s schedule; and the bottom, the results for pigeons pecking keys on a variable ratio (VR) 15 schedule. Each graph represents the proportion of total-session responses during successive 5-min intervals in the session. Throughout this article, we calculated proportions by dividing the number of responses during a 5-min interval by the total number of responses during the session. Although within-session changes in operant responding have been observed in the past (e.g., McSweeney & Roll, 1993), these changes have been treated as problems to control by procedures, such as giving warmup trials (e.g., Hodos & Bonbright, 1972) or time to adapt to the apparatus (e.g., Papini & Overmier, 1985), rather than as phenomena to study. Further consideration suggests that within-session changes deserve study in

Within-session patterns of responding when the operandum changes during the session

Eight pigeons and 10 rats responded on variable interval 1-min or variable interval 30-s schedules. During baseline, the same operandum produced reinforcers throughout the session. During the switch conditions, performing one response produced reinforcers early in the session and performing a different response produced reinforcers later. The switch between the two responses occurred 10, 20, 30, or 40 min into the session in different conditions. Within-session patterns of responding were not usually different when the same operandum produced reinforcers throughout the session and when the operandum changed from the right to the left key (pigeons) or lever (rats). Changes from a treadle to a key (pigeons) or from a lever to a key (rats) also failed to alter within-session patterns of responding. These results suggest that factors related to the form of the instrumental response (e.g., muscular warm-up, fatigue) contributed little to within-session changes in responding under the present circumstances.

Effects of intertrial reinforcers on self-control choice.

In three experiments, pigeons chose between a small amount of food delivered after a short delay and a larger amount delivered after a longer delay. A discrete-trial adjusting-delay procedure was used to estimate indifference points--pairs of delay-amount combinations that were chosen about equally often. In Experiment 1, when additional reinforcers were available during intertrial intervals on a variable-interval schedule, preference for the smaller, more immediate reinforcer increased. Experiment 2 found that this shift in preference occurred partly because the variable-interval schedule started sooner after the smaller, more immediate reinforcer, but there was still a small shift in preference when the durations and temporal locations of the variable-interval schedules were identical for both alternatives. Experiment 3 found greater increases in preference for the smaller, more immediate reinforcer with a variable-interval 15-s schedule than with a variable-interval 90-s schedule. The results were generally consistent with a model that states that the impact of any event that follows a choice response declines according to a hyperbolic function with increasing time since the moment of choice.

A QUANTITATIVE ANALYSIS OF SENSITIVITY TO THE CONDITIONED REINFORCING VALUE OF TERMINAL‐LINK STIMULI IN A CONCURRENT‐CHAINS SCHEDULE

Pigeons were exposed to a concurrent-chains schedule in which a single variable-interval 30-s schedule was used in the initial links and fixed-time schedules were used in the terminal links. Three types of keylight conditions were used in the terminal links. In the first condition, different delays were associated with different keylight stimuli (cued condition). In the second condition, different delays were associated with the same stimulus, either a blackout (uncued blackout condition) or a white key (uncued white condition). Paired values of terminal-link fixed-time schedules differed by a constant ratio of 3:1, while the absolute value of delays was varied from 3 s to 54 s. The results showed that choice proportions for the shorter of two delays increased when the absolute size of the delays was increased for all keylight conditions. Further, the choice proportions for the shorter delay increased from the uncued blackout condition, to the uncued white condition, to the cued condition. A modified version of Fantino's (1969) delay-reduction model (expressed as a function relating the response ratio to the delay-reduction ratio) can be applied to these data by showing that sensitivity to delay reduction increased from the uncued blackout condition, to the uncued white condition, to the cued condition. Thus, the present study demonstrated that a modified version of the delay-reduction model can be used to assess quantitative differences in the terminal-link keylight condition in terms of sensitivity to delay reduction (i.e., the conditioned reinforcing value of the terminal-link keylight stimuli).

BEHAVIORAL CONTRAST FOR KEY PECKING AS A FUNCTION OF COMPONENT DURATION WHEN ONLY ONE COMPONENT VARIES

Pigeons pecked keys for food reinforcers delivered by multiple variable-interval 2-min variable-interval 2-min schedules. Positive behavioral contrast was created by changing one component to extinction; negative contrast was achieved by changing one component to a variable-interval 15-s schedule. The duration of each component was varied independently of the other from 5 to 960 s. The size of positive contrast was greatest when the extinction component was 30 or 60 s long. It did not change significantly with changes in the duration of the variable-interval 2-min component. The absolute size of negative contrast decreased with increases in the duration of the variable-interval 2-min component. It did not change significantly with changes in the duration of the variable-interval 15-s component. These results show that the size of contrast is determined primarily by the duration of the component that provides the less favorable conditions of reinforcement. These results are not predicted by current theories.

Microinjections of flupenthixol into the caudate-putamen but not the nucleus accumbens, amygdala or frontal cortex of rats produce intra-session declines in food-rewarded operant responding

Abstract Results of recent studies suggest that dopamine (DA) transmission in the caudate-putamen may be involved in food reward-related learning. The purpose of the present study was to evaluate this hypothesis by injecting the DA antagonist cis-flupenthixol (25 μg in 0.5 μl) into the dorsal caudate-putamen of rats ( n = 19) trained to lever press for food presented according to a variable interval 30-s schedule. Additional groups received non-reward ( n = 8), systemic cis-flupenthixol (0.01, 0.1 mg/kg i.p.; ns= 8), dorsal caudate-putamen injections of the inactive isomer trans-flupenthixol ( n = 10), frontal cortical (dorsal to the caudate-putamen site) injections of cis-flupenthixol ( n = 6), or cis- or trans-flupenthixol injected into the nucleus accumbens ( ns= 9, 8) or amygdala ns= 6, 5). Rats were tested in 30-min sessions and response rates were recorded every 5 min. As expected, non-reward produced a gradual decline in responding. A similar pattern was seen in the groups receiving systemic (0.1 mg/kg) or dorsal caudate-putamen injections of cis-flupenthixol. No significant effect was seen following systemic (0.01 mg/kg), cortical or amygdala cis-flupenthixol or dorsal caudate-putamen or amygdala trans-flupenthixol. Accumbens cis-flupenthixol reduced rates but did not produce a gradual decline in responding; however, accumbens trans-flupenthixol led to a time-dependent elevation in response rates making interpretation of the accumbens results difficult. It was concluded that dopaminergic projections to the dorsal caudate-putamen may play a critical role in mediating the effects of food-reward on operant responding.

Microinjections of flupenthixol into the caudate putamen of rats produce intrasession declines in food-rewarded operant responding

Results of recent studies suggest that dopamine (DA) transmission in the caudate putamen may be involved in food reward-related learning. The purpose of the present study was to evaluate the contribution of DA terminals in the dorsal caudate putamen to food-rewarded operant responding. Experiment 1, a study measuring circling behavior in 18 rats receiving systemic amphetamine (1.5 mg/kg) and unilateral intracaudate putamen injections of cis-flupenthixol (0.0, 1.0, 10.0, and 25.0 μg in 0.5 μl), a DA receptor antagonist, or its pharmacologically inactive isomer trans-flupenthixol (25.0 μg in 0.5 μl), determined a behaviourally effective dose of cis-flupenthixol. Results showed that cis-flupenthixol dose dependently increased ipsiversive turning and trans-flupenthixol did not. In Experiment 2, an operant study, 36 rats were trained to press a lever for food on a variable interval 30-s schedule. Rats were then randomly assigned to four groups, three of which received one of the following bilateral intracaudate injections prior to three subsequent test sessions: saline ( n = 6; 0.5 μl), cis-fluoenthixol ( n = 10; 25.0 μg/0.5 μl), and trans-fluoenthixol ( n = 10; 25.0 μg/0.5 μl). Rats in the home cage control group ( n = 10) received two bilateral intracaudate putamen injections of cis-flupenthixol (25.0 μg/0.5 μl) in their home cages and a final injection of cis-flupenthixol prior to a test session. The results showed that cis-flupenthixol, but not trans-flupenthixol or saline, produced a time-dependent intrasession decline in operant responding. This pattern resembled that seen in extinction. The intrasession pattern of responding in the home cage control did not differ significantly from that of the first test day but did differ significantly from that of the third test day of the cis-flupenthixol group, suggesting that the extinction-like pattern of responding in the cis-flupenthixol group was not a result of repeated central injections per se. These results provide support for the hypothesis that DA transmission may be involved in incentive learning and, further, that dopaminergic projections to the dorsal caudate putamen may play a role in food reward-related incentive learning.

PSYCHOLOGICAL DISTANCE TO REWARD: SEGMENTATION OF APERIODIC SCHEDULES OF REINFORCEMENT

College students responded for monetary rewards in two experiments on choice between differentially segmented aperiodic schedules of reinforcement. On a microcomputer, the concurrent chains were simulated as an air-defense video game in which subjects used two radars for detecting and destroying enemy aircraft. To earn more cash-exchangeable points, subjects had to shoot down as many planes as possible within a given period of time. For both experiments, access to one of two radar systems (terminal link) was controlled by a pair of independent concurrent variable-interval 60-s schedules (initial link) with a 4-s changeover delay always in effect. In Experiment 1, the appearance of an enemy aircraft in the terminal link was determined by a variable-interval (15 s or 60 s) schedule or a two-component chained variable-interval schedule of equal duration. Experiment 2 was similar to Experiment 1 except for the segmented schedule, which had three components. Subjects preferred the unsegmented schedule over its segmented counterpart in the conditions with variable-interval 60 s, and preference tended to be more pronounced with more components in the segmented schedule. These findings are compatible with those from previous studies of periodic and aperiodic schedules with pigeons or humans as subjects.

Further studies on N-methyl-1(3, 4-methylenedioxyphenyl)-2-aminopropane as a discriminative stimulus: Antagonism by 5-hydroxytryptamine 3 antagonists

Using a standard two-lever operant paradigm, male Sprague-Dawley rats were trained to discriminate 1.5 mg/kg N-methyl-1(3,4-methylenedioxyph enyl)-2-aminopropane (MDMA) from saline using a variable-interval 15-s schedule of reinforcement for food reward. Tests of stimulus antagonism were conducted to further define the mechanism of action of MDMA as a discriminative stimulus. Low doses of the 5-hydroxytryptamine 1A (5-HT 1A) antagonist NAN-190, the 5-HT 2 antagonist pirenperone, and the dopamine antagonist haloperidol were able to somewhat attenuate the MDMA stimulus; however, none of these agents decreased MDMA-appropriate responding to less than 46%. The 5-HT 3 antagonists zacopride and LY 278544 (ID 50 = 0.02 μg/kg) antagonized the MDMA discriminative stimulus. Zacopride also attenuated the stimulus effects of 1-(2,5-dimethoxy-4-methylphenyl)-2-aminopropane (DOM) in DOM-trained animals but not those of (+)amphetamine in (+)amphetamine-trained animals. Several possible mechanistic interpretations are provided but it is concluded that MDMA produces its stimulus effects via complex mechanism involving both dopaminergic and serotonergic components.

Key pecking of pigeons under variable-interval schedules of briefly signaled delayed reinforcement: effects of variable-interval value.

Key pecking of 4 pigeons was maintained under a multiple variable-interval 20-s variable-interval 120-s schedule of food reinforcement. When rates of key pecking were stable, a 5-s unsignaled, nonresetting delay to reinforcement separated the first peck after an interval elapsed from reinforcement in both components. Rates of pecking decreased substantially in both components. When rates were stable, the situation was changed such that the peck that began the 5-s delay also changed the color of the keylight for 0.5 s (i.e., the delay was briefly signaled). Rates increased to near-immediate reinforcement levels. In subsequent conditions, delays of 10 and 20 s, still briefly signaled, were tested. Although rates of key pecking during the component with the variable-interval 120-s schedule did not change appreciably across conditions, rates during the variable-interval 20-s component decreased greatly in 1 pigeon at the 10-s delay and decreased in all pigeons at the 20-s delay. In a control condition, the variable-interval 20-s schedule with 20-s delays was changed to a variable-interval 35-s schedule with 5-s delays, thus equating nominal rates of reinforcement. Rates of pecking increased to baseline levels. Rates of pecking, then, depended on the value of the briefly signaled delay relative to the programmed interfood times, rather than on the absolute delay value. These results are discussed in terms of similar findings in the literature on conditioned reinforcement, delayed matching to sample, and classical conditioning.

EFFECTS OF REINFORCER RATE AND REINFORCER QUALITY ON TIME ALLOCATION: EXTENSIONS OF MATCHING THEORY TO EDUCATIONAL SETTINGS

We examined how 3 special education students allocated their responding across two concurrently available tasks associated with unequal rates and equal versus unequal qualities of reinforcement. The students completed math problems from two alternative sets on concurrent variable-interval (VI) 30-s VI 120-s schedules of reinforcement. During the equal-quality reinforcer condition, high-quality (nickels) and low-quality items ("program money" in the school's token economy) were alternated across sessions as the reinforcer for both sets of problems. During the unequal-quality reinforcer condition, the low-quality reinforcer was used for the set of problems on the VI 30-s schedule, and the high-quality reinforcer was used for the set of problems on the VI 120-s schedule. Equal- and unequal-quality reinforcer conditions were alternated using a reversal design. Results showed that sensitivity to the features of the VI reinforcement schedules developed only after the reinforcement intervals were signaled through countdown timers. Thereafter, when reinforcer quality was equal, the time allocated to concurrent response alternatives was approximately proportional to obtained reinforcement, as predicted by the matching law. However the matching relation was disrupted when, as occurs in most natural choice situations, the quality of the reinforcers differed across the response options.

The effect of signalled reinforcement on a synthetic VI schedule

The present experiment investigated whether a signal for reinforcement would facilitate the acquisition of a required response sequence, despite enhanced acquisition of the sequence resulting in a lower response rate. Two groups of hungry rats responded on a synthetic variable interval (VI) schedule for food reinforcement. On this schedule, reinforcement was available for the first response to the target lever after a VI 60-s schedule had elapsed. However, reinforcement was also made contingent on a second criterion, which required that the subject emit a specified number of responses to a second lever prior to the response to the target lever. Rats that received a brief tone simultaneously with the delivery of reinforcement performed the task requirements more accurately than rats lacking a reinforcement signal. This enhanced performance of the required response sequence was accompanied by a lower rate of response to the target lever. This finding provides support for the notion that reinforcement signals on a conventional VI schedule attenuate responding due to a more “accurately” structured emission of the response that precedes reward.

Effects of diazepam and chlorpromazine on socially induced anxiety in pigeons

The effects of diazepam and chlorpromazine on response suppression in a social situation were studied in pigeons. Three groups of pigeons were trained to peck a key on a variable-interval 60-s schedule of reinforcement, then exposed to the pain reaction of adjoining pigeon to electric shock. Although every pigeon showed suppression of response, the suppression decreased with repeated exposures. A conditioning group received the electric shock with the exposure to the pain reaction of adjoining bird; a shock exposure group received the electric shock without any axplicit conditioned stimulus; and a control group did not receive any shock. After these treatments every group was exposed to the pain reaction of the adjoining bird. The conditioning group and the shock exposure group showed clear response suppression, but the control group did not. Although chlorpromazine generally reduced response rate in all groups, diazepam selectively abolished the response suppression.