Response-reinforcer Contiguity Research Articles

Interval, blocking, and marking effects during the development of schedule-induced drinking in rats.

Schedule-induced drinking (SID) can occur when food-deprived rats are given access to water while receiving pellets on an intermittent reinforcement schedule. These conditions can increase water intake excessively. The possible role of adventitious reinforcement of postpellet drinking was assessed by testing whether response-reinforcer contiguity, the relative predictiveness of a response, and whether it is marked are important in the development of SID. Rats exposed to a short interpellet interval acquired SID most rapidly, with this acquired drinking response maintained when animals were transferred to a longer interpellet interval, thus indicating an easy-to-hard effect (Experiment 1). Further experiments demonstrated that a stimulus (a brief-flashing house light) occurring prior to pellet delivery could block the acquisition of SID (Experiment 2), while a lick-contingent tone, intended to increase the associability of this response, produced more rapid acquisition of SID (Experiment 3). Analysis of lick distributions revealed that licking became concentrated in the first half of an interpellet interval only after several sessions. Overall, the results indicated that similar factors affect the acquisition of both SID and instrumental conditioning with delayed reinforcement, as is consistent with a superstitious conditioning account of SID development.

Positive Reinforcement as a Correlation Between Lever Pressing and Inerease in Frequency of food Pellet Delivery in rats

Lever pressing in food-deprived rats was acquired when it increased the probability of food pellet delivery even when a pellet could not follow a response by Iess thai 10 seconds. The probabiity of acquisition was decreased by increasing the frequenCy of pellet delivery in the absence of responding. Response rates were higher when sorne responses were immediately followed by a pellet but were Iow when pellets were delivered independently of responding. The results support te notion that a Colfelation between responding and positive teinforcer delivery. iii te sense that responding increases the frequency of reinforcer delivery, is suffkient to produce response acquisition in te absence of response-reinforcer contiguity.

Effects of free food deliveries and temporal contiguity on choice under concurrent-chain schedules

Eight pigeons responded in a concurrent chain with variable-interval (VI) 10-sec and VI 20-sec terminal links. Free food deliveries were then added to the initial links according to a variable-time (VT) 20-sec schedule in two conditions that differed in terms of whether a differential reinforcement of other (DRO) contingency was also arranged, which ensured that those deliveries could not occur within 2-sec of a response. Preference for the VI 10-sec terminal link increased when VT food was added, but not when the DRO contingency was operative, showing that free food deliveries affected preference only when those deliveries could be temporally contiguous with choice responding. This finding suggests that Mazur's (2003) report of increased preference with added VT food, replicated here, was due to adventitious reinforcement. Current models for behavioral choice are limited because they are based entirely on temporal relations between stimuli and reinforcers and fail to take into account response-reinforcer contiguity.

ACQUISITION OF COCAINE SELF-ADMINISTRATION WITH UNSIGNALED DELAYED REINFORCEMENT IN RHESUS MONKEYS

Six experimentally naive rhesus monkeys produced 0.01 mg/kg/infusion cocaine by lever pressing under a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior schedule. One lever press initiated an unsignaled 15- or 30-s delay culminating in cocaine delivery. Each press made during the delay reset the delay interval. With two exceptions, responding was acquired and maintained at higher rates than responding on a second (inoperative) lever. For the exceptions, a cancellation contingency was arranged in which each formerly inoperative-lever response reset the tandem schedule. This manipulation reduced presses on the inoperative lever. Subsequently, the consequences of responding on the two levers were reversed, and the monkeys again responded at higher rates on the operative lever. As a comparison, 3 additional experimentally naive monkeys received response-independent cocaine deliveries. Although lever pressing was observed, it extinguished and was subsequently reestablished under the tandem schedule. The results suggest that although response-reinforcer contiguity is not required for cocaine to acquire reinforcing functions, a response-reinforcer relation appears necessary.

Human time-based schedule performance: Exploring interactions between instructions and response-reinforcer contiguity.

Spontaneous regulation of human behavior on a simple time-based mixed schedule of reinforcement was studied as a function of instructions and response-reinforcer contiguity. Seventeen subjects were exposed to a mixed fixed-interval (FI) 10 s schedule (Condition A) and a conjunctive fixed-time 10 s fixed ratio 1 (Condition B) schedule of reinforcement using either an ABAB or a BABA design across 5 sessions. One of three instructional categories were employed (normal, ratio, and interval instructions) on a simple button pressing task. Subjects either produced a high rate of responding with short postreinforcement pauses (PRPs), a high rate of responding decreasing to a low rate with long PRPs, or a low rate of responding with long PRPs. Evidence of spontaneous regulation in terms of consistent differential responding across components was displayed only under interval instructions. These results highlight the interaction of instructions with the role of response-reinforcer contiguity in controlling spontaneous regulation of human performance on simple time-based schedules of reinforcement. ********** Typical human performance on a fixed-interval (FI) schedule of reinforcement is commonly described as either a high rate performance with a small postreinforcement pause (PRP), or a low rate performance with a long PRP (e.g., Leander, Lippman & Meyer, 1968; Lippman & Meyer, 1967; Weiner, 1969). These findings differ from nonhuman performance on the FI where a scallop or break-and-run is the typical pattern (Baron & Leinenweber, 1994; Hyten & Madden, 1993; Perone, Galizio & Baron, 1988). Attempts to explain the discrepant findings between human and nonhumans, and the differences in findings between humans, have lead to the development of a number of research areas. These include studies concerned with the effect of the experimental setting (Barnes & Keenan, 1989; Orne, 1962), the effect of conditioning histories (e.g., Weiner, 1969, 1972), and the determinants of choice (e.g., Belke, Pierce, & Powell, 1989; Darcheville, River6re, & Wearden, 1993). One of the defining features of human behavior, the complexity of the verbal repertoire, also has drawn special attention in the study of schedules of reinforcement. Many studies have investigated verbal behavior by focusing on the role of either experimenter-generated or self-generated instructions on schedules of reinforcement (e.g., Baron & Galizio, 1983; Galizio, 1979; Hayes, Brownstein, Zettle, Rosenfarb, & Korn, 1986; LeFrangois, Chase, & Joyce, 1988; Matthews, Shirnoff, Catania, & Sagvolden, 1977). Conclusions to date highlight the power of instructions in generating responding which may be insensitive to the programmed contingencies (Baron & Galizio; for a more recent analysis see Madden, Chase & Joyce, 1998). Some researchers have also pointed to the role of self-generated instructions in determining behavior (Bentall & Lowe, 1987; Lowe, 1979). Before we can fully understand the role played by verbal behavior in human schedule performance we must have a clear understanding of how the formal structure of a schedule of reinforcement critically determines the dynamics involved in behavioral adaptation. To date most research investigating the interaction between instructions and programmed contingencies have employed schedules simply as tools for generating behavior. The problem with this is that we do not fully understand how the tools themselves function. That is, we are limited in the extent to which we understand schedules as contexts for controlling dynamic processes in the multiple determination of behavior (Keenan & Toal, 1991). If we are limited in our understanding of these fundamental issues, the addition of an another layer of complexity (verbal behavior) makes our task more difficult. A detailed analysis of the workings of simple schedules of reinforcement is therefore necessary. …

PAVLOVIAN CONTINGENCIES AND RESISTANCE TO CHANGE IN A MULTIPLE SCHEDULE

According to theoretical accounts of behavioral momentum, the Pavlovian stimulus—reinforcer contingency determines resistance to change. To assess this prediction, 8 pigeons were exposed to an unsignaled delay‐of‐reinforcement schedule (a tandem variable‐interval fixed‐time schedule), a signaled delay‐of‐reinforcement schedule (a chain variable‐interval fixed‐time schedule), and an immediate, zero‐delay schedule of reinforcement in a three‐component multiple schedule. The unsignaled delay and signaled delay schedules employed equal fixed‐time delays, with the only difference being a stimulus change in the signaled delay schedule. Overall rates of reinforcement were equated for the three schedules. The Pavlovian contingency was identical for the unsignaled and immediate schedules, and response—reinforcer contiguity was degraded for the unsignaled schedule. Results from two disruption procedures (prefeeding subjects prior to experimental sessions and adding a variable‐time schedule to timeout periods separating baseline components) demonstrated that response—reinforcer contiguity does play a role in determining resistance to change. The results from the extinction manipulation were not as clear. Responding in the unsignaled delay component was consistently less resistant to change than was responding in both the immediate and presignaled segments of the signaled delay components, contrary to the view that Pavlovian contingencies determine resistance to change. Probe tests further supported the resistance‐to‐change results, indicating consistency between resistance to change and preference, both of which are putative measures of response strength.

Periodic Response-Reinforcer Contiguity: Temporal Control but not as We Know It!

Two experiments using rats show the effects of introducing response-reinforcer contiguity on a modified recycling conjunctive fixed-time 30 s fixed-ratio 1 schedule of food reinforcement. In Experiment 1, the frequency of programmed contiguity was varied across conditions. The general finding was an increase in overall rate of responding for all rats when the incidence of obtained cont iguity was relatively high. This increase in responding was accompanied by unusual response patterns for some rats. These patterns persisted when the rats were subsequently exposed to a fixed-interval schedule. Other rats produced fixed-interval-like performance when the incidence of obtained contiguity increased. Similar findings were observed in Experiment 2 where a more direct comparison was made between the modified recycling conjunctive schedule and the fixed-interval schedule. Results in general emphasize the importance of the modified recycling conjunctive schedule as a tool for exploring behavioral adaptation to periodic reinforcement. The history effects reported in both experiments are discussed with reference to the notion of schedule-induced behavior and temporal control.

Periodic Reinforcement and Second-Order Schedules

In two experiments, rats were trained initially on a recycling conjunctive schedule in which a food pellet was delivered after 30 s provided at least one response had occurred; otherwise the next cycle began immediately. This produced low rate responding characterized by either a pause-respond-pause pattern or else a pause-respond pattern. The schedule then was changed so that half of the intervals ended with the presentation of a brief stimulus instead of food. Patterning after food was little affected, whereas patterning after the brief stimulus varied across rats. Generally, a short pause after the brief stimulus was followed by an initial increase in responding that led to either a fairly constant rate, or else to a decrease in rate throughout the interval. In later conditions, when the incidence of response-food and response- stimulus contiguity were manipulated separately, only the former increased response rate; this was so even when the brief stimulus was paired with food in some conditions. Rate increases were accompanied by changes in patterning across all intervals. These results do not support a simple conditioned reinforcement interpretation of the control acquired by a brief stimulus on a second-order schedule with fixed-interval components. Rather, they suggest that a number of interrelated variables combine to maintain responding. These variables include response-reinforcer contiguity, the temporal location of the response dependency, and the contingency between the brief stimulus and the reinforcer.

EFFECTS OF ALTERNATIVE REINFORCEMENT SOURCES: A REEVALUATION

The effects of two alternative sources of food delivery on the key-peck responding of pigeons were examined. Pecking was maintained by a variable-interval 3-min schedule. In the presence of this schedule in different conditions, either a variable-time 3-min schedule delivering food independently of responding or an equivalent schedule that required a minimum 2-s pause between a key peck and food delivery (a differential-reinforcement-of-other-behavior schedule) was added. The differential-reinforcement-of-other-behavior schedule reduced response rates more than did the variable-time schedule in most instances. The delay between a key peck and the next reinforcer consistently was longer under the differential-reinforcement-of-other-behavior schedule than under the variable-time schedule. Response rates and median delay between responses and reinforcers were negatively correlated. These results contradict earlier conclusions about the behavioral effects of alternative reinforcement. They suggest that an interpretation in terms of response-reinforcer contiguity is consistent with the data.

VARYING RESPONSE‐REINFORCER CONTIGUITY IN A RECYCLING CONJUNCTIVE SCHEDULE

Three experiments describe the effects of manipulating the frequency of response-reinforcer contiguity in a recycling conjunctive schedule. The schedule arranged that a reinforcer was delivered after 30 s provided at least one response had occurred; otherwise the next cycle started immediately. In Experiment 1, this schedule produced the typical pause-respond-pause pattern, with most responses at mid-interval; and, when a limited number of contiguities between responses and food delivery were added, the pattern became more like the monotonic scallop seen on fixed-interval schedules. In Experiment 2, the schedule was initially presented with an additional contingency that allowed contiguity on every trial. Fixed-interval-like behavior occurred and tended to persist as contiguities were gradually eliminated. In Experiment 3, the recycling conjunctive schedule alternated with a condition in which a large number of contiguities occurred. The pause-respond-pause pattern and fixed-interval-like performance occurred with few or many contiguities, respectively. The results of all three experiments illustrate how contiguity interacts with a small number of other variables to determine performance on interval schedules and illuminate previous findings with fixed-interval and fixed-time schedules.

SEPARATING RESPONSE DEPENDENCY AND RESPONSE‐REINFORCER CONTIGUITY WITHIN A RECYCLING CONJUNCTIVE SCHEDULE

A procedure is described which disrupts response-reinforcer contiguity and response dependency and which demonstrates how the location of the response dependency in interval schedules can be regarded as a controlling variable in its own right. Rats' lever pressing produced sucrose on a recycling conjunctive fixed-time 30-second fixed-ratio 1 schedule of reinforcement. Reinforcement occurred only at the end of the fixed-time component on this schedule and only if a response had occurred during that component. This produced a pause-respond-pause pattern during the interreinforcer interval for all animals. When the location of the response dependency was then restricted to a 10-second period in the middle of the fixed-time component, the pattern was accentuated and response rates increased for all animals, while postreinforcement pauses decreased sharply for two animals. When responding was required in the first 10 seconds of the fixed-time component, responding peaked earlier in the interval for all animals. Response rates were slightly below those in the previous conditions, while postreinforcement pauses were between 2 and 6 seconds across animals.

Contiguity and contingency in the acquisition and maintenance of an operant

Three experiments compared the effects of nondifferential and differential reinforcement of response location on a circular dimension. Rats were required to operate a vertical joystick to produce food. When food was delivered immediately after responses, but independent of response location, the spatial concentration of responding was low and no progressive changes were observed. Traditional and percentile schedules of differential reinforcement for response location produced highly reliable acquisition of spatially concentrated responding. Once concentrated responding had been established, nondifferential reinforcement was sufficient to maintain it in some subjects. Since only the differential reinforcement schedules established a contingency with respect to response location, it was concluded that this relationship was necessary for acquisition, but that response-reinforcer contiguity may be sufficient for maintenance. This conclusion is consistent with the view that operant conditioning is a contiguity-based process, but that contingencies are required to produce reliable contiguity between reinforcers and particular responses.

Classical conditioning of proboscis extension in honeybees (Apis mellifera).

Extension of the proboscis was conditioned in restrained honeybees with odor as the conditioned stimulus (CS) and sucrose solution--delivered to the antenna (to elicit extension of the proboscis) and then to the proboscis itself--as the unconditioned stimulus (US). In a first series of experiments, acquisition was found to be very rapid, both in massed and in spaced trials; its associative basis was established by differential conditioning and by an explicitly unpaired control procedure (which produced marked resistance to acquisition in subsequent paired training); and both extinction and spontaneous recovery in massed trials were demonstrated. In a series of experiments on the nature of the US, eliminating the proboscis component was found to lower the asymptotic level of performance, whereas eliminating the antennal component was without effect; reducing the concentration of sucrose from 20% to 7% slowed acquisition but did not lower the asymptotic level of performance; and second-order conditioning was demonstrated. In a series of experiments on the role of the US, an omission contingency designed to eliminate adventitious response-reinforcer contiguity was found to have no adverse effect on acquisition. In a series of experiments designed to analyze the resistance to acquisition found after explicitly unpaired training in the first experiments, no significant effect was found of prior exposure either to the CS alone or to the US alone, although the unpaired procedure again produced substantial resistance that was shown to be due to inhibition rather than to inattention; extinction after paired training was found to be facilitated by unpaired presentations of the US. The relation between these results for honeybees and those of analogous experiments with vertebrates is considered.

Response-reinforcer relations and the hippocampus

The operant performance of intact rats is guided by response-reinforcer contingency but that of animals with hippocampal lesions is not. Despite this impairment, lesioned animals perform a number of tasks without difficulty. This experiment examines the possibility that response-reinforcer contiguity is the relationship upon which the operant performance of hippocampals is exclusively based. This hypothesis is tested in a yoked-control design where pellets are dispensed freely and responses are without effect on pellet delivery rate. The temporal patterning of deliveries is rearranged such that they become unpaired, paired, or random with respect to lever presses. Thus, contiguity is manipulated independently of contingency. The behavior of rats with hippocampal lesions was robustly superstitious, and contiguity effects were strong. Pairing channeled superstition into the lever press topography, while noncontiguity displaced the activities away from the lever. Random delivery resulted in intermediate rates of lever pressing.

The rat's resistance to superstition: role of the hippocampus.

Superstitious operant behavior, in all possible respects similar to that described for pigeons, was found to characterize the behavior of male rats with bilateral hippocampal lesions (hippocampals). Robust superfluous activitiess accompanied noncontingent (random time) and shifts from contingent (random interval) to noncontingent pellet delivery. These activities were minimized but not eliminated by an operant contingency. In contrast, sham-operate performance was distinguished by a stripping away of superfluities in all contingency conditions. Although a variety of responses were emitted superstitiously, the topography of behavior in hippocampal rats was not different from that of sham operates. Unlike sham operates, which varied their behavior spontaneously and in response to experimental changes, hippocampals did not cease their repetitive, high frequency sequences until reinforcers were withheld. As the detection of a free operant contingency seems to depend upon response-cued discrimination learning, and in view of the hippocampal failure to generate these cues, a link between superstitious response output and response rigidity is hypothesized. It is proposed that the hippocampus permits the control of behavior by contingency and that without the structure, operant behavior is guided by simple response-reinforcer contiguity.

Analysis of autoshaping in goldfish

Stimulus-reinforcer and response-reinforcer control of target-striking in goldfish were studied in a series of discrete-trials experiments. Evidence of control by adventitious response-reinforcer contiguity was provided by the first experiment, which showed less response in animals given omission training than in yoked animals with the same stimulus-reinforcer experience. In the next three experiments, evidence of control by stimulus-reinforcer contiguity apart from response-reinforcer contiguity was sought in within-subjects comparisons of omission and extinction. Only the last of these experiments, in which the CS duration was short and the ITI long, showed greater response in omission. A subsidiary finding is that autoshaped goldfish respond very little, either to the CS target or to the feeder, when target and feeder are spatially discontiguous.

CONDITIONING OF TWO‐RESPONSE PATTERNS OF KEY PECKING IN PIGEONS1

On discrete trials, two response keys were made available to hungry pigeons and food reinforcement depended on the order in which the required two key pecks occurred. In different phases, only one of the four possible two-peck sequences (left-left, left-right, right-left, and right-right) produced food reinforcement. In each case, the pigeons learned to perform the correct two-peck sequence more often than the incorrect sequences. Furthermore, the course of differentiation mastery indicated that both reinforcement history and response-reinforcer contiguity influenced performance. These results reveal that response patterns comprising two instances of the same response left-left and right-right) or instances of two different responses (left-right and right-left) may function as operants, thereby extending the generality of conditioning principles from discrete responses to structured sequences of behavior. These and other results are discussed in terms of contiguity-based and memory-based models of learning.

CONDITIONING OF WITHIN‐TRIAL PATTERNS OF KEY PECKING IN PIGEONS1

The possibility of conditioning systematic patterns of responding during brief discrete trials was studied by requiring hungry pigeons to key peck and then pause or to pause and then key peck in order to gain access to food. These schedules were highly effective in promoting decelerated and accelerated rates of responding, respectively, within individual trials; indeed, performance was quite similar to that observed when explicit external stimuli were correlated with "peck" and "pause" portions of the daily trials. Finally, schedules of reinforcement that did not selectively reinforce peck-pause or pause-peck patterns neither generated these patterns nor maintained them at the previous high levels. The results, therefore, confirm Shimp's (1976) proposal that organized groupings of discrete responses may function as operants-even in the absence of strict response-reinforcer contiguity.

RATE AND TEMPORAL PATTERN OF KEY PECKING UNDER AUTOSHAPING AND OMISSION SCHEDULES OF REINFORCEMENT1

The role of response-reinforcer contiguity on autoshaped key pecking in pigeons was studied by scheduling response-dependent nonreinforcement at the beginning or the end of brief (8-sec) discrete trials. Schedules that permitted chance conjunctions of key pecking and food sustained high rates of responding, whereas those that prevented the occurrence of key peck-food intervals shorter than 4 sec sustained low response rates. In addition, selective reinforcement schedules supported accelerating or decelerating rates of responding within individual trials. These effects were traceable to response-reinforcer (operant), but not stimulus-reinforcer (respondent) factors.

CONJUNCTIVE SCHEDULES OF RESPONSE‐DEPENDENT AND RESPONSE‐INDEPENDENT REINFORCEMENT1

Pigeons received food when they emitted the number of responses specified by a fixed-ratio schedule, and the time specified by a fixed-time schedule had elapsed. The order of meeting the response and time requirements was irrelevant. In different conditions, stimuli signalled completion of one, both, or neither requirement. Ratio size interacted with stimulus condition to determine performance. When a stimulus signalled the end of the fixed-time period, under all ratios the birds tended to respond after the stimulus appeared. When stimuli followed both components, small ratios produced responding during the fixed-time period, and other ratios resulted in responses after the time period had elapsed. With either no stimulus changes, or with a stimulus correlated with completion of the ratio alone, responding first increased and then decreased as the ratio increased. Low and high ratios produced stable response frequencies and patterns in successive intervals. Intermediate ratios resulted in two types of performance. Intervals with long initial pauses and few responses during the fixed-time period were followed by intervals with short pauses and numerous responses and vice versa. The source of these dynamic effects was hypothesized to be number of responses per reinforcer in one condition and response-reinforcer contiguity in the other.