Random-interval Reinforcement Research Articles

A discriminated rapid-acquisition laboratory procedure for human continuous choice.

Previous continuous choice laboratory procedures for human participants are either prohibitively time-intensive or result in inadequate fits of the generalized matching law (GML). We developed a rapid-acquisition laboratory procedure (Procedure for Rapidly Establishing Steady-State Behavior, or PRESS-B) for studying human continuous choice that reduces participant burden and produces data that is well-described by the GML. To test the procedure, 27 human participants were exposed to 9 independent concurrent random-interval random-interval reinforcement schedules over the course of a single, 37-min session. Fits of the GML to the participants' data accounted for large proportions of variance (median R2 : 0.94), with parameter estimates that were similar to those previously found in human continuous choice studies [median a: 0.67; median log(b): -0.02]. In summary, PRESS-B generates human continuous choice behavior in the laboratory that conforms to the GML with limited experimental duration.

Saccadic latency and choice in a concurrent random interval reinforcement schedule.

Conventional decision models view reaction time as a consequence of the duration of decision-making process. However, some studies have shown that reaction time distributions may be strongly affected by reinforcement contingencies (Madelain et al., 2007). Here, we probe the possibility to voluntarily control saccadic latencies in a choice paradigm. Three subjects (including the two authors) tracked a visual target stepping horizontally by 10 deg between two fixed locations on a screen. Any trial with saccadic latencies greater than 300 ms or shorter than 80 ms was interrupted. Using the first and last quartiles of individual baseline latency distributions, we first defined two classes, i.e. short and long saccadic latencies, respectively. We then concurrently reinforced each type of latencies in random interval reinforcement schedules: short and long latencies were reinforced with three different sets of probabilities such that the relative ratio of reinforcing short latencies was either 9/1, 1/9 or 1/1. After training (20 800 trials), we observed bimodal latency distributions –with a peak for short and another for long latencies– for two subjects and to a lesser extent for the third subject. To further probe the extent of control over saccadic latencies, we then analyzed the data using the generalized matching law (Baum, 1974) –which states that the relative proportion of choices made to an option matches the relative proportion of reinforcers earned from that option. We found an almost perfect match between the relative proportion of short and long latencies and the relative obtained reinforcement for two subjects (sensitivities were equal to 0.95 and 0.87) and typical undermatching parameter for the third one (sensitivity = 0.58). These results indicate that saccades may be allocated in time following the reinforcement contingencies in force, which support the idea of a voluntary control of saccadic reaction time.

A quantitative analysis of the behavior maintained by delayed reinforcers.

Random-interval reinforcement was arranged for a sequence of pigeon first-key pecks followed by second-key pecks. First-key pecks, separated from reinforcers by delays that included number of second-key pecks and time, decreased in rate as delays increased. Delay functions, or gradients, were obtained in one experiment with reinforced sequences consisting of M first-key pecks followed by N second-key pecks (M + N = 16), in a second where required first-key pecks were held constant (M = 8), and in a third where minimum delay between most recent first-key pecks and reinforcers varied. In each, gradients were equally well fitted by exponential, hyperbolic and logarithmic functions. Performances were insensitive to reinforcer duration and functions were consistent across varied random-interval values. In one more experiment, time and number delays were independently varied using differential reinforcement of rate of second-key pecks. Delay gradients depended primarily on time rather than on number of second-key pecks. Thus, reinforcers have effects based on earlier responses, not just the ones that produced them, with the contribution of each response weighted by the time separating it from the reinforcer rather than by intervening behavior. Situations where unwanted responses (e.g., errors) often precede reinforced corrects can maintain them unless designed to avoid such effects of delay.

Contextual control of instrumental actions and habits.

After a relatively small amount of training, instrumental behavior is thought to be an action under the control of the motivational status of its goal or reinforcer. After more extended training, behavior can become habitual and insensitive to changes in reinforcer value. Recently, instrumental responding has been shown to weaken when tested outside of the training context. The present experiments compared the sensitivity of instrumental responding in rats with a context switch after training procedures that might differentially generate actions or habits. In Experiment 1, lever pressing was decremented in a new context after either short, medium, or long periods of training on either random-ratio or yoked random-interval reinforcement schedules. Experiment 2 found that more minimally trained responding was also sensitive to a context switch. Moreover, Experiment 3 showed that when the goal-directed component of responding was removed by devaluing the reinforcer, the residual responding that remained was still sensitive to the change of context. Goal-directed responding, in contrast, transferred across contexts. Experiment 4 then found that after extensive training, a habit that was insensitive to reinforcer devaluation was still decremented by a context switch. Overall, the results suggest that a context switch primarily influences instrumental habit rather than action. In addition, even a response that has received relatively minimal training may have a habit component that is insensitive to reinforcer devaluation but sensitive to the effects of a context switch.

Prolonged Behavioral Effects of in Utero Exposure to Lead or Methyl Mercury: Reduced Sensitivity to Changes in Reinforcement Contingencies during Behavioral Transitions and in Steady State

Postnatal exposure to lead or methyl mercury results in mental retardation, learning deficits, and other neurobehavioral effects in humans, and adverse consequences of prenatal exposure have been clearly documented with methyl mercury. To examine the developmental neurotoxicity of these metals, especially lead, concurrent schedules of food reinforcement were used to identify learning deficits in squirrel monkeys exposed during gestation to either methyl mercury or lead. Pregnant squirrel monkeys were administered methyl mercury (0.7 to 0.9 ppm in maternal blood) or lead (21 to 79 μg/dl in maternal blood) during the last half to two-thirds of gestation. At about 5-6 years of age, offspring were trained to lever press under concurrent schedules of reinforcement in which separate random interval reinforcement schedules operated independently on two levers. Reinforcement densities were varied such that 20 to 90% of the reinforcers were programmed to derive from the left lever (i.e., one lever was "richer" than the other). At steady state, the behavior of the controls was sensitive to reinforcement density and showed little lever bias, but the behavior of monkeys exposed to more than 40 μg/dl of lead and to methyl mercury was less sensitive to reinforcement rates and heavily biased. When relative reinforcement density on a lever changed, the unexposed animals′ response rates gradually shifted to the newly rich lever. The behavior of monkeys exposed to methyl mercury or more than 40 μg/dl of lead changed slowly, not at all, or in the wrong direction. Steady-state behavior of monkeys exposed to less than 40 μg/dl resembled controls, but acquisition progressed more slowly and required 2-4 times as many reinforcers to complete. These effects suggest a behavioral mechanism-insensitivity to changing reinforcement contingencies-by which learning deficits and behavioral changes associated with these metals might be related to toxicant exposure. Since maternal blood levels corresponded to those that could be experienced in occupational settings, the present data raise the possibility of fetal hazards associated with maternal lead exposures at levels tolerated in humans in occupational settings.

A matching law analysis of the effects of dopamine receptor antagonists

Herrnstein's matching equation was used to analyze drug effects on performance in random interval reinforcement schedules. Pimozide caused effects compatible with both motor and motivational impairments, in a 5-component multiple schedule, a 3-schedule 3-day cycle (ALT-3), and a 2-schedule 2-day cycle (ALT-2). However, at low doses, both sulpiride and SCH-23390, tested in the ALT-3 and ALT-2 procedures, caused effects compatible with selective motivational impairments. In experiments using the non-multiple schedules, motivational effects increased during the course of the experimental session, under all three drugs. The interpretation of "motor" and "motivational" deficits in the ALT-2 procedure was validated by experiments in which the response-force and deprivation level were systematically varied. The results support the view that dopamine may be involved in the maintenance of rewarded behaviour, but not differentially implicate the D1 or the D2 receptor subtype.

Variability of force and interresponse time under random interval reinforcement schedules

Response variability was examined under five random interval schedules with minimum mean interreinforcement intervals of 15, 30, 60, 120, and 240 sec. Distributions of response force-time integrals showed no systematic variability changes with increases in reinforcement intermittency. Corresponding distributions of interresponse times revealed increases in variability accompanying increases in reinforcement intermittency. The relationship of variability to these two measures, and the potential effect of behavior outside the experimentally measured response class, on recorded variability within the class, are considered.

PROPERTIES OF BEHAVIOR UNDER RANDOM INTERVAL REINFORCEMENT SCHEDULES.

In a temporally defined system of reinforcement schedules, the fixed interval case is defined when reinforcement probability, P, is equal to unity for the first response in any cycle length, T; when P is less than 1.0, random interval schedules emerge wherein T/P specifies the expected interval between reinforcements. Key-pecking rates were found to be: (a) inversely related to T/P; (b) higher at T=1.0 second than at other T parameter values; (c) low and linear at several T and T/P values. The mean post-reinforcement pause, if initially small, increased, and if initially large, decreased, as T/P increased.