Obtained Reinforcement Rate Research Articles

Evolutionary theory prediction: Response rate as a joint function of reinforcement rate and reinforcer magnitude.

Artificial organisms (AOs) animated by an evolutionary theory of behavior dynamics (ETBD) worked on concurrent interval schedules with a standard reinforcer magnitude on 1 alternative and a range of reinforcer magnitudes on the other. The reinforcer magnitudes on the second alternative were hedonically scaled using the generalized matching law. The AOs then worked on single interval schedules that arranged various combinations of the scaled reinforcer magnitudes and a range of nominal schedule values. This produced bivariate response rate data to which 5 candidate equations were fitted. One equation was found to provide the best description of the bivariate data in terms of percentage of variance accounted for, information criterion value, and residual profile. This equation consisted of 2 factors, 1 entailing the scaled magnitude, 1 entailing the obtained reinforcement rate, and both expressed in the form of exponentiated hyperbolas. The theory's prediction of the bivariate equation, along with additional predictions of the theory, were tested on data from an experiment in which rats pressed levers for various concentrations of sucrose pellets. The bivariate equation predicted by the theory was confirmed, as were all the additional predictions of the theory that could be tested on this data set.

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference.

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.

RESIDENCE TIME AND CHOICE IN CONCURRENT FORAGING SCHEDULES

Five pigeons were trained on a concurrent-schedule analogue of the "some patches are empty" procedure. Two concurrently available alternatives were arranged on a single response key and were signaled by red and green keylights. A subject could travel between these alternatives by responding on a second yellow "switching" key. Following a changeover to a patch, there was a probability (p) that a single reinforcer would be available on that alternative for a response after a time determined by the value of lambda, a probability of reinforcement per second. The overall scheduling of reinforcers on the two alternatives was arranged nonindependently, and the available alternative was switched after each reinforcer. In Part 1 of the experiment, the probabilities of reinforcement, rho(red) and rho(green), were equal on the two alternatives, and the arranged arrival rates of reinforcers, lambda(red) and lambda(green), were varied across conditions. In Part 2, the reinforcer arrival times were arranged to be equal, and the reinforcer probabilities were varied across conditions. In Part 3, both parameters were varied. The results replicated those seen in studies that have investigated time allocation in a single patch: Both response and time allocation to an alternative increased with decreasing values of lambda and with increasing values of rho, and residence times were consistently greater than those that would maximize obtained reinforcer rates. Furthermore, both response- and time-allocation ratios undermatched mean reinforcer-arrival time and reinforcer-frequency ratios.

CONCURRENT‐SCHEDULE PERFORMANCE IN DAIRY COWS: PERSISTENT UNDERMATCHING

Performance of dairy cows responding under concurrent variable-interval variable-interval schedules of food delivery was examined, with results analyzed in terms of the generalized matching equation. In Experiment 1, bias measures indicated that crushed barley was preferred over meatmeal when these foods were available under the alternative schedules. For whole-session data, substantial undermatching of response and time-allocation ratios to obtained reinforcement ratios was evident. Postreinforcement pause time ratios approximately matched obtained reinforcement rates. Subtracting these times from total time-allocation values yielded net time-allocation ratios that undermatched obtained reinforcement ratios to a greater degree than did whole-session time-allocation ratios. In Experiment 2, substantial undermatching was evident when the same foods (hay for 2 cows, crushed barley for 2 others) were available under the alternative schedules. Food-related activities and other defined behavior not related to food were quantified by direct observation, and were found to occupy a substantial proportion (roughly 40% to 80%) of experimental sessions. Subtracting the time spent in these activities from the time allocated to each component schedule did not reduce the degree of undermatching obtained. Across all conditions in both experiments, slopes of regression lines relating behavioral outputs to environmental inputs characteristically were below 0.6, which agrees with prior findings and suggests that, contrary to suggestions in the literature, undermatching in dairy cows is not the result of using different foods under alternative schedules or differential pausing under those schedules.

Sparrows, matching and the ideal free distribution: can biological and psychological approaches be synthesized?

Abstract. Two remarkably similar theories predict how animals will distribute themselves between foraging sites. The ideal free distribution predicts that, in continuous input situations, the equilibrium ratio of animals between sites will match the food input ratio. A psychological model, the matching law, predicts that the equilibrium allocation of the behaviour of individual animals will match their obtained reinforcer rates. An aviary experiment with a flock of six house sparrows, Passer domesticus, was conducted to test these ideas. Five different food input ratios were used. The behavioural allocation of the flock substantially undermatched the food input ratio. Individual food intake rates were not the same at each site. The behavioural allocation of individual birds also significantly undermatched their individual reinforcer rates. When the behavioural allocation of individual birds was plotted as a function of the overall reinforcer rates, the slopes were positively correlated with the birds' dominance ranks. In situations where there are competitive asymmetries Parker & Sutherland (1986, Anim. Behav. , 34, 1222-1242) have suggested that the sum of the competitive abilities of the foragers in a patch should match the food input ratio. However, the sum of competitive abilities in a patch again undermatched the food input ratio. As the individual birds also undermatched, it is suggested that perceptual constraints may explain some of the observed departures from the predictions of Parker & Sutherland. Despite high levels of agonistic interactions the flock behaved as a relatively cohesive unit. This exacerbated the costs of these interactions and may have also contributed to the substantial undermatching observed. The importance of social constraints on flock foraging animals is stressed.

REIN FORCER MAGNITUDE (SUCROSE CONCENTRATION) AND THE MATCHING LAW THEORY OF RESPONSE STRENGTH

This experiment investigated the relationship between reinforcer magnitude (sucrose concentration) and response rate. The purpose was to evaluate the behavior of two parameters of an equation that predicts absolute response rate as a function of reinforcement rate and two free parameters. According to Herrnstein's (1970) theory of reinforced behavior, one parameter of this "response-strength equation" measures the efficacy of the reinforcer maintaining responding and the other parameter measures motoric components of response rate, such as response duration. Seven rats served as subjects. Experimental sessions consisted of a series of five different variable-interval schedules of reinforcement, each in effect for 5 minutes. Within each session, obtained reinforcement rates varied over more than a 30-fold range, from about 20 per hour to 700 per hour. The reinforcer was sucrose solution, and, between sessions, its concentration was varied from 0.0 to 0.64 molar (0 to 21.9%). For sucrose concentrations of 0.16 to 0.64 m, response rate was a negatively accelerated function of reinforcement rate. Increases in sucrose concentration increased response rates maintained by low but not high reinforcement rates. This pattern of changes corresponds to a change in the reinforcement-efficacy parameter of the response-strength equation. In contrast, the motor-performance parameter did not change as a function of sucrose concentration. These findings are inconsistent with the results of a similar study (Bradshaw, Szabadi, & Bevan, 1978) but support Herrnstein's theory of reinforced behavior.

Does the effect of central 5-hydroxytryptamine depletion on timing depend on motivational change?

In a previous experiment we found that destruction of the ascending 5-hydroxytryptaminergic (5HTergic) pathways by microinjection of 5,7-dihydroxytryptamine into the dorsal and median raphe nuclei resulted in impaired acquisition of temporal differentiation under an interresponse-time-greater-than-15-s (IRT > 15 s) schedule of sucrose reinforcement. This paper reports three experiments, the results of which bear on the interpretation of that finding. In Experiment 1, 32 rats were trained for 120 sessions under the IRT > 15 s schedule; then 16 received lesions of the 5HTergic pathways and 16 received sham lesions. Comparisons of the IRT frequency distributions of the two groups showed that the lesion produced a significant reduction of the mean IRT and an increase in the dispersion of IRTs, as expressed by the coefficient of variation. Obtained reinforcement rates were significantly reduced in the lesioned group, but response rates were not significantly altered. Levels of 5HT and 5-hydroxyindoleacetic acid were markedly reduced in all forebrain areas examined, without significant change in noradrenaline and dopamine levels. The results indicate that destruction of the 5HTergic pathways disrupts performance as well as acquisition of temporal differentiation. Experiments 2 and 3 examined whether changes in deprivation level and reinforcer magnitude, which are known to affect reinforcer value, would influence temporal differentiation in a similar fashion to destruction of the 5HTergic pathways. In experiment 2, 20 rats were trained under the IRT > 15 s schedule while maintained at 80% or 90% of free-feeding body weight; the more severe deprivation condition was associated with a longer mean IRT and a lower coefficient of variation.(ABSTRACT TRUNCATED AT 250 WORDS)

Maximizing reinforcement rate on spaced-responding schedules under conditions of temporal uncertainty

The problem of finding the response strategy that maximizes reinforcement rate under differential-reinforcement-of low-rate (DRL) schedules, when emitted inter-response-time (IRT) distributions have some variance, is considered. In general, if IRT distributions are generated so that their standard deviation remains a constant fraction of the mean IRT, then linear overestimation of the DRL schedule value is the optimal strategy, with the degree of overestimation which is optimal depending on the degree of temporal uncertainty, but generally being in the region of 10–20%. If the mean of emitted IRT distributions underestimates the schedule value, the reinforcement rate obtained is below maximal, and increases in IRT distribution variability actually increase obtained reinforcement rate. Thus, in conditions in which underestimation is forced, the DRL schedule may discourage accurately-regulated timing. Various satisficing policies producing 25, 50, and 75% of the maximum obtainable reinforcement rate) will lead to linear underestimation of the schedule value. Experimental data showing both linear over- and under-estimation can be found in the experimental literature. In general, therefore, deviations from accurate timing of behaviour on DRL schedules may not be the result of inaccuracies of underlying timing processes, but may occur because of maximizing or satisficing reinforcement rate.

Optimization of reward rate on concurrent variable-interval variable-interval schedules

Six rats were placed on concurrent variable-interval variable-interval schedules with a 15-sec changeover delay (COD). The variable-interval schedules were varied such that the COD comprised between 25% and 100% of the average interreinforcement interval of the more favorable alternative. The obtained reinforcement rate and the rate of changing from one schedule to the other were compared to predictions of Houston and McNamara’s (1981) optimality model of concurrent choice. The pattern of behavioral allocation was consistent with the predictions of the model, although none of the animals was able to achieve optimal performance on any of the presented schedules. Observed behavior reliably tracked optimal behavior in that the ratio of obtained reinforcers to the optimum predicted by Houston and McNamara did not vary as the underlying schedule parameters was changed.

A QUANTITATIVE ANALYSIS OF CHAIN‐SCHEDULE PERFORMANCE

Six pigeons were trained with a chain variable-interval variable-interval schedule on the left key and with reinforcers available on the right key on a single variable-interval schedule arranged concurrently with both links of the chain. All three schedules were separately and systematically varied over a wide range of mean intervals. During these manipulations, the obtained reinforcer rates on constant arranged schedules also frequently changed systematically. Increasing reinforcer rates in Link 2 of the chain increased response rates in both links and decreased response rates in the variable-interval schedule concurrently available with Link 2. Increasing Link-1 reinforcer rates increased Link-1 response rates and decreased Link-2 response rates. Increasing reinforcer rates on the right-key schedule decreased response rates in Link 1 of the chain but did not affect the rate in Link 2. The results extend and amplify previous analyses of chain-schedule performance and help define the effects that a quantitative model must describe. However, the complexity of the results, and the fact that constant arranged reinforcer schedules did not necessarily lead to constant obtained reinforcer rates, precluded a quantitative analysis.

Responding on random interval schedules with high rates of reinforcement

Four rats leverpressing for food were exposed to a series of random interval schedules that provided high rates of reinforcement. In three of the four rats, response rates decreased with increases in obtained reinforcement rates. In two subjects, response rates maintained by the two schedules with the highest reinforcement rates differed even though the obtained reinforcement rates were approximately equivalent. The results generally support theories which predict a bitonic relation between response rate and reinforcement rate. They also suggest that, in the development and testing of models, more attention needs to be paid to schedule feedback functions and the role of the scheduled reinforcement rate.

Reward, performance, and the response strength method in self-stimulating rats: Validation and neuroleptics

The response strength method consists of exposing the subject to a series of variable interval schedules of reinforcement at differing densities. Response rate is plotted against obtained reinforcement rate for each schedule. The data conform to a negatively accelerated curve that is fit well by an analytical representation which contains two parameters. The values of these parameters are obtained from the fitted curve, and are suggested to independently reflect reinforcement and performance functions. In a first experiment, two manipulations were conducted that validated these suggestions. First, lowering the frequency of brain stimulation pulses induced a relatively selective shift in the reinforcement parameter. Second, increasing the force required to press the lever primarily altered the performance parameter. In a second experiment, the effects of neuroleptic administration on these two parameters were noted and compared to the results of the first experiment. In general, neuroleptics were seen to produce both reward and motor/performance impairments in self-stimulating rats.

STIMULUS AND REINFORCER RELATIVITY IN MULTIPLE SCHEDULES: LOCAL AND DIMENSIONAL EFFECTS ON SENSITIVITY TO REINFORCEMENT

Pigeons' responses in two successive components of multiple schedules were reinforced according to variable-interval schedules of reinforcement that varied over five different conditions. Within each session of all conditions, line orientations of 0 degrees , 30 degrees , or 45 degrees in Component 1 alternated with orientations of 45 degrees , 60 degrees , or 90 degrees in Component 2. Response rates were recorded in three successive subintervals of each component. Ratios were taken between the response rate in each Component 1 line orientation and the response rate in each Component 2 orientation. These ratios were found to be power functions of the corresponding ratios of obtained reinforcement rates. Sensitivity of response ratios to changes in reinforcer ratios, given by the value of the exponent of the power function, increased systematically with increasing disparity between the dimensional values of orientation stimuli. In addition, sensitivity decreased systematically over successive subintervals of components, that is, with increasing time since component alternation. Dimensional and local (subinterval) effects interacted in that sensitivity increased with stimulus disparity to a far greater extent in the first subinterval than later in components. The data could be described by a combination of rectangular hyperbolae which attributed the interaction between local and dimensional effects to limits set by local effects on the extent that stimulus differences could affect sensitivity.

CONCURRENT SCHEDULE ASSESSMENT OF FOOD PREFERENCE IN COWS

Six dairy cows (Bos taurus) were trained on several pairs of concurrent variable-interval schedules with different types of food available on each alternative. The required response was a plate press made by the animal's muzzle. Performance generally replicated that found with other species. The generalized matching law accounted for the preference data, showing that food preference could be quantitatively analyzed as a special case of response bias. The preference functions showed that the response- and time-allocation ratios were not as extreme as obtained reinforcement rate ratios (undermatching).

RESPONSE AND TIME ALLOCATION IN CONCURRENT SECOND‐ORDER SCHEDULES1

Six pigeons were trained on two-key concurrent variable-interval schedules in which the required response was the completion of a fixed number of key pecks. When the required number of pecks was equal on the two keys, response- and time-allocation ratios under-matched obtained reinforcement rate ratios. A similar result was found when the required number of pecks was unequal, except that performance, measured in response terms, was biased to the shorter required number of pecks and was less sensitive to reinforcement-rate changes. No such differences were found in the data on time spent responding. When the variable-interval schedules were kept constant and the required numbers of pecks were systematically varied, response ratios changed inversely with the ratio of the required number of pecks, but time-allocation ratios varied directly with the same independent variable. Thus, on response measures, pigeons "prefer" the schedule with the smaller peck requirement, but on time measures they "prefer" the schedule with the larger peck requirement. This finding is inconsistent with a commonsense notion of choice, which sees response and time-allocation measures as equivalent.

PREFERENCE FOR FIXED‐INTERVAL SCHEDULES: AN ALTERNATIVE MODEL1

Pigeons were trained under concurrent chain schedules in which the initial links were equal aperiodic schedules and the terminal links were fixed-interval schedules. Choice proportions in the initial links were measured in 26 experimental conditions. The data showed the inadequacy of previous models of concurrent chain performance. A new model was suggested in which choice is a joint function of terminal-link times, overall reinforcement rates, and terminal-link entries. This model accounted for 94% of the variance in the present data and for substantial percentages of the variance in previously reported data. The model simplifies to matching between response ratios and obtained reinforcement rate ratios for simple concurrent schedule performance.