Variable Reinforcement Schedule Research Articles

Comparison of progressive hold and progressive response schedules of reinforcement

Progressive ratio (PR) schedules have been widely used to study motivation to work for a reinforcer. After a post-reinforcer pause, subjects engage in pressing a lever until a reinforcer is obtained. However, the discrete nature of lever presses allows alternative behaviors during inter-response time and has led to several behavioral explanations of pauses and work time. A progressive hold-down (PH) is incompatible with alternative responses and may allow a precise estimation of work time. Performance of rats trained in both PR and PH that received sucrose or intracranial self-stimulation (ICSS) as reinforcer were compared. We observed that rats mastered the PR and PH schedules. Post-reinforcer pauses (PSRP), work time and inter-reinforcer time increased as a function of the response or hold requirement. However, rats’ performance suggested that the PH progression may be experienced by the rats as easier that the PR progression. Elimination of consummatory behavior with ICSS reduced post-reinforcer pause in accordance with predictions of explanatory models of fixed and variable schedules of reinforcement. In the case of PH performance, pauses showed little variation across intermediate requirements but increased rapidly on later requirements; since rats controlled their pause length and work time was close to the hold requirement, time allocation between PR and PH schedules diverged. Finally, the Mathematical Principles of Reinforcement model of Bradshaw and Killeen (Psychopharmacology 2012, 222: 549) rendered a good description of the performance in both PR and PH schedules.

Health Promotion through Monetary Incentives: Evaluating the Impact of Different Reinforcement Schedules on Engagement Levels with a mHealth App

Background: Financial rewards can be employed in mHealth apps to effectively promote health behaviors. However, the optimal reinforcement schedule—with a high impact, but relatively low costs—remains unclear. Methods: We evaluated the impact of different reinforcement schedules on engagement levels with a mHealth app in a six-week, three-arm randomized intervention trial, while taking into account personality differences. Participants (i.e., university staff and students, N = 61) were awarded virtual points for performing health-related activities. Their performance was displayed via a dashboard, leaderboard, and newsfeed. Additionally, participants could win financial rewards. These rewards were distributed using a fixed schedule in the first study arm, and a variable schedule in the other arms. Furthermore, payouts were immediate in the first two arms, whereas payouts in the third arm were delayed. Results: All three reinforcement schedules had a similar impact on user engagement, although the variable schedule with immediate payouts was reported to have the lowest cost per participant. Additionally, the impact of financial rewards was affected by personal characteristics. Especially, individuals that were triggered by the rewards had a greater ability to defer gratification. Conclusion: When employing financial rewards in mHealth apps, variable reinforcement schedules with immediate payouts are preferred from the perspective of cost and impact.

Dolphins maintain high echolocation vigilance for eight hours without primary (food) reinforcement.

Studies have demonstrated that dolphins can maintain continuous auditory or echolocation vigilance for up to 5 to 15 days when provided with continuous primary reinforcement (i.e., food reward after each correct detection). The goals of this study were to examine whether dolphins could perform an 8-h echolocation vigilance task featuring variable reinforcement schedules, where correct responses were intermittently rewarded, and variable acoustic secondary reinforcement (feedback) patterns. Three dolphins were trained to echolocate simulated targets and press a response paddle upon detecting echoes. Three conditioned reinforcement conditions were utilized: no (acoustic) feedback, acoustic feedback, and structured acoustic feedback. The probability of primary reinforcement following a correct response began at 50% for all dolphins but was sequentially reduced to 25%, 12%, 6%, and 0% each time performance criteria were met. Conditions including acoustic feedback resulted in two dolphins successfully performing the echolocation vigilance task under the 0% primary reinforcement schedule (8 h before receiving primary reinforcement). None of the animals reached 0% reinforcement probability in the no feedback condition. The results demonstrate that dolphins can perform experimental echolocation tasks for extended time periods without primary reinforcement and suggest that secondary reinforcement may be important to maintain this behavior.

The copyist model of response emission

The variety of different performances maintained by schedules of reinforcement complicates comprehensive model creation. The present account assumes the simpler goal of modeling the performances of only variable reinforcement schedules because they tend to maintain steady response rates over time. The model presented assumes that rate is determined by the mean of interresponse times (time between two responses) between successive reinforcers, averaged so that their contribution to that mean diminishes exponentially with distance from reinforcement. To respond, the model randomly selects an interresponse time from the last 300 of these mean interresponse times, the selection likelihood arranged so that the proportion of session time spent emitting each of these 300 interresponse times is the same. This interresponse time defines the mean of an exponential distribution from which one is randomly chosen for emission. The response rates obtained approximated those found on several variable schedules. Furthermore, the model reproduced three effects: (1) the variable ratio maintaining higher response rates than does the variable interval; (2) the finding for variable schedules that when the reinforcement rate varies from low to high, the response rate function has an ascending and then descending limb; and (3) matching on concurrent schedules. Because these results are due to an algorithm that reproduces reinforced interresponse times, responding to single and concurrent schedules is viewed as merely copying what was reinforced before.

Choice behavior in spontaneously hypertensive rats: Variable vs. fixed schedules of reinforcement

Spontaneously hypertensive rats (SHR) and Wistar rats were evaluated in the successive-encounters procedure (an operant simulation of natural foraging) with the idea of assessing differences between them in their preference for variable schedules of reinforcement. In this procedure, after satisfying a schedule of reinforcement associated with search time, the subjects could "accept" or "reject" another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available: a fixed interval (FI), and a variable interval (VI) with the same mean value. The results indicated preference for the variable schedule in both strains, as suggested by the observation that the VI was always accepted while the FI was often rejected. The difference in FI acceptability between strains was not statistically significant, a result which is relevant for the current debate of SHR as an adequate animal model of Attention Deficit Hyperactivity Disorder.

Impaired Behavior Regulation Under Conditions of Concurrent Variable Schedules of Reinforcement in Children With ADHD

Objective: To bridge theory of response inhibition and learning in children with ADHD. Method: Thirty ADHD and 30 non-ADHD children (ages 9-12) were compared under concurrent variable interval (VI-15 sec., VI-30 sec. and VI- 45 sec.) reinforcement schedules that required the child to switch between the three schedules under conditions of experimentally controlled inhibition (change over delay [COD] vs. No COD). Classical matching law was used to evaluate children’s success in maximizing reinforcement opportunities. Results: Children with ADHD showed normal matching only when immediate reinforcement for responding was blocked by the presence of a 3-s COD. Without a COD, ADHD children failed to conform to the normal matching law. Non-ADHD children’s behavior fit the matching law (i.e., rate of response was proportional to rate of reinforcement) whether a COD was present or absent. Conclusions: Results supported other findings that response inhibition is a key mechanism in responses to reinforcement schedules by children with ADHD and that the absence of contingencies that inhibit impulsive responding might impair complex learning in which the child must choose between many different behavioral options, each with associated reinforcement schedules. (J. of Att. Dis. 2010; 13(4) 358-368)

GENERATING VARIABLE AND RANDOM SCHEDULES OF REINFORCEMENT USING MICROSOFT EXCEL MACROS

Variable reinforcement schedules are used to arrange the availability of reinforcement following varying response ratios or intervals of time. Random reinforcement schedules are subtypes of variable reinforcement schedules that can be used to arrange the availability of reinforcement at a constant probability across number of responses or time. Generating schedule values for variable and random reinforcement schedules can be difficult. The present article describes the steps necessary to write macros in Microsoft Excel that will generate variable-ratio, variable-interval, variable-time, random-ratio, random-interval, and random-time reinforcement schedule values.

Energy budget versus temporal discounting as determinants of preference in risky choice.

Four pigeons and three ringneck doves responded on an operant simulation of natural foraging. After satisfying a schedule of reinforcement associated with search time, subjects could "accept" or "reject" another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available, a variable interval, and a fixed interval with the same mean value. Food available in the session (a variable related to the energy budget) was manipulated in the different conditions either by increases of the value of the search state schedule of reinforcement, or by increases in the mean value of the handling state schedules. The results indicate that the amount of food available in the session did not affect the preference for variable schedules of reinforcement, as would be predicted by an influential theory of risk sensitive foraging. Instead, the preference for variability depended on the relationship between the time spent in the search and the handling states, as is predicted by a family of models of choice that are based on the temporal proximity to the reinforcer.

Transitional choice behavior in concurrent-chain schedules

The choice responses of four pigeons were examined in 20 periods of transition in a concurrent-chain procedure with variable-interval schedules as initial links and fixed delays to reinforcement as terminal links. In some conditions, the delays to reinforcement were different for the two terminal links, and changes in preference were recorded after the delays for the two response keys were switched. In other conditions, the reinforcer delays were equal for the two keys, but which key delivered 80% of the reinforcers was periodically switched. Choice proportions changed more quickly after a switch in reinforcement percentages than after a switch in the delays, thereby contradicting the hypothesis that faster changes would occur when the switch in conditions was easier to discriminate. Analyses of response sequences showed that the effects of individual reinforcers were larger and lasted longer in conditions with changing reinforcement percentages than in conditions with changing terminal-link delays. Rates of change in choice behavior do not appear to be limited by the unpredictability of variable reinforcement schedules, because the changes in behavior were slow and gradual even when there was a large and sudden change in reinforcer delays.

Drug discrimination is a continuous rather than a quantal process following training on a VI-TO schedule of reinforcement

Debate continues as to whether drug discrimination in animals is an inherently quantal or continuous process. This issue is important in determining the appropriate interpretation of results from drug discrimination studies designed to assess the nature of drug-induced interoceptive cues. The quantal approach holds that subjects perceive a drug cue in an all-or-none manner, while the continuous view proposes that when appropriate training and testing procedures are used, subjects can discriminate along a continuum of interoceptive cues. Data consistent with the quantal view have consistently been generated by animals trained to respond on schedules of reinforcement having an FR component. Since quantal responding is a characteristic of these schedules, results from drug discrimination studies using training schedules with FR components are of little value in empirically determining whether drug discrimination reflects a quantal or continuous process. Use of variable schedules of reinforcement might be more appropriate because the pattern of responding generated does not preclude results consistent with either of the competing views. Data from the following studies that trained subjects using VI schedules with a concurrent TO for incorrect lever responding were analyzed: Barrett et al. (1982): L-5-hydroxytryptophan versus saline; Smith (1990): diazepam versus pentylenetetrazol; Barrett et al. (1992): amphetamine versus haloperidol; Barrett and Steranka (1983): amphetamine versus haloperidol. In every case, when experimental conditions produced a group mean intermediate to that for the training drugs, the distribution of scores for individual animals was normally rather than bimodally distributed.

GOING BEYOND WITH BAYESIAN UPDATING

It all started innocently enough. One spring‐like winter day, I happened to ask Brendan whether economists ever dealt with escalation.“With what?” he replied.“A phenomenon where people keep investing in the face of continuing losses,” I answered.Then I described how industrial/organizational psychologists had become intrigued with situations in which investors seemed to throw good money after bad, how their explanations for the phenomenon centered on individual characteristics such as commitment, how Sonia Goltz (1992) used a standard bread‐and‐butter operant procedure—fixed and variable schedules of reinforcement—to explain their persistence, and how Goltz's experiments had shown that during the extinction phase investors even increased their investments for a while when the news was all bad.Without a moment's hesitation, he exclaimed, “I bet I can predict the turning point.”Another arrogant‐economist remark, I thought to myself. “How?”“Bayesian updating.”Then we talked at length about Bayesian analysis techniques and how they could be used to predict the shape of extinction curves. I realized that these techniques might be just what psychologists needed as an enticement to study sequences of behavior over time.And that's how this commentary got started.

Cocaine tolerance under variable schedules of reinforcement: Marc N. Branch. University of Florida

Cocaine tolerance under variable schedules of reinforcement: Marc N. Branch. University of Florida

Continuous Reinforcement to Facilitate Extinction to Counter Exhaustion

A problem behaviour which had been maintained on a variable reinforcement schedule was first put onto a continuous schedule and subsequently treated by extinction. The rapid response to extinction was judged to have been due to the prior move to a CRF schedule. Reasons for the emergence and maintenance of problem behaviours in certain situations are discussed.

Spinners and Chartmoves:

A practical child management procedure is presented which incorporates self-recording, variable schedules of reinforcement, and a practical data collection system. The proposed system has two basic components, chartmoves and spinners. The chart is generally a picture with 30 to 100 dots and functions as a data collection system and indicates when a reinforcer has been programmed. Reinforcers are set up by randomly placing large dots (reinforcer dot), which make up the picture. When a child exhibits an appropriate target behavior, he is allowed to connect two dots on the picture. Each dot is covered with a scratch off ink; if the last dot connected is large the child receives a reinforcer. Chartmoves (connection of dots) are dated and initialed by the teacher allowing a record of progress to be kept with little effort by the teacher or parent. The second component to the system is the spinner. The spinner is an unevenly divided circle (generally five sections) with each section containing a reinforcer, plu...

The Individual Point Card:

A procedure for implementing an individualized point card for use with token economies is introduced. The point card incorporates a "Constant" section which is paid off on an FR or CFR schedule of reinforcement for routine, discrete tasks, such as cleaning a room. A "Variable" section is included which is paid off on a VR schedule of reinforcement for behaviors which are less discrete and are to be maintained at a steady state, such as being polite to others. Also included is a "General" section, which is to be filled in with behaviors which are individual to the client, such as not stealing from others. The behaviors in the General section can be paid off on either fixed or variable schedules of reinforcement.

Methods of Delivering Variable Schedules of Reinforcement

The difficulties encountered in implementing variable schedules of reinforcement were presented and discussed. Procedures then were given and described for constructing instruments that are designed to help teachers deliver variable ratio, variable interval, and variable DRO schedules of reinforcement accurately and pragmatically. A step by step description of the instruments and their use was given and described.

Incidence of Physical Disorders Among Psychiatric In-patients

<h3>Abstract</h3> Progressive ratio (PR) schedules had been widely used to study motivation to work for a reinforcer. After a post-reinforcer pause, subjects engage pressing a lever until a reinforcer is obtained. However, the discrete nature of lever presses allows alternative behaviors during inter-response time and had lead to the suggestion of several behavioral categories to explain pauses and work time. A progressive hold-down (PH) is incompatible with alternative responses and may allow a precise estimation of work time. Performance of rats trained in both PR and PH that received sucrose or intracranial self-stimulation (ICSS) as reinforcer were compared. We observed that rats mastered the PR and PH schedules. Post-reinforcer pauses, work time and inter-reinforcer time increased as a function of the response or hold requirement. However, rat’s performance suggest that the PH progression may be experienced by the rats as easier that the PR progression. Elimination of consummatory behavior with ICSS reduced PS^RP and in accordance with predictions of explanatory models of fixed and variable schedules of reinforcement. In the case of PH performance, intermediate requirements leveled off pauses but inceased rapidly on later requirements; since rats controlled pause length and work time was close to hold requirement, time allocation between PR and PH schedules diverged. Finally, the Mathematical Principles of Reinforcement model of Bradshaw and Killeen (Psychopharmacology 2012, 222: 549) rendered a good description of the performance in both PR and PH schedules.