Primary Reinforcer Research Articles

Rats' choices with token stimuli in concurrent variable-interval schedules.

Four rats responded on concurrent variable-interval schedules that delivered token stimuli (stimulus lights arranged vertically above each of two side levers). During exchange periods, each token could be exchanged for one food pellet by responding on a center lever, with one response required for each pellet delivery. In different conditions, the exchange requirements (number of tokens that had to be earned before they could be exchanged for food) varied between one and four for the two response levers. The experiments were closely patterned after research with pigeons by Mazur and Biondi (2013), and the results from the rats in the present experiment were similar. Response percentages on the two levers changed as each additional token was earned, and these patterns indicated that choice was controlled by both the time to the exchange periods and the number of food pellets that were delivered in the exchange period. In some conditions, the exchange requirement was three tokens for each lever, but the token lights were not turned on as they were earned for one of the two keys. The rats showed a slight preference for the lever without the token lights, which may indicate that the token lights were not serving as conditioned reinforcers (a result also found by Mazur and Biondi with pigeons). Overall, these results suggest that, in this choice procedure, the token stimuli served primarily as discriminative stimuli that signaled the temporal proximity and quantity of the primary reinforcer, food.

Observing of chain-schedule stimuli

A classical-conditioning account of the processes maintaining behavior under chained schedules entails a backward transmission of conditioned-reinforcement effects. Assessing this process in traditional chain schedules is limited because the response maintained by stimulus onset accompanied by each link in a chain schedule may also be maintained by the primary reinforcer. In the present experiment, an observing response was used to measure the conditioned-reinforcing effects of stimuli associated with a three-link chain variable-time (VT) food schedule, and resistance-to-change tests (extinction and prefeeding) were implemented to examine if a backward transmission of reinforcement effects occur. Four pigeons served as subjects. Observing was maintained by the production of stimuli correlated with links of a three-link chain VT schedule with the middle-link stimulus maintaining the highest rate of observing, followed by the initial-link stimulus and the terminal-link stimulus maintaining the lowest observing rate. Results from resistance-to-change tests of extinction and prefeeding were not supportive of a backward transmission of reinforcement effects and in general, the pattern of resistance-to-change was forward. Based on past and current research, it appears that a backward pattern of relative rate decreases in responses maintained by stimuli correlated with a chain schedule due to disruption (i.e., extinction and prefeeding) is not a ubiquitous process that is evident within different chain-schedule arrangements.

Touching moments: desire modulates the neural anticipation of active romantic caress

A romantic caress is a basic expression of affiliative behavior and a primary reinforcer. Given its inherent affective valence, its performance also would imply the prediction of reward values. For example, touching a person for whom one has strong passionate feelings likely is motivated by a strong desire for physical contact and associated with the anticipation of hedonic experiences. The present study aims at investigating how the anticipatory neural processes of active romantic caress are modulated by the intensity of the desire for affective contact as reflected by passionate feelings for the other. Functional magnetic resonance imaging scanning was performed in romantically involved partners using a paradigm that allowed to isolate the specific anticipatory representations of active romantic caress, compared with control caress, while testing for the relationship between neural activity and measures of feelings of passionate love for the other. The results demonstrated that right posterior insula activity in anticipation of romantic caress significantly co-varied with the intensity of desire for union with the other. This effect was independent of the sensory-affective properties of the performed touch, like its pleasantness. Furthermore, functional connectivity analysis showed that the same posterior insula cluster interacted with brain regions related to sensory-motor functions as well as to the processing and anticipation of reward. The findings provide insight on the neural substrate mediating between the desire for and the performance of romantic caress. In particular, we propose that anticipatory activity patterns in posterior insula may modulate subsequent sensory-affective processing of skin-to-skin contact.

Incentive motivational salience and the human brain

In this paper the concept of incentive motivational salience is briefly described, pioneering studies on the subject of the mesolimbic motivational system are reviewed, and studies we have been involved in conducting which elaborate on this subject are discussed. In particular, we aim to show that the mesolimbic motivational system is recruited as a reaction to primary and secondary reinforcers as a function of salience, that is independent of valence. Furthermore, studies showing that both psychological and pharmacological interventions can affect the function of the mesolimbic motivational system and how its' dysfunction is related to psychopathological phenomena with an emphasis on psychosis are discussed.

Response to Ahrendt, Houlihan, and Buchanan.

The implementation of seclusion time-out (STO) described by Ahrendt, Houlihan, and Buchanan (2013) reveals several contemporary practice standards within the domain of clinical behavior analysis. First, the "time-in" environment of the inpatient facility included a token-based incentive program by which staff reinforced "positive behavior" and consumers exchanged tokens for primary reinforcers. Thus, the intent was to increase motivation to avoid STO in favor of remaining in the reinforcing environment (Marlow, Tingstrom, Olmi, & Edwards, 1997).

Investigations of the Neurobiological Bases of Inter-Temporal Choice Behaviour

Inter-temporal choice schedules entail choice between rewarding outcomes in which the primary reinforcers differ along two or more dimensions (usually size and delay). This paper summarizes a programme of research on the neurobiology of inter-temporal choice carried out in the authors’ laboratory. A theoretical model of inter-temporal choice (the multiplicative hyperbolic model, MHM) based on the principles of delay discounting and diminishing marginal utility is described. This model provides a basis for designing experiments which enable the influences of delay and size of reinforcers to be separately quantified. Experimental results are reviewed which implicate the orbital prefrontal cortex (OPFC) and the core of the nucleus accumbens (AcbC) in sensitivity to delay of reinforcement (delay discounting) and the OPFC and the subthalamic nucleus (STN) in sensitivity to reinforcer size. It is suggested that the OPFC may collate the influences of multiple features of reinforcers, and thereby determine the overall values of reinforcing outcomes. The implications of the results for the concepts of ‘value’ and ‘impulsiveness’ are discussed.

Smokers Discount Their Drug of Abuse in the Same Way as Other Consumable Rewards

Recent research shows that drug abusers discount delayed monetary rewards more than nonabusers do, and they discount delayed substances of abuse (e.g., drugs) more than delayed money. Furthermore, non-drug-abusers discount food and substances of abuse (e.g., alcohol), more than money. Here, we compare the delay and probability discounting of money with that of a directly consumable reward (chocolate) and with that of a substance of abuse (cigarettes), in a drug-using population (smokers). In line with previous research, we found in two experiments that delay discounting differentiated between smokers and nonsmokers, and between money and a nonabused directly consumable reward (chocolate). In addition, our results show that there appears to be no difference in the extent to which smokers discount their abused substance compared to another directly consumable reward. These findings support the contention that drugs and food are part of the same category of primary reinforcers, whereas money is discounted differently, as a conditioned reinforcer.

Comparison of 3 different reinforcements of learning in dogs (Canis familiaris)

Abstract Effective dog training involves reinforcement of the dog's correct actions in response to specific cues given by the trainer. Food is usually selected as the primary reinforcer (reward), although this selection does not necessarily account for the learning preferences of individual dogs. We evaluated the relationship between reward and learning efficiency. Fifteen dogs were allocated to 3 different reward groups (food, stroking, and praise) and trained by an identical process. The food reward was the only one that shortened the time taken for the response to the command to be completed. However, this difference occurred only in the early training stages and not later in the training process.

Reinstatement in a cocaine versus food choice situation: reversal of preference between drug and non‐drug rewards

Recent studies show that when given a mutually exclusive choice between cocaine and food, rats almost exclusively choose food. The present experiment investigated potential shifts in preference between levers associated with either food or cocaine that might occur during extinction (food and cocaine no longer available) and during footshock-induced, cocaine-primed and food-primed reinstatement. During self-administration sessions where food and cocaine were simultaneously available, rats demonstrated a stable food preference, choosing food over cocaine on 83% of trials. During extinction when neither reinforcer was available, no preference between levers was evident and responding decreased until rats responded on the previously food- and cocaine-associated levers at equally low rates. Footshock resulted in a non-specific reinstatement of responding upon both levers, while cocaine priming resulted in a significant preference for cocaine seeking over food seeking. This suggests that the mechanism underlying footshock-induced reinstatement is distinct from that of cocaine-primed reinstatement. Food priming engendered a mild, non-specific increase in responding on both levers. Although rats generally prefer food over cocaine when presented with a choice between these primary reinforcers, the present results suggest that in certain situations, cocaine-seeking behavior prevails over food-seeking behavior.

Monetary Effects on Fear Conditioning

Previous research has found that the loss of money as a negative secondary reinforcer was as effective as a primary reinforcer during fear conditioning. The purpose of the present study was to explore the effect of monetary gain as a positive secondary reinforcer in fear conditioning. Participants were assigned to a high-reward group or low-reward group. Three kinds of squares prompting non-compensation shock, compensation shock, and no shock were presented. Skin conductance responses (SCRs) and self-ratings were recorded. The results revealed that (a) both SCRs and self-ratings in the compensation shock condition were lower than in the non-compensation shock condition, suggesting that money might block the learning stage of fear conditioning; and (b) a higher ratio of fear reduction was present in self-rating when compared to SCRs, suggesting that people might overstate the utility of money, subjectively. Monetary effects, the effects of different amounts of money, and the differences between subjective and physiological levels are discussed.

The effects of differential outcomes and different types of consequential stimuli on 7-year-old children’s discriminative learning and memory

Researchers have demonstrated that discriminative learning is facilitated when a particular outcome is associated with each relation to be learned. Our primary purpose in the two experiments reported here was to assess whether the differential outcomes procedure (DOP) would enhance 7-year-old children's learning of symbolic discriminations using three different forms of consequences in which (1) reinforcers are given when correct choices are made ("+"), (2) reinforcers are withdrawn when errors are made ("-"), or (3) children receive a reinforcer following a correct choice and lose one following an incorrect choice ("+/-"), as well as different types of reinforcers (secondary and primary reinforcers, Experiment 1; primary reinforcers alone, Experiment 2). Participants learned the task faster and showed significantly better performance whenever differential outcomes were arranged independently of (1) the way of providing consequences (+, -, or +/-) and (2) the type of reinforcers being used. Interestingly, as in a previous study with 5-year-old children (Martínez, Estévez, Fuentes, & Overmier, The Quarterly Journal of Experimental Psychology 62(8):1617-1630, 2009), the use of the DOP also enhanced long-term persistence of learning.

CONDITIONED REINFORCEMENT IN CHAIN SCHEDULES WHEN TIME TO REINFORCEMENT IS HELD CONSTANT

Two alternative approaches describe determinants of responding to a stimulus temporally distant from primary reinforcement. One emphasizes the temporal relation of each stimulus to the primary reinforcer, with relative proximity of the stimulus determining response rate. A contrasting view emphasizes immediate consequences of responding to the stimulus, the key factor being the conditioned reinforcement value of those immediate consequences. To contrast these approaches, 4 pigeons were exposed to a two-component multiple schedule with three-link chain schedules in each component. Only middle-link stimuli differed between chains. Baseline reinforcement probabilities were 0.50 for both chains; during discrimination phases it was 1.0 for one chain and 0.0 for the other. During discrimination phases pigeons responded more to the reinforcement-correlated middle link than to the extinction-correlated middle link, demonstrating that responding was affected by the probability change. Terminal link responding was also higher in the reinforced chain, even though the terminal link stimulus was identical in both chains. Of greatest interest is initial link responding, which was temporally most distant from reinforcement. Initial link responding, necessarily equal in the two chains, was significantly higher during the 1.0/0.0 discrimination phases, even though overall reinforcement probability remained constant. For 3 of 4 birds, in fact, initial-link response rates were higher than terminal-link response rates, an outcome that can be ascribed only to the potent conditioned reinforcement properties of the middle-link stimulus during the discrimination phases. Results are incompatible with any account of chain behavior based solely on relative time to reinforcement.

Establishment and Maintenance of Socially Learned Conditioned Reinforcement in Young Children: Elimination of the Role of Adults and View of Peers' Faces

Prior research has demonstrated the establishment of reinforcers for learning and maintenance with young children as a function of social learning where a peer and an adult experimenter were present. The presence of an adult experimenter was eliminated in the present study to test if the effect produced in the prior studies would occur with only the peers present. Neutral stimuli, metal washers, were first demonstrated not to reinforce maintenance of responding nor learning compared to primary reinforcers determined by pre-intervention functional analyses with three 4-year-old children with slight language delays. In the intervention, washers were delivered mechanically to a peer confederate in the participants’ view as the participants and peers completed a performance task side by side. For several brief sessions, peers received washers via an opaque chute into a transparent plastic cup on the desk in view of the participants for responding, while participants, who also responded, did not receive washers. After the target students demonstrated extinction or criterion for termination of the procedure was reached, we returned to pre-intervention assessments, and participants responded equally under washer and primary reinforcers with effects comparable to prior experiments where the experimenter was present. Washers also maintained reinforcement effects 6 to 10 weeks later in classroom instruction.

Concerns with attempts by neuroeconomics to answer the philosophical question “Is it rational to donate money for charity?”

GENERAL COMMENTARY article Front. Psychol., 30 August 2013Sec. Theoretical and Philosophical Psychology Volume 4 - 2013 | https://doi.org/10.3389/fpsyg.2013.00585

Choices between segmented and unsegmented schedules and the self-control paradigm

A concurrent-chains procedure was used to examine choice between a segmented (two- or three-terminal-link segments schedules) and an unsegmented schedule (simple schedule) in terminal links with equal interreinforcement intervals. In most such experiments, preference for the unsegmented schedule has been found, but in a recent study with humans (Alessandri et al., 2010) a reversal in preference was found when, in the segmented schedule, the terminal link segmenting stimulus was presented briefly and closer to food delivery such that the early terminal link stimulus was temporally closer to the food delivery. In Experiment 1, an attempt to replicate this latter effect with pigeons was unsuccessful but this outcome was consistent with an account in terms of a self-control contingency involving conditioned reinforcers. According to this account, the unsegmented alternative consisted of an immediate, smaller presentation of a conditioned reinforcer (i.e., the impulsive, and thus usually the preferred, option in several experiments) and the segmented schedule led to a delayed, larger conditioned reinforcer (i.e., the self-control option). In Experiment 2, a reversal of preference toward the segmented schedule was found when a delay was added to both terminal links between the reinforced initial-link response and the onset of the corresponding terminal link stimulus. This result is consistent with a similar effect found with primary reinforcers in the self-control literature suggesting the utility of self-control as an account of preferences for unsegmented terminal links of concurrent chains schedules.

Transient inactivation of basolateral amygdala during selective satiation disrupts reinforcer devaluation in rats.

Basolateral amygdala (BLA) function is critical for flexible, goal-directed behavior, including performance on reinforcer devaluation tasks. Here we tested, in rats, the hypothesis that BLA is critical for conditioned reinforcer devaluation during the period when the primary reinforcer (food) is being devalued (by feeding it to satiety), but not thereafter for guiding behavioral choices. We used a spatially independent task that used two visual cues, each predicting one of two foods. An instrumental action (lever press) was required for reinforcer delivery. After training, rats received BLA or sham lesions, or cannulae implanted in BLA. Under control conditions (sham lesions, saline infusions), devaluation of one food significantly decreased responding to the cue associated with that food, when both cues were presented simultaneously during extinction. BLA lesions impaired this devaluation effect. Transient inactivation of BLA by microinfusion of the γ-aminobutyric acid receptor type A agonist muscimol resulted in an impairment only when BLA was inactivated during satiation. When muscimol was infused after satiation and therefore, BLA was inactivated only during the choice test, rats showed no impairment. Thus, BLA is necessary for registering or updating cues to reflect updated reinforcer values, but not for guiding choices once the value has been updated. Our results are the first to describe the contribution of rat BLA to specific components of reinforcer devaluation and are the first to show impairment in reinforcer devaluation following transient inactivation in the rat.

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5 s light onset) with fixed ratio 1 (FR 1), variable-interval 1 min (VI 1 min), or variable-interval 6 min (VI 6 min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6 min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1 min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6 min schedule relative to the FR 1 and VI 1 min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1 min groups as compared to the VI 6 min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.

Monetary loss alters perceptual thresholds and compromises future decisions via amygdala and prefrontal networks.

The influence of monetary loss on decision making and choice behavior is extensively studied. However, the effect of loss on sensory perception is less explored. Here, we use conditioning in human subjects to explore how monetary loss associated with a pure tone can affect changes in perceptual thresholds for the previously neutral stimulus. We found that loss conditioning, when compared with neutral exposure, decreases sensitivity and increases perceptual thresholds (i.e., a relative increase in the just-noticeable-difference). This was so even when compared with gain conditioning of comparable intensity, suggesting that the finding is related to valence. We further show that these perceptual changes are related to future decisions about stimuli that are farther away from the conditioned one (wider generalization), resulting in overall increased and irrational monetary loss for the subjects. We use functional imaging to identify the neural network whose activity correlates with the deterioration in sensitivity on an individual basis. In addition, we show that activity in the amygdala was tightly correlated with the wider behavioral generalization, namely, when wrong decisions were made. We suggest that, in principle, less discrimination can be beneficial in loss scenarios, because it assures an accurate and fast response to stimuli that resemble the original stimulus and hence have a high likelihood of entailing the same outcome. But whereas this can be useful for primary reinforcers that can impact survival, it can also underlie wrong and costly behaviors in scenarios of contemporary life that involve secondary reinforcers.

The Effect of Sensorimotor Replacement on Smoking Cessation and Craving

Current treatments for smoking cessation such as nicotine replacement therapy or varenicline address the primary reinforcer of smoking (nicotine), but sensorimotor stimuli (e.g. smell/taste of smoke, inhaling/exhaling, airway sensations, holding the cigarette) may act as secondary reinforcers and also contribute to smoking reward. Addressing both these aspects of smoking may help to enhance smoking cessation treatment. The aim of this review was to examine whether sensorimotor replacement can help to alleviate craving and aid smoking cessation. Three sensorimotor replacement products were examined: non-nicotine inhalators/aerosols, de-nicotinised cigarettes and electronic cigarettes. The current research suggests that sensorimotor replacement may enhance the efficacy of nicotine replacement therapy, but is unlikely to be useful if used alone. Electronic cigarettes may be the most promising approach, due to the combination of nicotine delivery and sensorimotor input.

P300- like event related potential amplitude in rats is a correlate of conditioned reinforcement

AbstractWe have developed a methodology for recording a robust P300 event related potential (ERP) in rats. In these experiments a contingency shaped model of the human “oddball’ paradigm was employed in which rats were shaped to press a lever for food reinforcement signaled by the click of the pellet dispenser. A target tone cued the insertion of the lever that retracted after 1-sec or immediately following a single reinforced response, while a non-target tone was randomly presented. Brain activity was recorded through stainless steel electrodes implanted 1mm below the skull. Here, we compared the amplitude of the P300 response to the click of the pellet dispenser to the amplitude of the P300 response to the target and non-target tones. We found that the amplitude to food click was significantly greater that the amplitude to the target tone that cued lever insertion. Since the food click is more proximal to the primary reinforcer than the lever tone, it is a stronger conditioned reinforcer than the lever tone that sets the occasion for the food click. Accordingly we suggest that the P300 in rats is a correlate of conditioned reinforcement.