Partial Reinforcement Conditions Research Articles

Evidence for Different Roles of Inhibitory and Prospective Intolerance of Uncertainty During Threat Discrimination Learning

Uncertainty is a core component of threat and associated learning processes. One methodological factor impacting uncertainty in threat learning paradigms is the threat reinforcement rate, which refers to the proportion of times a cue is reinforced with an aversive stimulus. This study tested the effect of partial vs continuous threat reinforcement on threat / safety discrimination learning, as indexed by skin conductance response (SCR). Using a within-participants design, fifty-nine participants completed a task in which three colored shapes were paired with electric shock at reinforcement schedules of 100% (CS+), 50% (CS+) and 0% (CS-). In addition, the study examined the relationship between the Intolerance of Uncertainty scale (IU) and two subscales – inhibitory and prospective IU – with threat discrimination learning. The data show heightened SCR in the continuous vs partial reinforcement condition to all stimuli, but limited evidence of enhanced discrimination learning. Furthermore, no association was observed between total IU score and threat-safety discrimination. However, using a two-factor model of IU, findings showed higher inhibitory IU and higher prospective IU were associated with diminished and heightened threat discrimination, respectively. These results contribute to a fast-growing literature exploring how the uncertainty inherent to predictors of threat, individual differences in sensitivity to uncertainty, and interactions between these two factors, can shape the acquisition of threat memory.

Partial reinforcement in rat autoshaping with a long CS: Effects of pramipexole and chlordiazepoxide on sign and goal tracking

Abstract In Pavlovian autoshaping, sign-tracking responses (lever pressing) to a conditioned stimulus (CS) are usually invigorated under partial reinforcement (PR) compared to continuous reinforcement (CR). This effect, called the PR acquisition effect (PRAE), can be interpreted in terms of increased incentive hope or frustration-induced drive derived from PR training. Incentive hope and frustration have been related to dopaminergic and GABAergic activity, respectively. We examined the within-trial dynamics of sign and goal tracking in rats exposed to 20-s-long lever presentations during autoshaping acquisition under PR vs. CR conditions under the effects of drugs tapping on dopamine and GABA activity. There was no evidence of the PRAE in these results, both groups showing high, stable sign-tracking response rates. However, the pharmacological treatments affected behavior as revealed in within-trial changes. The dopamine D2 receptor agonist pramipexole (0.4 mg/kg) suppressed lever pressing and magazine entries relative to saline controls in a within-subject design, but only in PR animals. The allosteric benzodiazepine chlordiazepoxide (5 mg/kg) failed to affect either sign or goal tracking in either CR or PR animals. These results emphasize the roles of dopamine and GABA receptors in autoshaping performance, but remain inconclusive with respect to incentive hope and frustration theories. Some aspects of within-trial changes in sign and goal tracking are consistent with a mixture of reward timing and response competition.

Working Memory and Reinforcement Schedule Jointly Determine Reinforcement Learning in Children: Potential Implications for Behavioral Parent Training.

Introduction: Behavioral Parent Training (BPT) is often provided for childhood psychiatric disorders. These disorders have been shown to be associated with working memory impairments. BPT is based on operant learning principles, yet how operant principles shape behavior (through the partial reinforcement (PRF) extinction effect, i.e., greater resistance to extinction that is created when behavior is reinforced partially rather than continuously) and the potential role of working memory therein is scarcely studied in children. This study explored the PRF extinction effect and the role of working memory therein using experimental tasks in typically developing children.Methods: Ninety-seven children (age 6–10) completed a working memory task and an operant learning task, in which children acquired a response-sequence rule under either continuous or PRF (120 trials), followed by an extinction phase (80 trials). Data of 88 children were used for analysis.Results: The PRF extinction effect was confirmed: We observed slower acquisition and extinction in the PRF condition as compared to the continuous reinforcement (CRF) condition. Working memory was negatively related to acquisition but not extinction performance.Conclusion: Both reinforcement contingencies and working memory relate to acquisition performance. Potential implications for BPT are that decreasing working memory load may enhance the chance of optimally learning through reinforcement.

The effects of acute stress exposure on striatal activity during Pavlovian conditioning with monetary gains and losses.

Pavlovian conditioning involves the association of an inherently neutral stimulus with an appetitive or aversive outcome, such that the neutral stimulus itself acquires reinforcing properties. Across species, this type of learning has been shown to involve subcortical brain regions such as the striatum and the amygdala. It is less clear, however, how the neural circuitry involved in the acquisition of Pavlovian contingencies in humans, particularly in the striatum, is affected by acute stress. In the current study, we investigate the effect of acute stress exposure on Pavlovian conditioning using monetary reinforcers. Participants underwent a partial reinforcement conditioning procedure in which neutral stimuli were paired with high and low magnitude monetary gains and losses. A between-subjects design was used, such that half of the participants were exposed to cold stress while the remaining participants were exposed to a no stress control procedure. Cortisol measurements and subjective ratings were used as measures of stress. We observed an interaction between stress, valence, and magnitude in the ventral striatum, with the peak in the putamen. More specifically, the stress group exhibited an increased sensitivity to magnitude in the gain domain. This effect was driven by those participants who experienced a larger increase in circulating cortisol levels in response to the stress manipulation. Taken together, these results suggest that acute stress can lead to individual differences in circulating cortisol levels which influence the striatum during Pavlovian conditioning with monetary reinforcers.

Learned suppression of photopositive tendencies in Drosophila melanogaster.

A task was designed to teach individual flies to avoid a lighted area after they had displayed an initial preference for it. The flies walked in a T-maze and chose between a lighted and a darkened alley leading, respectively, to a lighted and a darkened vial. Flies that were photopositive on a first trial were subjected to an aversive stimulus (a filter paper inserted into the lighted vial and wetted with a quinine solution), and they performed 16 training trials; they learned to avoid the lighted vial. The flies trained with water instead of quinine in the lighted vial still display avoidance of the lighted vial, but to a lesser extent. The flies trained with a dry filter paper in the lighted vial did not show any increase in avoidance during training. Like the flies trained with no quinine at all, those trained to avoid the lighted vial under a partial reinforcement condition (one half of the trials with quinine, the other half with a dry vial) did not master the task. Finally, removal of the quinine after an avoidance acquisition criterion was reached resulted in an extinction process.

The partial reinforcement extinction effect in humans: effects of schizophrenia, schizotypy and low doses of amphetamine

The partial reinforcement extinction effect (PREE) was studied in human subjects. It has been suggested that the PREE depends on neural mechanisms critical to the cognitive dysfunction which underlines acute schizophrenia. We therefore predicted that the PREE should be reduced, through decreased resistance to extinction in the partial reinforcement (PR) condition, in various types of individual: (a) healthy volunteers given low doses of oral amphetamine; (b) those in the acute (but not chronic) phase of a schizophrenic illness and; (c) healthy volunteers with high scores on personality measures of schizotypy. Despite obtaining robust demonstrations of PREE in all experiments, none of these predictions were confirmed. A single, low dose, of amphetamine had no effect on either continuous reinforcement (CR) or partial reinforcement (PR). Acute and chronic schizophrenic patients showed a reduced PREE compared to controls. However this was due to increased resistance to extinction in the CR groups. Finally, high schizotypy scores were associated with greater PREE, attributable to both decreased extinction in the CR condition and increased extinction in the PR condition. The results of these experiments on human PREE provide no support that PREE is a valid paradigm with which to explore the cognitive dysfunction underlying schizophrenia.

Impaired fear conditioning in Alzheimer’s disease

Classical conditioning of the fear response is a basic form of nondeclarative (nonconscious) memory that mediates both normal and pathological responses to aversive stimuli. Because fear conditioning critically depends on the amygdala, a medial temporal lobe structure that frequently undergoes significant pathological changes early in the course of Alzheimer’s disease (AD), we hypothesized that fear conditioning would be impaired in patients with mild to moderate AD. We examined simple classical fear conditioning in a group of 10 patients with probable AD and 14 demographically matched, neurologically intact elderly controls. During conditioning, one stimulus (e.g. a green rectangle, the conditioned stimulus (CS+)), was paired with an aversive stimulus (a loud noise, the unconditioned stimulus (US)) using a partial reinforcement conditioning schedule. The opponent color (e.g. red rectangle), the CS−, was never paired with the US. The elderly controls acquired robust fear responses as demonstrated by their differential skin-conductance responses to the CS+ and CS−. In contrast, the AD group showed a marked impairment in conditioning, failing to exhibit significant conditioned fear responses. This failure to acquire conditioned responses could not be attributed to diminished responding by patients, relative to controls, to the aversive US. The results indicate that fear conditioning, an amygdala-dependent form of memory, is impaired in AD. These findings complement previous reports of impairments in declarative emotional memory in AD by demonstrating that a basic form of nondeclarative emotional memory is also impaired in AD.

Changes in Medial Prefrontal Cortical Dopamine Levels Associated with Response-Contingent Food Reward: An Electrochemical Study in Rat

Voltammetry was used to monitor in rats changes in medial prefrontal cortex (PFC) dopamine (DA) levels associated with response-contingent presentation of a condensed milk reward. During two initial training sessions, minor DA signal fluctuations were seen when animals consumed a standard 30 sec (0.2 ml) meal earned on a continuous reinforcement schedule. There was no evidence of experience-dependent changes in these fluctuations. Under delayed reinforcement conditions, lever-presses were followed by DA signal increases that were time-locked to the delay duration, and these were followed by signal decreases when animals eventually received the reward. Such decreases became more pronounced when the standard rate of milk delivery was tripled, but were attenuated when milk delivery was reduced to half the usual rate. Withholding earned milk resulted in signal increases. In contrast, DA signal increases were observed during milk consumption when the standard meal duration was unexpectedly shortened to 15 sec or lengthened to 60 or 90 sec. Orderly changes in DA signal were also observed under partial reinforcement conditions. Unreinforced responses were associated with DA signal decreases, whereas transient increases were seen during the 30 sec meal that followed reinforced responses. These findings indicate that response-contingent reward presentation elicits synchronous changes in PFC DA transmission. They also suggest that the DA input to PFC is activated when rewards are presented under conditions that deviate from those that the animals had come to expect, particularly so when the temporal structure of learned associations is altered.

Behavior-Relevant Changes in Nucleus Accumbens Dopamine Transmission Elicited by Food Reinforcement: An Electrochemical Study in Rat

Voltammetry was used to monitor dopamine (DA) transmission in nucleus accumbens (NAcc) of rats lever-pressing for food. Under standard conditions, animals responded on a fixed ratio 1 (FR1) schedule for 0.2 ml of milk delivered over 30 sec; milk delivery was paired with a 30 sec cue light. Consumption of the initial few milk rewards of the session caused DA signal increases. These initial signal increases were typical of the first and, at times, the second test days. On subsequent days, the most pronounced initial signal increases coincided with presentation of conditioned stimuli that marked the start of the session. Biphasic changes in DA signal that were time-locked to each reinforced lever-press were also observed; responses were preceded by increases and were followed, during milk consumption, by decreases in DA signal. At the end of milk delivery, the signal increased again in apparent anticipation of the next lever-press. Delaying milk delivery caused a corresponding delay in DA signal decreases, and the amount of time signals remained depressed was bound by the duration of milk consumption. Greater decreases in DA signal were observed when the rate of milk delivery was doubled or tripled, and such increases in reward value were associated with more pronounced signal increases during the period that preceded each lever-press. In contrast, DA signal increases were seen when milk was delivered at half the usual rate or was withheld altogether or when animals were denied access to the lever. Under partial reinforcement conditions, reinforced lever-presses were preceded by more pronounced signal increases and decreases of comparable magnitude accompanied milk consumption. These results suggest that meso-NAcc DA neurons are activated primarily in response to the incentive rather than to the reinforcing properties of rewards.

Choice with certain and uncertain reinforcers in an adjusting-delay procedure.

A discrete-trials adjusting-delay procedure was used to investigate the conditions under which pigeons might show a preference for partial reinforcement over 100% reinforcement, an effect reported in a number of previous experiments. A peck on a red key always led to a delay with red houselights and then food. In each condition, the duration of the red-houselight delay was adjusted to estimate an indifference point. In 100% reinforcement conditions, a peck on a green key always led to a delay with green houselights and then food. In partial-reinforcement conditions, a peck on the green key led either to the green houselights and food or to white houselights and no food. In some phases of the experiment, statistically significant preference for partial reinforcement over 100% reinforcement was found, but this effect was observed in only about half of the pigeons. The effect was largely eliminated when variability in the delay stimulus colors was equated for 50% reinforcement conditions and 100% reinforcement conditions. Idiosyncratic preferences for certain colors or for stimulus variability may be at least partially responsible for the effect.

A single dose of nicotine proactively enhances the partial reinforcement extinction effect in the rat

On the basis of previous data showing (1) the dependence of the partial reinforcement extinction effect (PREE) upon the release of noradrenaline (NA) within the hippocampal formation and (2) a long-term proactive increase in intrahippocampal NA release after systemic treatment with nicotine, it was predicted that this compound would proactively enhance the PREE. The PREE was measured in rats as running speed in the straight alley after training on continuous reinforcement (CRF) or partial reinforcement (PRF) schedules of food reward, the PREE consisting in increased resistance to extinction of the running response after PRF relative to CRF training. The prediction was confirmed, largely reflecting a nicotine-induced increase in resistance to extinction in the PRF condition, in three experiments: a single injection of nicotine (0.8 mg/kg s.c.) preceding runway training and extinction at one trial a day, seven daily injections of nicotine preceding the same paradigm, and a single injection of nicotine preceding training and extinction at six trials a day. The observed behavioral changes occurred up to a month after nicotine administration, a time course similar to that seen in previous neurochemical experiments. The effects of the one- and seven-injection regimes on the PREE were statistically indistinguishable, also in agreement with previous neurochemical data.

Erratum to: Dependence of the proactive behavioral effects of theta-driving septal stimulation on stimulation frequency and behavioral experience: 2. Continuously and partially reinforced running

Two experiments were conducted with Sprague-Dawley rats, chronically implanted with medial septal stimulating and subicular recording electrodes, to test the hypothesis that low-frequency electrical stimulation of the septal area, driving the hippocampal theta rhythm, has frequency-dependent proactive efects on behavior: specifically, that theta-driving stimulation at 7.5 Hz gives rise to tolerance for subsequent frustrative nonreward, whereas stimulation at 7.7 Hz sensitizes the animal to this event. After a 10-day stimulation regime (fifteen 6-sec stimulation trains at a 30-sec intertrain interval daily; 7.5 Hz in Experiment 1, 7.7 Hz in Experiment 2), rats were trained to run in a straight alley for food reward at 1 trial/day for 16 days on either a continuous reinforcement (CRF) or a partial reinforcement (PRF) reinforcement schedule followed by extinction; implanted controls were not stimulated but were otherwise treated identically. As predicted by the hypothesis, the partial reinforcement extinction effect (PREE; i.e., greater resistance to extinction after PRF than after CRF training) was proactively reduced by 7.5-Hz stimulation, owing to lowered resistance to extinction in the stimulated PRF-trained animals relative to their nonstimulated controls; whereas, again as predicted, the PREE was increased by 7.7-Hz stimulation, owing both to increased resistance to extinction in the stimulated PRF condition and reduced resistance to extinction in the stimulated CRF condition relative to the respective non-stimulated control groups. Thus the proactive behavioral effects of 7.5- and 7.7-Hz stimulation were opposite in sign, dependent in both cases upon the schedule of reinforcement encountered subsequent to stimulation, and consistent, respectively, with stimulation-induced tolerance for nonreward (7.5 Hz) or sensitization of the response to nonreward (7.7 Hz).

Dependence of the proactive behavioral effects of theta-driving septal stimulation on stimulation frequency and behavioral experience: 2. Continuously and partially reinforced running

Two experiments were conducted with Sprague-Dawley rats, chronically implanted with medial septal stimulating and subicular recording electrodes, to test the hypothesis that low-frequency electrical stimulation of the septal area, driving the hippocampal theta rhythm, has frequency-dependent proactive efects on behavior: specifically, that theta-driving stimulation at 7.5 Hz gives rise to tolerance for subsequent frustrative nonreward, whereas stimulation at 7.7 Hz sensitizes the animal to this event. After a 10-day stimulation regime (fifteen 6-sec stimulation trains at a 30-sec intertrain interval daily; 7.5 Hz in Experiment 1, 7.7 Hz in Experiment 2), rats were trained to run in a straight alley for food reward at 1 trial/day for 16 days on either a continuous reinforcement (CRF) or a partial reinforcement (PRF) reinforcement schedule followed by extinction; implanted controls were not stimulated but were otherwise treated identically. As predicted by the hypothesis, the partial reinforcement extinction effect (PREE; i.e., greater resistance to extinction after PRF than after CRF training) was proactively reduced by 7.5-Hz stimulation, owing to lowered resistance to extinction in the stimulated PRF-trained animals relative to their nonstimulated controls; whereas, again as predicted, the PREE was increased by 7.7-Hz stimulation, owing both to increased resistance to extinction in the stimulated PRF condition and reduced resistance to extinction in the stimulated CRF condition relative to the respective non-stimulated control groups. Thus the proactive behavioral effects of 7.5- and 7.7-Hz stimulation were opposite in sign, dependent in both cases upon the schedule of reinforcement encountered subsequent to stimulation, and consistent, respectively, with stimulation-induced tolerance for nonreward (7.5 Hz) or sensitization of the response to nonreward (7.7 Hz).

DISCRIMINATION TRAINING, PARTIAL REINFORCEMENT, AND INCREASES IN INTERTRIAL INTERVAL ALL REDUCE RESPONSE SPEED IN A CONTINUOUSLY REINFORCED KEY‐PECKING TASK

Pigeons were trained in a discrete-trial task in which a response to the center key obtained illumination of a side key and a single response to the side key terminated a trial with either reinforcement or nonreinforcement. Center-key speeds (i.e., reciprocals of latencies) declined with increases in intertrial interval, and it is argued that this effect is related to a decreased likelihood as intertrial interval increases that birds will be near the key at trial onset. Side-key speeds on trials with reinforcement decreased both with increases in intertrial interval and with shifts from continuous reinforcement to either a discrimination or a partial-reinforcement condition. The effects on side-key speeds are compared with effects observed in alley-running tasks using rats, and an interpretation in terms of frustration theory is offered for the results obtained in both types of task.

Sympathovagal interaction in stress and relaxation; heart rate variability in home nurses

Classical conditioning of the fear response is a basic form of nondeclarative (nonconscious) memory that mediates both normal and pathological responses to aversive stimuli. Because fear conditioning critically depends on the amygdala, a medial temporal lobe structure that frequently undergoes significant pathological changes early in the course of Alzheimer’s disease (AD), we hypothesized that fear conditioning would be impaired in patients with mild to moderate AD. We examined simple classical fear conditioning in a group of 10 patients with probable AD and 14 demographically matched, neurologically intact elderly controls. During conditioning, one stimulus (e.g. a green rectangle, the conditioned stimulus (CS+)), was paired with an aversive stimulus (a loud noise, the unconditioned stimulus (US)) using a partial reinforcement conditioning schedule. The opponent color (e.g. red rectangle), the CS−, was never paired with the US. The elderly controls acquired robust fear responses as demonstrated by their differential skin-conductance responses to the CS+ and CS−. In contrast, the AD group showed a marked impairment in conditioning, failing to exhibit significant conditioned fear responses. This failure to acquire conditioned responses could not be attributed to diminished responding by patients, relative to controls, to the aversive US. The results indicate that fear conditioning, an amygdala-dependent form of memory, is impaired in AD. These findings complement previous reports of impairments in declarative emotional memory in AD by demonstrating that a basic form of nondeclarative emotional memory is also impaired in AD.

Attribute × Treatment Interaction Analysis of the Processes Underlying Food Orientation in the Double Runway

Abstract A two-step procedure, the first exploratory and the second confirmatory, was proposed for the study of attribute × treatment interactions (ATIs). ATI patterns resulting from the exploratory step suggested a number of hypotheses underlying food orientation in the double runway. The confirmatory step for testing those hypotheses was detailed. A second-order general running time factor pervaded all sessions. In addition, three first-order factors indicated that the processes underlying acquisition and extinction are symmetrical and perhaps activated by the relative strengths of hunger and exploratory disposition as they change over sessions. A fourth first-order factor seemed to measure emotional reactivity to thwarted expectancies and the ambiguity inherent in conditions of partial reinforcement.

Oddity of visual patterns conceptualized by pigeons

Pigeons were trained to learn an instrumental oddity-from-sample discrimination involving visual forms. One group, the few examples group, dealt with 5 patterns in 40 different com­ binations. Another group, the examples group, dealt with 20 patterns in 160 different combinations. After both groups had reached asymptotic performance and had learned to oper­ ate under partial reinforcement conditions, they were tested for transfer under extinction con­ ditions with two different groups of 5 novel patterns, each in 40 combinations. All animals showed significant above chance transfer to both of these novel stimulus sets. Transfer perfor­ mance with test stimuli of similar geometric design to training stimuli was better than perfor­ mance with stimuli of markedly different design. The transfer performance of the ex­ amples group was marginally better than that of the few examples group, even though the latter's performance on the training stimuli was better throughout. It is concluded that pigeons can learn to employ an oddity concept and that this may be promoted by the use of many train­ ing exemplars. Furthermore, it is inferred that pigeons may normally use a mixture of strategies to solve oddity and identity problems. The matching-to-sample discrimination paradigm, in which subjects must decide which of two comparison stimuli is identical with a previously presented sample stimulus, has been extensively used with pigeons since Cumming and Berryman (1961) developed an efficient version for use with this species. But there still seems to be doubt about whether pigeons can learn the general principles of identity and oddity that are conducive to an efficient mastery of the task. Carter and Werner (1978) reviewed the evidence extant up to 1978 and decided that pigeons did not learn identity/oddity concepts, but rather were conditioned to apply discrete rules of a stimulus-response type. Soon afterwards, Zen tall and Hogan (1978) came to the opposite conclu­ sion. The empirical evidence that has accrued since then has been equivocal, sometimes supporting one posi­ tion, sometimes the other (e.g., Holmes, 1979; Wright & Sands, 1981; Zentall, Edwards, Moore, & Hogan, 1981). Two recent reviewers (Mackintosh, 1983; Macphail, 1982) accordingly reached confliciting, but re­ served, opinions about pigeons' capabilities in this matter.

The strength of the orienting response during Pavlovian conditioning.

The strength of orienting toward a light that signaled food was studied in five experiments. Experiment 1 demonstrated that this response declined in strength during conditioning but was temporarily restored during extinction. In Experiment 2 the light was again paired immediately with the unconditioned stimulus (US), whereas in Experiments 3 and 4 it signaled a tone which in turn signaled the US. In these three experiments we again found that continuous reinforcement resulted in a decline in the strength of light orientation. These studies also revealed that under conditions of partial reinforcement, orientation to the light was sustained. Experiment 5 demonstrated that the decline in light orientation with a continuous reinforcement procedure can be retarded either by preexposing the light for a number of trials prior to conditioning or by intermixing reinforced light trials with nonreinforced presentations of a tone. This experiment also revealed that reversing the reinforcement contingency associated with the tone restored orientation to the light. This pattern of results can be most readily explained by the proposal that the strength of orientation toward the light is inversely related to the predictive accuracy of this stimulus.

Effects of pimozide on lever pressing behavior maintained on an intermittent reinforcement schedule

Lever pressing for food on a variable interval (2.5 min) schedule was challenged by pretreatment with a 1.0 mg/kg dose of the dopamine receptor blocker pimozide. Large decreases in response rate were recorded even during the first few minutes of the test session before the rats ahd received any reinforcement. Pimozide also caused extinction-like effects, but it was clear, from comparisons between pimozide-treated rats that were rewarded and pimozide-treated rats that were not rewarded, that the rewarding effects of food were not totally blocked. It is suggested that an important aspect of the pimozide-produced response decrement is its effect on the incentive motivational properties of food-associated apparatus cues known to be important in sustaining responding under extinction and partial reinforcement conditions.

Effects of Partial and Continuous Reinforcement on Conditioned Heart Rate and Corneoretinal Potential Responses in the Rabbit (Oryctolagus Cuniculus)

Corneoretinal potential (CRP) and heart rate (HR) changes were assessed in a simple classical conditioning experiment in which different groups of rabbits received (a) partial reinforcement, (b) continuous reinforcement, or (c) random CS-UCS presentations. The results showed that the initial conditioned heart rate response, under both partial and continuous reinforcement conditions, was a deceleration in rate. However, after considerable experience this initial deceleration changed to an acceleration in the continuous reinforcement group; accelerative CRS were minimal under partial reinforcement conditions. Although partial reinforcement lowered CRP response frequency, it enhanced HR decelerative CRS. Several possible explanations for these somatic-autonomic response system divergencies are discussed.