Pavlovian Contingencies Research Articles

Mesostriatal dopamine is sensitive to changes in specific cue-reward contingencies.

Learning causal relationships relies on understanding how often one event precedes another. To investigate how dopamine neuron activity and neurotransmitter release change when a retrospective relationship is degraded for a specific pair of events, we used outcome-selective Pavlovian contingency degradation in rats. Conditioned responding was attenuated for the cue-reward contingency that was degraded, as was dopamine neuron activity in the midbrain and dopamine release in the ventral striatum in response to the cue and subsequent reward. Contingency degradation also abolished the trial-by-trial history dependence of the dopamine responses at the time of trial outcome. This profile of changes in cue- and reward-evoked responding is not easily explained by a standard reinforcement learning model. An alternative model based on learning causal relationships was better able to capture dopamine responses during contingency degradation, as well as conditioned behavior following optogenetic manipulations of dopamine during noncontingent rewards. Our results suggest that mesostriatal dopamine encodes the contingencies between meaningful events during learning.

Autoshaped impulsivity: Some explorations with a neural network model

This study evaluated the effect of delay and magnitude of reinforcement in Pavlovian contingencies, extending the understanding of the phenomenon of autoshaped impulsivity as described in Alcalá's thesis (2017) and Burgos and García-Leal (2015). The effects of adding a trace interval were analyzed on the maintained responses of impulsive choice, seen as the preference of a small and immediate reinforcer over a larger and delayed one, and the role of the contextual unit, as well as the inhibitory units according to the Diffuse Discrepancy Model. In the Simulation, the model with inhibitory units was used, trained in two signals with different delays and reinforcement magnitudes, and subsequently presented concurrently in choice tasks without reinforcement nor learning, using an ABA within-subject design. In general, the DD model successfully simulated the phenomenon of autoshaped impulsivity, consistent with studies from Alcalá's thesis (2017), Burgos and García-Leal (2015), and Picker and Poling (1982). It also predicted the elimination of this effect (autoshaped impulsivity) after introducing a trace interval. The observed results and their implications are discussed, as well as possible future studies with animals and humans.

The mediodorsal thalamus supports adaptive responding based on stimulus-outcome associations.

The ability to engage into flexible behaviors is crucial in dynamic environments. We recently showed that in addition to the well described role of the orbitofrontal cortex (OFC), its thalamic input from the submedius thalamic nucleus (Sub) also contributes to adaptive responding during Pavlovian degradation. In the present study, we examined the role of the mediodorsal thalamus (MD) which is the other main thalamic input to the OFC. To this end, we assessed the effect of both pre- and post-training MD lesions in rats performing a Pavlovian contingency degradation task. Pre-training lesions mildly impeded the establishment of stimulus-outcome associations during the initial training of Pavlovian conditioning without interfering with Pavlovian degradation training when the sensory feedback provided by the outcome rewards were available to animals. However, we found that both pre- and post-training MD lesions produced a selective impairment during a test conducted under extinction conditions, during which only current mental representation could guide behavior. Altogether, these data suggest a role for the MD in the successful encoding and representation of Pavlovian associations.

Evidence for model-based encoding of Pavlovian contingencies in the human brain

Prominent accounts of Pavlovian conditioning successfully approximate the frequency and intensity of conditioned responses under the assumption that learning is exclusively model-free; that animals do not develop a cognitive map of events. However, these model-free approximations fall short of comprehensively capturing learning and behavior in Pavlovian conditioning. We therefore performed multivoxel pattern analysis of high-resolution functional MRI data in human participants to test for the encoding of stimulus-stimulus associations that could support model-based computations during Pavlovian conditioning. We found that dissociable sub-regions of the striatum encode predictions of stimulus-stimulus associations and predictive value, in a manner that is directly related to learning performance. Activity patterns in the orbitofrontal cortex were also found to be related to stimulus-stimulus as well as value encoding. These results suggest that the brain encodes model-based representations during Pavlovian conditioning, and that these representations are utilized in the service of behavior.

Temporal specificity in Pavlovian-to-instrumental transfer.

Presentation of a previously trained Pavlovian conditioned stimulus while an organism is engaged in operant responding can moderate the rate of responding, a phenomenon known as Pavlovian-to-instrumental transfer. Although it is well known that Pavlovian contingencies will generate conditioned behavior that is temporally organized with respect to the arrival of the predicted outcome, little work has examined the temporal dynamics of responding during Pavlovian-instrumental transfer. We trained rats using a fixed time 60-sec, fixed time 120-sec, or random time 60-sec schedule in an appetitive Pavlovian task, and found that presentation of the conditioned stimulus potentiated operant responding in a manner that reflected these previously established temporal expectancies. Further, this temporal specificity conformed to the scalar property as seen with other forms of interval timing behavior. Surprisingly, this effect was only seen when the conditioned stimulus was a visual cue, but not when it was an auditory cue. These data suggest that the motivational processes triggered by Pavlovian cues are not static, but fluctuate in strength as a function of temporally specific expectations of reward.

I Think, Therefore Eyeblink

Can conditioning occur without conscious awareness of the contingency between the stimuli? We trained participants on two separate reaction time tasks that ensured attention to the experimental stimuli. The tasks were then interleaved to create a differential Pavlovian contingency between visual stimuli from one task and an airpuff stimulus from the other. Many participants were unaware of the contingency and failed to show differential eyeblink conditioning, despite attending to a salient stimulus that was contingently and contiguously related to the airpuff stimulus over many trials. Manipulation of awareness by verbal instruction dramatically increased awareness and differential eyeblink responding. These findings cast doubt on dual-system theories, which propose an automatic associative system independent of cognition, and provide strong evidence that cognitive processes associated with awareness play a causal role in learning.

Flexible Use of Predictive Cues beyond the Orbitofrontal Cortex: Role of the Submedius Thalamic Nucleus.

The orbitofrontal cortex (OFC) is known to play a crucial role in learning the consequences of specific events. However, the contribution of OFC thalamic inputs to these processes is largely unknown. Using a tract-tracing approach, we first demonstrated that the submedius nucleus (Sub) shares extensive reciprocal connections with the OFC. We then compared the effects of excitotoxic lesions of the Sub or the OFC on the ability of rats to use outcome identity to direct responding. We found that neither OFC nor Sub lesions interfered with the basic differential outcomes effect. However, more specific tests revealed that OFC rats, but not Sub rats, were disproportionally relying on the outcome, rather than on the discriminative stimulus, to guide behavior, which is consistent with the view that the OFC integrates information about predictive cues. In subsequent experiments using a Pavlovian contingency degradation procedure, we found that both OFC and Sub lesions produced a severe deficit in the ability to update Pavlovian associations. Altogether, the submedius therefore appears as a functionally relevant thalamic component in a circuit dedicated to the integration of predictive cues to guide behavior, previously conceived as essentially dependent on orbitofrontal functions. Significance statement: In the present study, we identify a largely unknown thalamic region, the submedius nucleus, as a new functionally relevant component in a circuit supporting the flexible use of predictive cues. Such abilities were previously conceived as largely dependent on the orbitofrontal cortex. Interestingly, this echoes recent findings in the field showing, in research involving an instrumental setup, an additional involvement of another thalamic nuclei, the parafascicular nucleus, when correct responding requires an element of flexibility (Bradfield et al., 2013a). Therefore, the present contribution supports the emerging view that limbic thalamic nuclei may contribute critically to adaptive responding when an element of flexibility is required after the establishment of initial learning.

Pavlovian conditioning enhances resistance to disruption of dogs performing an odor discrimination.

Domestic dogs are used to aid in the detection of a variety of substances such as narcotics and explosives. Under real-world detection situations there are many variables that may disrupt the dog's performance. Prior research on behavioral momentum theory suggests that higher rates of reinforcement produce greater resistance to disruption, and that this is heavily influenced by the stimulus-reinforcer relationship. The present study tests the Pavlovian interpretation of resistance to change using dogs engaged in an odor discrimination task. Dogs were trained on two odor discriminations that alternated every six trials akin to a multiple schedule in which the reinforcement probability for a correct response was always 1. Dogs then received several sessions of either odor Pavlovian conditioning to the S+ of one odor discrimination (Pavlovian group) or explicitly unpaired exposure to the S+ of one odor discrimination (Unpaired group). The remaining odor discrimination pair for each dog always remained an unexposed control. Resistance to disruption was assessed under presession feeding, a food-odor disruptor condition, and extinction, with baseline sessions intervening between disruption conditions. Equivalent baseline detection rates were observed across experimental groups and odorant pairs. Under disruption conditions, Pavlovian conditioning led to enhanced resistance to disruption of detection performance compared to the unexposed control odor discrimination. Unpaired odor conditioning did not influence resistance to disruption. These results suggest that changes in Pavlovian contingencies are sufficient to influence resistance to change.

Extinction Can Reduce the Impact of Reward Cues on Reward-Seeking Behavior

Reward-associated cues are thought to promote relapse after treatment of appetitive disorders such as drug-taking, binge eating, and gambling. This process has been modelled in the laboratory using a Pavlovian-instrumental transfer (PIT) design in which Pavlovian cues facilitate instrumental reward-directed action. Attempts to reduce facilitation by cue exposure (extinction) have produced mixed results. We tested the effect of extinction in a recently developed PIT procedure using a natural reward, chocolate, in human participants. Facilitation of instrumental responding was only observed in participants who were aware of the Pavlovian contingencies. Pavlovian extinction successfully reduced, but did not completely eliminate, expectancy of reward and facilitation of instrumental responding. The results indicate that exposure can reduce the ability of cues to promote reward-directed behavior in the laboratory. However, the residual potency of extinguished cues means that additional active strategies may be needed in clinical practice to train patients to resist the impact of these cues in their environment.

Autoshaped choice in artificial neural networks: Implications for behavioral economics and neuroeconomics

An existing neural network model of conditioning was used to simulate autoshaped choice. In this phenomenon, pigeons first receive an autoshaping procedure with two keylight stimuli X and Y separately paired with food in a forward-delay manner, intermittently for X and continuously for Y. Then pigeons receive unreinforced choice test trials of X and Y concurrently present. Most pigeons choose Y. This preference for a more valuable response alternative is a form of economic behavior that makes the phenomenon relevant to behavioral economics. The phenomenon also suggests a role for Pavlovian contingencies in economic behavior. The model used, in contrast to others, predicts autoshaping and automaintenance, so it is uniquely positioned to predict autoshaped choice. The model also contemplates neural substrates of economic behavior in neuroeconomics, such as dopaminergic and hippocampal systems. A feedforward neural network architecture was designed to simulate a neuroanatomical differentiation between two environment-behavior relations X-R1 and X-R2, where R1 and R2 denote two different emitted responses (not unconditionally elicited by the reward). Networks with this architecture received a training protocol that simulated an autoshaped-choice procedure. Most networks simulated the phenomenon. Implications for behavioral economics and neuroeconomics, limitations, and the issue of model appraisal are discussed.

Dissociable effects of anterior and mediodorsal thalamic lesions on spatial goal-directed behavior.

Goal-directed behaviors are thought to be supported by a neural circuit encompassing the prefrontal cortex, the dorsomedial striatum, the amygdala, and, as more recently suggested, the limbic thalamus. Since evidence indicates that the various thalamic nuclei contribute to dissociable functions, we directly compared the functional contribution of the mediodorsal thalamus (MD) and of the anterior thalamic nuclei (ATN) in a new task assessing spatial goal-directed behavior in a cross-maze. Rats sustaining lesions of the mediodorsal or the anterior thalamus were trained to associate each of the two goal arms with a distinctive food reward. Unlike control rats, both lesioned groups failed to express a bias for the goal arm corresponding to the non-devalued outcome following devaluation by sensory-specific satiety. In addition, MD rats were slower than the other groups to complete the trials. When tested for spatial working memory using a standard non-matching-to-place procedure in the same apparatus, ATN rats were severely impaired but MD rats performed as well as controls, even when spatial or temporal challenges were introduced. Finally, all groups displayed comparable breaking points in a progressive ratio test, indicating that the slower choice performance of MD rats did not result from motivational factors. Thus, a spatial task requiring the integration of instrumental and Pavlovian contingencies reveals a fundamental deficit of MD rats in adapting their choice according to goal value. By contrast, the deficit associated with anterior thalamic lesions appears to simply reflect the inability to process spatial information.

The effects of acute stress exposure on striatal activity during Pavlovian conditioning with monetary gains and losses.

Pavlovian conditioning involves the association of an inherently neutral stimulus with an appetitive or aversive outcome, such that the neutral stimulus itself acquires reinforcing properties. Across species, this type of learning has been shown to involve subcortical brain regions such as the striatum and the amygdala. It is less clear, however, how the neural circuitry involved in the acquisition of Pavlovian contingencies in humans, particularly in the striatum, is affected by acute stress. In the current study, we investigate the effect of acute stress exposure on Pavlovian conditioning using monetary reinforcers. Participants underwent a partial reinforcement conditioning procedure in which neutral stimuli were paired with high and low magnitude monetary gains and losses. A between-subjects design was used, such that half of the participants were exposed to cold stress while the remaining participants were exposed to a no stress control procedure. Cortisol measurements and subjective ratings were used as measures of stress. We observed an interaction between stress, valence, and magnitude in the ventral striatum, with the peak in the putamen. More specifically, the stress group exhibited an increased sensitivity to magnitude in the gain domain. This effect was driven by those participants who experienced a larger increase in circulating cortisol levels in response to the stress manipulation. Taken together, these results suggest that acute stress can lead to individual differences in circulating cortisol levels which influence the striatum during Pavlovian conditioning with monetary reinforcers.

Expectancy, Ambiguity, and Behavioral Flexibility: Separable and Complementary Roles of the Orbital Frontal Cortex and Amygdala in Processing Reward Expectancies

Appetitive goal-directed behavior can be associated with a cue-triggered expectancy that it will lead to a particular reward, a process thought to depend on the OFC and basolateral amygdala complex. We developed a biologically informed neural network model of this system to investigate the separable and complementary roles of these areas as the main components of a flexible expectancy system. These areas of interest are part of a neural network with additional subcortical areas, including the central nucleus of amygdala, ventral (limbic) and dorsomedial (associative) striatum. Our simulations are consistent with the view that the amygdala maintains Pavlovian associations through incremental updating of synaptic strength and that the OFC supports flexibility by maintaining an activation-based working memory of the recent reward history. Our model provides a mechanistic explanation for electrophysiological evidence that cue-related firing in OFC neurons is nonselectively early after a contingency change and why this nonselective firing is critical for promoting plasticity in the amygdala. This ambiguous activation results from the simultaneous maintenance of recent outcomes and obsolete Pavlovian contingencies in working memory. Furthermore, at the beginning of reversal, the OFC is critical for supporting responses that are no longer inappropriate. This result is inconsistent with an exclusive inhibitory account of OFC function.

Competition between an avoidance response and a safety signal: Evidence for a single learning system

Two experiments examined competition between an instrumental avoidance response and a Pavlovian safety signal for association with omission of electric shock in a human fear conditioning paradigm. Self-reported shock expectancies and skin conductance responses were consistent with blocking of learning of the instrumental contingency by prior training of the Pavlovian contingency, and vice versa. The results support the idea that a common learning mechanism underlies both Pavlovian and instrumental conditioning. The expectancy data suggest that this learning mechanism is cognitive in nature, and that Pavlovian and instrumental learning involve external and internal attributions, respectively. The procedure may thus serve as a laboratory model for attributional processes involved in the acquisition of threat expectancies in anxiety and anxiety disorders.

The “lunching” effect: Pigeons track motion towards food more than motion away from it

Four experiments measured pigeons' pecking at a small touch-screen image (the CS) that moved towards or away from a source of food (the US). The image's effectiveness as a CS was dependent on its motion, direction, and distance relative to the US. Pecking to the CS increased with proximity to the US when the CS moved towards the US (Experiment 1). This held true even when a departing CS signalled a US of greater magnitude (Experiment 2). Response rates to stationary stimuli were greater the closer they were to the hopper; but rate was less than when the same spot was part of a motion towards food, and greater than when it was part of a motion away (Experiment 3). The rate of responding in all three cases (motion towards, stationary, motion away) decreased exponentially with distance from the hopper. The distance and motion effects observed under these Pavlovian contingencies were different when pecking to the spot was required for reinforcement (Experiment 4).

Resistance to change within heterogeneous response sequences.

Three experiments investigated how instrumental and Pavlovian contingencies contribute to resistance to change (RTC) in different ordinal response positions within heterogeneous response sequences in pigeons. RTC in the initial and terminal response positions of a three-response sequence were compared in Experiment 1, which presented three colored key lights in succession in each trial; and in Experiment 2, which severely degraded Pavlovian contingencies by presenting the lights simultaneously at each ordinal position. Experiment 3 eliminated the instrumental contingency in a high-order sign-tracking procedure. When the instrumental contingency was in effect, RTC of the initial position was greater than the terminal position (Initial RTC > Terminal RTC) when the Pavlovian contingencies were strong and when they were degraded. When the instrumental contingency was eliminated, RTC patterns reversed, producing a graded pattern of RTC (Initial < Middle < Terminal). Current theoretical approaches (e.g., behavioral momentum theory, conditioned reinforcement, and motivational control of instrumental conditioning) cannot account for these results. An alternative approach (a gradient model) shows that obtained measures of RTC in heterogeneous sequences may reflect a combination of three dissociable processes.

Evidence for expectancy as a mediator of avoidance and anxiety in a laboratory model of human avoidance learning

A laboratory model was developed to study human avoidance learning. Participants could avoid an electric shock signalled by a 5-s conditioned stimulus (CS) by pressing one of a set of response buttons. Self-reported shock expectancy and skin conductance were recorded during a subsequent 10-s interval before shock. Shock expectancy declined when the correct avoidance response was learned and returned when the response was unavailable. Learning transferred to another shock CS. Parallel effects were observed on skin conductance once performance anxiety was controlled by requiring responding on all trials. Learning was faster when the Pavlovian contingencies were trained before introduction of the instrumental response. The results support a cognitive model of anxiety in which performance of an avoidance response reduces expectancy of an aversive outcome and thereby reduces anxiety.

Pavlovian-Instrumental Transfer of the Discriminative Stimulus Effects of Nicotine and Ethanol in Rats

To date, only 1 study has evaluated the impact of a Pavlovian drug conditional stimulus (CS) on operant responding. A within-subject operant 1-lever go/no-go (across sessions) design was used to evaluate the impact of Pavlovian contingencies on the discriminative stimulus effects of nicotine (0.4 mg/kg) and ethanol (800 mg/kg) in male Sprague Dawley rats. Drugs were administered 10 min before each acquisition and test session. One drug predicted sessions of food reinforcement and the other drug predicted sessions of non reinforcement; stimulus roles were counterbalanced. In Experiment 1 (n = 7), operant lever pressing was initially maintained on a VI-30 s schedule of food reinforcement. This phase was followed by 20 (CS+ vs. CS-) Pavlovian drug discrimination training sessions without the levers present. Two extinction tests revealed significantly more operant lever pressing under the CS+ drug conditions compared to the CS- drug conditions, suggesting evidence for Pavlovian-instrumental transfer. Operant training significantly strengthened stimulus control. In Experiment 2 (n = 7), the drugs functioned as operant drug discriminative stimuli first. Next, the predictive roles of the drug S D s and S Δ S were reversed under Pavlovian CS- and CS+ contingencies, respectively. The original stimulus control was significantly undermined but was not reversed. These studies suggest that Pavlovian drug-reinforcer contingencies embedded within the operant 3-term contingency may play a partial role in mediating the discriminative stimulus effects of drugs.

Contingency knowledge is necessary for learned motivated behaviour in humans: relevance for addictive behaviour

Many forms of human conditioned behaviour depend upon explicit knowledge of the predictive contingency between stimuli, responses and the reinforcer. However, it remains uncertain whether the conditioning of three key behaviours in drug addiction-selective attention, instrumental drug-seeking behaviour and emotional state--are dependent upon contingency knowledge. To test this possibility, we employed an avoidance procedure to generate rapidly these three forms of conditioned behaviour without incurring the methodological problems of drug conditioning. In two experiments, participants (16 students) were trained on a schedule in which one stimulus (S +) predicted the occurrence of a startling noise, which could be cancelled by performing an instrumental avoidance response. The allocation of attention to the S + and the rate and probability of the avoidance response in the presence of S + were measured. Following training, participants were tested for their knowledge of the stimulus-noise contingencies arranged in the study and rated the emotional qualities of the stimuli. Both experiments showed that S + gained control of selective attention, instrumental avoidance behaviour and subjective anxiety, but only in participants who reported explicit knowledge of the Pavlovian contingency between the S + and the startling noise. The implication of the present findings is that the control of selective attention, instrumental drug-seeking behaviour and emotional state by drug-paired stimuli is mediated by cognitive knowledge of the predictive contingency between the stimulus and the drug.

Human nicotine conditioning requires explicit contingency knowledge: is addictive behaviour cognitively mediated?

Two seemingly contrary theories describe the learning mechanisms that mediate human addictive behaviour. According to the classical incentive theories of addiction, addictive behaviour is motivated by a Pavlovian conditioned appetitive emotional response elicited by drug-paired stimuli. Expectancy theory, on the other hand, argues that addictive behaviour is mediated by an expectancy of the drug imparted by cognitive knowledge of the Pavlovian (predictive) contingency between stimuli (S+) and the drug and of the instrumental (causal) contingency between instrumental behaviour and the drug. The present paper reviewed human-nicotine-conditioning studies to assess the role of appetitive emotional conditioning and explicit contingency knowledge in mediating addictive behaviour. The studies reviewed here provided evidence for both the emotional conditioning and the expectancy accounts. The first source of evidence is that nicotine-paired S+ elicit an appetitive emotional conditioned response (CR), albeit only in participants who expect nicotine. Furthermore, the magnitude of this emotional state is modulated by nicotine deprivation/satiation. However, the causal status of the emotional response in driving other forms of conditioned behaviour remains undemonstrated. The second source of evidence is that other nicotine CRs, including physiological responses, self-administration, attentional bias and subjective craving, are also dependent on participants possessing explicit knowledge of the Pavlovian contingencies arranged in the experiment. In addition, several of the nicotine CRs can be brought about or modified by instructed contingency knowledge, demonstrating the causal status of this knowledge. Collectively, these data suggest that human nicotine conditioned effects are mediated by an explicit expectancy of the drug coupled with an appetitive emotional response that reflects the positive biological value of the drug. The implication of this conclusion is that treatments designed to modify the expected value of the drug may prove effective.