Pavlovian Mechanisms Research Articles

Individual differences could explain the failure in transitive inference formation in pigeons using probabilistic reinforcement.

In propositional logic, it is stated that "for if A is predicated for every B, and B for every C, A must necessarily be predicated of every C". Following a similar logical process, it can be said that If A > B and B > C, then A > C, this is called transitive inference (TI). Piaget developed a verbal task to evaluate TI in children. Subsequent studies adapted this task for animals using a conditioned discrimination between five-terms sequence of stimuli A + B-, B + C-, C + D-, and D + E-. If subjects prefer B over D during test, it is assumed that TI has occurred. In this experiment, we analyzed the effects of task complexity on TI by using a five-terms sequence of stimuli associated with probabilistic outcomes during training, in pigeons. Thus, both stimuli are reinforced in each pair but with different probability, 0.8 for + stimulus and 0.2 for the-stimulus. We found that performance during C + D- pair is impaired and preference in the test pair BD is affected. However, this impairment is dependent on individual differences in performance in C + D- pair. We compare our findings with previous research and conclude that Pavlovian mechanisms, as well as ordering of stimuli, can account for our findings.

Behavioural evidence for parallel outcome-sensitive and outcome-insensitive Pavlovian learning systems in humans.

There is a dichotomy in instrumental conditioning between goal-directed actions and habits that are distinguishable on the basis of their relative sensitivity to changes in outcome value. It is less clear whether a similar distinction applies in Pavlovian conditioning, where responses have been found to be predominantly outcome sensitive. To test for both devaluation insensitive and devaluation sensitive Pavlovian conditioning in humans, we conducted four experiments combining Pavlovian conditioning and outcome devaluation procedures while measuring multiple conditioned responses. Our results suggest that Pavlovian conditioning involves two distinct types of learning: one that learns the current value of the outcome which is sensitive to devaluation, and one that learns about the spatial localisation of the outcome which is insensitive to devaluation. Our findings have implications for the mechanistic understanding of Pavlovian conditioning and provide a more nuanced understanding of Pavlovian mechanisms that might contribute to a number of psychiatric disorders.

Nicotine-enhanced Pavlovian conditioned approach is resistant to omission of expected outcome

Conditioned stimuli contribute to the resilience of nicotine addiction in that nicotine-associated cues can influence smokers and promote relapse. These stimuli are thought to acquire incentive motivational properties through a Pavlovian mechanism, and this phenomenon can be measured in animals by observing conditioned approach to the conditioned stimulus (sign-tracking) or to the location of unconditioned stimulus delivery (goal-tracking). Goal-tracking is thought to be more flexible than sign-tracking in response to changes in expected outcome. Nicotine exposure can increase the expression of conditioned responses, and we hypothesized that animals exposed to nicotine would also exhibit less flexible conditioned responses after a change in the expected unconditioned stimulus. Adult male rats were exposed to nicotine (0.4mg/kg, s.c.) or saline before Pavlovian conditioned approach training sessions. After training, animals underwent test sessions that reduced (water substitution) or withheld (omission) the unconditioned stimulus (US, 20% sucrose). As expected, nicotine enhanced sign- and goal-tracking. Water substitution moderately and nonspecifically reduced both sign- and goal-tracking in all rats. In contrast, US omission only reduced goal-tracking, with robust effects in saline-exposed rats and smaller effects in nicotine-exposed rats. These data support the hypothesis that both sign-tracking and nicotine exposure confer behavioral inflexibility under US omission.

Associative Learning of Social Value in Dynamic Groups

Although humans live in societies that regularly demand engaging with multiple people simultaneously, little is known about social learning in group settings. In two experiments, we combined a Pavlovian learning framework with dyadic economic games to test whether blocking mechanisms support value-based social learning in the gain (altruistic dictators) and loss (greedy robbers) domains. Subjects first learned about an altruistic dictator, who subsequently made altruistic splits collectively with a partner. Results revealed that because the presence of the dictator already predicted the outcome, subjects did not learn to associate value with the partner. This social blocking effect was not observed in the loss domain: A kind robber’s partner, who could steal all the subjects’ money but stole little, acquired highly positive value—which biased subjects’ subsequent behavior. These findings reveal how Pavlovian mechanisms support efficient social learning, while also demonstrating that violations of social expectations can attenuate how readily these mechanisms are recruited.

The cognitive architecture of anxiety-like behavioral inhibition.

The combination of reward and potential threat is termed approach/avoidance conflict and elicits specific behaviors, including passive avoidance and behavioral inhibition (BI). Anxiety-relieving drugs reduce these behaviors, and a rich psychological literature has addressed how personality traits dominated by BI predispose for anxiety disorders. Yet, a formal understanding of the cognitive inference and planning processes underlying anxiety-like BI is lacking. Here, we present and empirically test such formalization in the terminology of reinforcement learning. We capitalize on a human computer game in which participants collect sequentially appearing monetary tokens while under threat of virtual “predation.” First, we demonstrate that humans modulate BI according to experienced consequences. This suggests an instrumental implementation of BI generation rather than a Pavlovian mechanism that is agnostic about action outcomes. Second, an internal model that would make BI adaptive is expressed in an independent task that involves no threat. The existence of such internal model is a necessary condition to conclude that BI is under model-based control. These findings relate a plethora of human and nonhuman observations on BI to reinforcement learning theory, and crucially constrain the quest for its neural implementation.

Prospective and Pavlovian mechanisms in aversive behaviour

Studying aversive behaviour is critical for understanding negative emotions and associated psychopathologies. However a comprehensive picture of the mechanisms underlying aversion is lacking, with associative learning theories focusing on Pavlovian reactions and decision-making theoretic approaches on prospective functions. We propose a computational model of aversion that combines goal-directed and Pavlovian forms of control into a unifying framework in which their relative importance is regulated by factors such as threat distance and controllability. Using simulations, we test whether the model can reproduce available empirical findings and discuss its relevance to understanding factors underlying negative emotions such as fear and anxiety. Furthermore, the specific method used to construct the model permits a natural mapping from its components to brain structure and function. Our model provides a basis for a unifying account of aversion that can guide empirical and interventional study contexts.

Models of morality

Moral dilemmas engender conflicts between two traditions: consequentialism, which evaluates actions based on their outcomes, and deontology, which evaluates actions themselves. These strikingly resemble two distinct decision-making architectures: a model-based system that selects actions based on inferences about their consequences; and a model-free system that selects actions based on their reinforcement history. Here, I consider how these systems, along with a Pavlovian system that responds reflexively to rewards and punishments, can illuminate puzzles in moral psychology.

Aversive pavlovian responses affect human instrumental motor performance.

In neuroscience and psychology, an influential perspective distinguishes between two kinds of behavioral control: instrumental (habitual and goal-directed) and Pavlovian. Understanding the instrumental-Pavlovian interaction is fundamental for the comprehension of decision-making. Animal studies (as those using the negative auto-maintenance paradigm), have demonstrated that Pavlovian mechanisms can have maladaptive effects on instrumental performance. However, evidence for a similar effect in humans is scarce. In addition, the mechanisms modulating the impact of Pavlovian responses on instrumental performance are largely unknown, both in human and non-human animals. The present paper describes a behavioral experiment investigating the effects of Pavlovian conditioned responses on performance in humans, focusing on the aversive domain. Results showed that Pavlovian responses influenced human performance, and, similar to animal studies, could have maladaptive effects. In particular, Pavlovian responses either impaired or increased performance depending on modulator variables such as threat distance, task controllability, punishment history, amount of training, and explicit punishment expectancy. Overall, these findings help elucidating the computational mechanisms underlying the instrumental-Pavlovian interaction, which might be at the base of apparently irrational phenomena in economics, social behavior, and psychopathology.

Excitotoxic amygdala lesions disrupt reinforcer devaluation effects on instrumental responding in rhesus monkeys

Event Abstract Back to Event Excitotoxic amygdala lesions disrupt reinforcer devaluation effects on instrumental responding in rhesus monkeys Sarah Rhodes1* and Elisabeth Murray1 1 NIMH, NIH, Laboratory of Neuropsychology, United States The amygdala plays an important role in appetitive emotional learning and memory and its involvement in representing the current value of a food outcome is well documented. For example, studies in which the value of a reward is temporarily reduced via satiety-specific reinforcer devaluation have shown that rats with lesions of the basolateral amygdala (BLA) fail to show the reduction in responding normally seen for a devalued reward. This effect has been observed in both Pavlovian and instrumental paradigms [1] indicating the BLA plays an integral role in the ability of the current outcome value to modulate behavior that is guided by either stimulus-outcome (S-O) or response-outcome (R-O) associations. Monkeys with amygdala lesions likewise show a disruption of satiety-specific devaluation effects in the context of object choices [2]. In the monkey studies, however, the task design left unresolved whether the instrumental R-O associations or S-O driven Pavlovian approach responses were guiding performance. To investigate the role of the amygdala specifically in the control of goal-directed behavior, we designed a task in which responding relied on instrumental control of behavior and could not be explained via Pavlovian mechanisms. Rhesus monkeys were trained to perform two different instrumental responses (tap and hold) on a touch-sensitive screen for two different food rewards. After evaluation of responding in baseline conditions they were tested under extinction for the effect of satiety-specific devaluation. Whereas unoperated controls showed a reduction in performance of the response associated with a devalued outcome relative to the nondevalued outcome, monkeys with excitotoxic amygdala lesions failed to show this shift in response preference. These preliminary data support the idea that the amygdala is required for updating reward value and is therefore essential for goal-directed behavior in appetitive contexts.

Effects of expectation on amygdala‐kindled convulsions and interictal behaviour in rats: confirmation of a Pavlovian mechanism

We found evidence that the effects of exposure to the stimulation environment on the convulsions and interictal behaviour of basolateral amygdala (BA)-kindled rats are the result of Pavlovian conditioning. In Experiment 1, the rats first received 45 BA stimuli in one environment (CS+) and 45 sham stimuli in another environment (CS-). During this first phase of the experiment, the CS+ and CS- began to exert differential effects on interictal behaviour and convulsions. Then, during the second phase, rats received 45 stimuli and 45 sham stimuli, either as before in their original CS+ and CS- (nonreversal rats) or with their original CS+ and CS- interchanged (reversal rats). During the second phase, the CS+ and CS- continued to exert the same pattern of differential effects on the interictal behaviour and convulsions of the nonreversal rats, whereas the reversal rats began to display the reverse pattern. In Experiment 2, rats received 60 sham stimuli in one environment (CS1). Then, half of the rats received 30 stimuli in a second environment (CS2) and 30 sham stimuli in CS1 (no-pre-exposure rats), whereas the other half (pre-exposure rats) received 30 stimuli and 30 sham stimuli in CS1. Pre-exposure to the stimulation environment attenuated the kindled convulsions. These findings confirm that the conditioned effects of kindling are mediated by a Pavlovian mechanism.

In search of the neurobiological underpinnings of the differential outcomes effect.

Correlating unique rewards with to-be-remembered events (the Differential Outcomes Procedure [DOP]) enhances learning and memory performance in a range of species. Recently, we have demonstrated that the DOP can be used to reduce or eliminate the learning and memory impairments associated with animal models of amnesia and dementia. This powerful phenomenon, the Differential Outcomes Effect (DOE), has led to the question: How does such a simple manipulation exert such dramatic influence on learning and memory performance? A revised two-process account of the DOE states that using the DOP results in the activation of reward expectancies through Pavlovian mechanisms. The use of unique reward expectancies alters the nature of cognitive processing used to solve discrimination tasks. The change in cognitive processing is represented by utilization of a different memory system than that commonly used to acquire and remember information when a Nondifferential Outcomes Procedure (NOP) is used. Using neurochemical manipulations, it has been demonstrated that different, potentially independent, brain systems modulate memory performance when subjects are trained with a NOP versus a DOP. This memory-based DOP/NOP distinction resembles other dissociative memory theories in which two psychological processes are purportedly served by distinct neurobiological mechanisms. In addition, such results have important ramifications for the treatment of memory disorders because they demonstrate that stimulus and behavioral manipulations, like drugs, can influence neurotransmitter functioning.

Pavlovian feed-forward mechanisms in the control of social behavior.

The conceptual and investigative tools for the analysis of social behavior can be expanded by integrating biological theory, control systems theory, and Pavlovian conditioning. Biological theory has focused on the costs and benefits of social behavior from ecological and evolutionary perspectives. In contrast, control systems theory is concerned with how machines achieve a particular goal or purpose. The accurate operation of a system often requires feed-forward mechanisms that adjust system performance in anticipation of future inputs. Pavlovian conditioning is ideally suited to subserve this function in behavioral systems. Pavlovian mechanisms have been demonstrated in various aspects of sexual behavior, maternal lactation, and infant suckling. Pavlovian conditioning of agonistic behavior has been also reported, and Pavlovian processes may likewise be involved in social play and social grooming. Several further lines of evidence indicate that Pavlovian conditioning can increase the efficiency and effectiveness of social interactions, thereby improving their cost/benefit ratio. We extend Pavlovian concepts beyond the traditional domain of discrete secretory and other physiological reflexes to complex real-world behavioral interactions and apply abstract laboratory analyses of the mechanisms of associative learning to the daily challenges animals face as they interact with one another in their natural environments.

SIGNALED AVOIDANCE IN THE EYE WITHDRAWAL REFLEX OF THE GREEN CRAB

Learning in a signaled avoidance procedure was studied in the eye withdrawal reflex of the green crab, Carcinus maenas. A puff of air to the eye, which causes eye retraction, was used as the unconditioned stimulus (US). A mild vibration on the carapace, which has no effect on untrained animals, was used as a warning (conditioned) stimulus (CS). Eye withdrawal during the CS led to the omission of the otherwise scheduled US. Acquisition was rapid, reaching about 75% avoidance after 30 trials. Extinction occurred slowly over the course of 40 CS-only trials. Yoked controls did not perform as well. The behavior of experimental animals in the avoidance procedure was found to be essentially identical to the performance of animals subjected to a classical conditioning paradigm in which CS responses had no effect on US presentation. Additional groups of animals were subjected to experiments in which (a) avoidance conditioning (60 trials) was followed by classical conditioning (40 trials) or (b) classical conditioning was followed by avoidance. The behavior of these groups was, again, essentially identical. The results suggest that there may be an underlying Pavlovian mechanism for the learned response, although the contribution of an operant process is not excluded. The results expand the range of invertebrate animals in which fundamental conditioning phenomena can be demonstrated, and may provide a neuronal model for learning in a signaled avoidance procedure.

Sign-tracking behavior in aversive conditioning: Its acquisition via a Pavlovian mechanism and its suppression by operant contingencies

When rats are presented with Pavlovian backward pairings in which shock is the unconditioned stimulus (US) and an elevated platform the conditioned stimulus (CS), they show a strong tendency to approach the platform. Attempts were made at evaluating the nature of the associative mechanisms responsible for the acwuisition and maintenance of the approach behavior. In Experiment 1, following a baseline period when platform presentations were given in the absence of shocks, first (training I), approach to the platform was prevented and subjects were exposed to either backward, forward, or random presentations of shock and platform: this procedure attempted to minimize the effects of response learning on the acquisition of approach behavior. Then (training II), all subjects received backward shock-platform pairings and had access to the platform. Approach to the platform was stronger in animals previously exposed to backward pairings than in the animals of the other two groups. Also, animals preexposed to forward platform-shock pairings showed suppression of approach behavior on the first trial of training II relative to their performance on the last trial of baseline. These data suggest that a stimulus-reinforcer learning mechanism alone is sufficent for the acquisition of approach. Experiments 2a and 2b evaluated the effects of “punishing” operant contingencies on Pavlovian-produced approach behavior. The results showed that approach can be eliminated, but only after repeated exposure to an operant contingency which consisted of the immediate closure of the platform upon approach responding. These data suggest that approach behavior produced by Pavlovian contingencies can be modified by operant contingencies. Taken together, these data provide additional support for a sign-tracking notion in aversive conditioning.

Preconditioning of reinforcing properties to an exteroceptive feedback stimulus

Three groups of rats were given Pavlovian preconditioning by means of a yoking procedure and subsequently trained to avoid. One group received a light stimulus during preconditioning at the same times as the master avoiding animals received an exteroceptive feedback stimulus. During the subsequent avoidance training, this group was then given the same light stimulus as a feedback stimulus on successful avoidance trials. The group learned two-way avoidance in about the same number of trials as one-way avoidance is typically learned and substantially faster than two other control groups. These data indicate that feedback stimuli develop reinforcing properties by a Pavlovian mechanism. The data of the control groups indicate that feedback stimuli do not provide “information” about responding in the present situation.