Pavlovian Processes Research Articles

Resistance to change, of behavior and of theory.

The persistence of operant behavior when disrupted tends to be positively related to how often reinforcers were delivered in the past. Behavioral momentum theory describes this finding as the outcome of Pavlovian processes. That is, the relation between discriminative stimuli and reinforcers that were delivered in their presence strengthens behavior, thereby making it more likely to persist. If only the story were that simple. A growing number of findings challenge the basic tenets of behavioral momentum theory. Some even call into question whether Pavlovian relations contribute to persistence in the first place. In this paper, I will review behavioral momentum theory and some of the data that have been problematic for the theory. I will argue that despite these very real challenges, the theory provides important utility not only to basic analyses of response persistence but also to clinical interventions directed at long-term reductions in problem behavior. It, for example, has set the stage for the development of alternative conceptual analyses of resistance to change, two of which will be highlighted for readers. Moreover, behavioral momentum theory may tell us something important about the reasons it continues to have an influence on the field, despite the challenging data that deter it.

From skinner box to daily life: Sign-tracker phenotype co-segregates with impulsivity, compulsivity, and addiction tendencies in humans

Pavlovian conditioning holds the potential to incentivize environmental cues, leading to approach behavior toward them, even outside our awareness. Animal models suggest that this is particularly true for the so-called sign-tracker (ST) phenotype, which is considered to reflect a predisposition toward developing addiction-related behaviours. Despite its potential clinical relevance, few studies have demonstrated the translational validity of this model, likely due to difficulties in studying Pavlovian processes in humans. To fill this gap, we combined an ecological momentary assessment with ambulatory peripheral autonomic monitoring to test the hypothesis that traits associated with ST in preclinical studies would be associated with attribution of high incentive salience to reward-related cues. Several times for 2 days, participants were asked to rate the attractiveness of several preselected ecological rewards (e.g., coffee) and the preceding cues (the smell of coffee) while their electrocardiogram was recorded. While no absolute difference in subjective and physiological measures of motivational approach to daily cues compared with rewards emerged, individuals with high levels of impulsivity, obsessive-compulsive, and addiction-prone behaviors rated as more attractive and showed a greater increase in sympathetic arousal to cues versus rewards. The opposite pattern emerged for those with low levels in those dispositional traits, who responded more (both subjectively and physiologically) to rewards compared with their preceding cues. This study represents an attempt to answer the call to parcel complex behaviors into smaller constructs, improving the early detection of those who are vulnerable to develop psychopathological disorders, particularly in the domain of impulse control such as addiction.

Pavlovian processes may produce contrast leading to bias and suboptimal choice.

Pavlovian processes are likely responsible for the varied contexts in which contrast occurs between what is expected and what is obtained. Such contrast effects result in paradoxical biases and even suboptimal choice by animals. For example, pigeons prefer a suboptimal alternative that results in a stimulus signaling a low probability (20%) high reward (ten pellets) over an optimal alternative that results in a stimulus signaling a high probability (100%) of a smaller reward (three pellets). This effect is analogous to human unskilled gambling. In another case, pigeons prefer a stimulus that has required many pecks to obtain over one the has required one peck to obtain (a so-called justification of effort effect). In a third line of research that investigated the preference for risky choice over safe choice, pigeons chose between two alternatives, a safe choice that resulted in two pellets of food, or a risky choice that resulted in either one or three pellets of food. In general, the pigeons preferred the risky alternative, but importantly, their choice was influenced by whether the choice reflected a gain or a loss - the difference between what was shown to them (one, two, or three pellets) and what they received. Each of these lines of research suggest the importance of contrast effects produced by Pavlovian processes that result in biases or suboptimal behavior.

Temporal Factors Modulate Haloperidol-Induced Conditioned Catalepsy.

Repeated pairings of a neutral context and the effects of haloperidol give rise to conditioned catalepsy when the context is subsequently presented in a drug-free test. In order to confirm whether this response is based on Pavlovian processes, we conducted two experiments involving two manipulations that affect conditioning intensity in classical conditioning procedures: time of joint exposure to the conditioned and the unconditioned stimulus, and the length of the inter-stimulus interval (ISI). The results revealed that both an increase in the length of context-drug pairings during conditioning and a reduced ISI between drug administration and context exposure increased conditioned catalepsy. These results are discussed in terms of the temporal peculiarities of those procedures that involve drugs as the unconditioned stimulus along with the role of Pavlovian conditioning in context-dependent catalepsy.

Temptation over time: Delays help

Does temptation decline over time? Recent studies have highlighted the importance of Pavlovian processes but less is known about how these responses change over time. In a laboratory experiment, every subject made a choice between a banana and chocolate, but a treatment group was informed in advance about the existence of the upcoming choice. These treated subjects were 28% more likely to ultimately choose a banana. Testing an alternative hypothesis of limited willpower, I find no evidence of a simple resource depletion effect using previously induced effort.

Touch for Health: Use of Pavlovian Processes with Physical Touch as a Means to Improve Menstrual Hygiene Management Initiatives, Measured by Willingness to Pay.

BackgroundTo combat the lack of proper facilities and menstrual health knowledge in developing countries, many WASH (Water, Hygiene, and Sanitation) initiatives are including menstrual hygiene management (MHM) components. However, evidence shows that prior efforts have not been ultimately successful in inducing relevant behavior changes, due in part to cultural constraints and unidimensional interventions. As such, MHM research may need to include consideration of new theories/approaches. Evidence is growing of the role that physical presence/proximity and ability to touch objects has on incentives for consumers to purchase goods, captured through willingness to pay (WTP) figures. Such findings can be partially explained by Pavlovian processes.ObjectiveThis study sought to provide field-work validation of such findings of the role of Pavlovian processes and endowment effect on WTP figures for a female hygiene kit, reflecting motivation to adopt better hygiene behaviors.MethodsThis study used primary survey data collected from females in two upper-level schools in southwestern Nepal (n = 169). When presented in conjunction with a hygiene education session, one group of females was allowed physical interaction with a female hygiene kit, while another was not, before being surveyed on their WTP for the kit. Both non-parametric and parametric statistical analyses were performed to assess the impact of this ability to touch the kits on WTP figures.ResultsResults show a statistically significant difference between the WTP figures of those females allowed interaction with the kits prior to being surveyed and those who did not. This confirms the positive impact of physical presence/touch on motivations to use/acquire hygiene-related tools.ConclusionsSuch findings reveal how, through application of the theory of Pavlovian processes, future (menstrual) health education efforts could harness the human instinct to consume, remember, and use those objects presented in physical form, and include personal contact and demonstrations of better health practices in future MHM and WASH education initiatives. Such an approach may allow protocols and interventions to have more success, and dissemination of healthier behaviors and knowledge to be more prominent.Electronic supplementary materialThe online version of this article (10.1007/s41669-019-0168-6) contains supplementary material, which is available to authorized users.

Control of habitual instrumental actions by anticipation of postingestive sensations.

In animal behavioral experiments, extended training causes instrumental actions that deliver ingestible substances to lose sensitivity to outcome devaluation and contingency degradation, and to gain direct sensitivity to the current motivational state. These features of habitual control have been attributed to a process that relies on stimulus-response (S-R) associations linking the context to instrumental actions and Pavlovian associations linking the same context to orally-sensed properties of substances attained there. This Pavlovian process was conceived based on the results of irrelevant incentive experiments, but it is not supported by the results of all such experiments. An alternative process is therefore proposed here. In this process, recall of the instrumental action is evoked by an S-R association, but the rate at which this action is then performed is controlled by anticipation of postingestive sensations that have frequently followed it. This anticipation relies on recall of an association linking the action directly to the postingestive sensations. This association is learned during the formation of a chunked action series that begins with the instrumental action and ends with a consummatory response. It enables a prediction of subjective value that is directly influenced by the current motivational state, but is not influenced by devaluation or non-contingent delivery of the substance that has produced the postingestive sensations.

Interactions between neural decision-making circuits predict long-term dietary treatment success in obesity

Although dietary decision-making is regulated by multiple interacting neural controllers, their impact on dietary treatment success in obesity has only been investigated individually. Here, we used fMRI to test how well interactions between the Pavlovian system (automatically triggering urges of consumption after food cue exposure) and the goal-directed system (considering long-term consequences of food decisions) predict future dietary success achieved in 39 months. Activity of the Pavlovian system was measured with a cue-reactivity task by comparing perception of food versus control pictures, activity of the goal-directed system with a food-specific delay discounting paradigm. Both tasks were applied in 30 individuals with obesity up to five times: Before a 12-week diet, immediately thereafter, and at three annual follow-up visits. Brain activity was analyzed in two steps. In the first, we searched for areas involved in Pavlovian processes and goal-directed control across the 39-month study period with voxel-wise linear mixed-effects (LME) analyses. In the second, we computed network parameters reflecting the covariation of longitudinal voxel activity (i.e. principal components) in the regions identified in the first step and used them to predict body mass changes across the 39 months with LME models. Network analyses testing the link of dietary success with activity of the individual systems as reference found a moderate negative link to Pavlovian activity primarily in left hippocampus and a moderate positive association to goal-directed activity primarily in right inferior parietal gyrus. A cross-paradigm network analysis that integrated activity measured in both tasks revealed a strong positive link for interactions between visual Pavlovian areas and goal-directed decision-making regions mainly located in right insular cortex. We conclude that adaptation of food cue processing resources to goal-directed control activity is an important prerequisite of sustained dietary weight loss, presumably since the latter activity can modulate Pavlovian urges triggered by frequent cue exposure in everyday life.

Pairing neutral cues with alcohol intoxication: new findings in executive and attention networks.

Alcohol-associated stimuli capture attention, yet drinkers differ in the precise stimuli that become paired with intoxication. Extending our prior work to examine the influence of alcoholism risk factors, we paired abstract visual stimuli with intravenous alcohol delivered covertly and examined brain responses to these Pavlovian-conditioned stimuli in fMRI when subjects were not intoxicated. Sixty healthy drinkers performed task-irrelevant alcohol conditioning that presented geometric shapes as conditioned stimuli. Shapes were paired with a rapidly rising alcohol limb (conditioned stimulus; CS+) using intravenous alcohol infusion targeting a final peak breath alcohol concentration of 0.045g/dL or saline (CS-) infusion at matched rates. On day 2, subjects performed monetary delay discounting outside the scanner to assess delay tolerance and then underwent event-related fMRI while performing the same task with CS+, CS-, and an irrelevant symbol. CS+ elicited stronger activation than CS- in frontoparietal executive/attention and orbitofrontal reward-associated networks. Risk factors including family history, recent drinking, sex, and age of drinking onset did not relate to the [CS+ > CS-] activation. Delay-tolerant choice and [CS+ > CS-] activation in right inferior parietal cortex were positively related. Networks governing executive attention and reward showed enhanced responses to stimuli experimentally paired with intoxication, with the right parietal cortex implicated in both alcohol cue pairing and intertemporal choice. While different from our previous study results in 14 men, we believe this paradigm in a large sample of male and female drinkers offers novel insights into Pavlovian processes less affected by idiosyncratic drug associations.

Age-dependent Pavlovian biases influence motor decision-making

Motor decision-making is an essential component of everyday life which requires weighing potential rewards and punishments against the probability of successfully executing an action. To achieve this, humans rely on two key mechanisms; a flexible, instrumental, value-dependent process and a hardwired, Pavlovian, value-independent process. In economic decision-making, age-related decline in risk taking is explained by reduced Pavlovian biases that promote action toward reward. Although healthy ageing has also been associated with decreased risk-taking in motor decision-making, it is currently unknown whether this is a result of changes in Pavlovian biases, instrumental processes or a combination of both. Using a newly established approach-avoidance computational model together with a novel app-based motor decision-making task, we measured sensitivity to reward and punishment when participants (n = 26,532) made a ‘go/no-go’ motor gamble based on their perceived ability to execute a complex action. We show that motor decision-making can be better explained by a model with both instrumental and Pavlovian parameters, and reveal age-related changes across punishment- and reward-based instrumental and Pavlovian processes. However, the most striking effect of ageing was a decrease in Pavlovian attraction towards rewards, which was associated with a reduction in optimality of choice behaviour. In a subset of participants who also played an independent economic decision-making task (n = 17,220), we found similar decision-making tendencies for motor and economic domains across a majority of age groups. Pavlovian biases, therefore, play an important role in not only explaining motor decision-making behaviour but also the changes which occur through normal ageing. This provides a deeper understanding of the mechanisms which shape motor decision-making across the lifespan.

Role Played by the Passage of Time in Reversal Learning.

Reversal learning is thought to involve an extinction-like process that inhibits the expression of the initial learning. However, behavioral evidence for this inhibition remains difficult to interpret as various procedures have been employed to study reversal learning. Here, we used a discrimination task in rats to examine whether the inhibition produced by reversal learning is as sensitive to the passage of time as the inhibition produced by extinction. Experiment 1 showed that when tested immediately after reversal training, rats were able to use the reversed contingencies to solve the discrimination task in an outcome-specific manner. This ability to use outcome-specific information was lost when a delay was inserted between reversal training and test. However, interpretation of these data was made difficult by a potential floor effect. This concern was addressed in Experiment 2 in which it was confirmed that the passage of time impaired the ability of the rats to use the reversed contingencies in an outcome-specific manner to solve the task. Further, it revealed that the delay between initial learning and test was not responsible for this impairment. Additional work demonstrated that solving the discrimination task was unaffected by Pavlovian extinction but that the discriminative stimuli were able to block conditioning to a novel stimulus, suggesting that Pavlovian processes were likely to contribute to solving the discrimination. We therefore concluded that the expression of reversal and extinction learning do share the same sensitivity to the effect of time. However, this sensitivity was most obvious when we assessed outcome-specific information following reversal learning. This suggests that the processes involved in reversal learning are somehow distinct from those underlying extinction learning, as the latter has usually been found to leave outcome-specific information relatively intact. Thus, the present study reveals that a better understanding of the mechanisms supporting reversal training requires assessing the impact that this training exerts on the content of learning rather than performance per se.

Do Pavlovian Processes Really Mediate Behavioral Momentum? Some Conflicting Issues

According to the behavioral momentum theory of response strength (Nevin et al., Journal of the Experimental Analysis of Behavior, 53, 359–379, 1990), steady-state responding reflects the contingency between a response and a reinforcer (response–reinforcer relationship), whereas behavior’s resistance to change is mediated by a contingency between a stimulus and the reinforcer (stimulus–reinforcer relationship). It is further presumed in this theory that a Pavlovian conditioned stimulus (CS)–unconditioned stimulus (US) contingency overlaps with the discriminative stimulus (SD), signaling a primary reinforcer (SR+) within the 3-term contingency (SD: response [R]–SR+). The mere arranging of a stimulus–reinforcer relation in an operant preparation, however, does not necessarily imply that the resulting behavioral process is Pavlovian. This article questions how important such Pavlovian CS–SR+ relations really are in governing operant behavior and its resistance to change in view of evidence from the operant and Pavlovian literatures showing dissociation between Pavlovian and operant stimulus control. To this end, we highlight studies published in the Pavlovian associative literature (Holman and Mackintosh, The Quarterly Journal of Experimental Psychology Section B: Comparative and Physiological Psychology, 33, 21–31, 1981; Rescorla, Current Directions in Psychological Science, 1, 66–70, 1992b) as well as at least 1 seldom-cited study published in the Journal of the Experimental Analysis of Behavior (Marcucella, Journal of the Experimental Analysis of Behavior, 36, 51–60, 1981) supporting the view that CS relations embedded in the 3-term operant contingency can act independently of the discriminative stimulus functions of the SD. These CS relations appear to be neither necessary nor sufficient for sustaining operant discriminative control. Pavlovian relations are likely to be artifacts of operant conditioning—not causal mediators. It is suggested that continued and excessive focus on Pavlovian processes that only have meager influence on operant behavior in general, and behavioral momentum more specifically, will likely be an empirical cul-de-sac for improvement of behavioral management for addiction relapse and other behavioral disorders.

The Neural Basis of Aversive Pavlovian Guidance during Planning.

Important real-world decisions are often arduous as they frequently involve sequences of choices, with initial selections affecting future options. Evaluating every possible combination of choices is computationally intractable, particularly for longer multistep decisions. Therefore, humans frequently use heuristics to reduce the complexity of decisions. We recently used a goal-directed planning task to demonstrate the profound behavioral influence and ubiquity of one such shortcut, namely aversive pruning, a reflexive Pavlovian process that involves neglecting parts of the decision space residing beyond salient negative outcomes. However, how the brain implements this important decision heuristic and what underlies individual differences have hitherto remained unanswered. Therefore, we administered an adapted version of the same planning task to healthy male and female volunteers undergoing functional magnetic resonance imaging (fMRI) to determine the neural basis of aversive pruning. Through both computational and standard categorical fMRI analyses, we show that when planning was influenced by aversive pruning, the subgenual cingulate cortex was robustly recruited. This neural signature was distinct from those associated with general planning and valuation, two fundamental cognitive components elicited by our task but which are complementary to aversive pruning. Furthermore, we found that individual variation in levels of aversive pruning was associated with the responses of insula and dorsolateral prefrontal cortices to the receipt of large monetary losses, and also with subclinical levels of anxiety. In summary, our data reveal the neural signatures of an important reflexive Pavlovian process that shapes goal-directed evaluations and thereby determines the outcome of high-level sequential cognitive processes.SIGNIFICANCE STATEMENT Multistep decisions are complex because initial choices constrain future options. Evaluating every path for long decision sequences is often impractical; thus, cognitive shortcuts are often essential. One pervasive and powerful heuristic is aversive pruning, in which potential decision-making avenues are curtailed at immediate negative outcomes. We used neuroimaging to examine how humans implement such pruning. We found it to be associated with activity in the subgenual cingulate cortex, with neural signatures that were distinguishable from those covarying with planning and valuation. Individual variations in aversive pruning levels related to subclinical anxiety levels and insular cortex activation. These findings reveal the neural mechanisms by which basic negative Pavlovian influences guide decision-making during planning, with implications for disrupted decision-making in psychiatric disorders.

The convergence of psychology and neurobiology in flavor-nutrient learning

Flavor evaluation is influenced by learning from experience with foods. One main influence is flavor-nutrient learning (FNL), a Pavlovian process whereby a flavor acts as a conditioned stimulus (CS) that becomes associated with the postingestive effects of ingested nutrients (the US). As a result that flavor becomes preferred and intake typically increases. This learning powerfully influences food choice and meal patterning. This paper summarizes how research elucidating the physiological and neural substrates of FNL has progressed in parallel with work characterizing how FNL affects perception, motivation, and behavior. The picture that emerges from this work is of a robust system of appetition (a term coined by Sclafani in contrast to the better-understood satiation signals) whereby ingested nutrients sensed in the gut evoke positive motivational responses. Appetition signals act within a meal to promote continued intake in immediate response to gut feedback, and act in the longer term to steer preference towards sensory cues that predict nutritional consequences.

The decision neuroscience perspective on suicidal behavior: evidence and hypotheses.

Suicide attempts are usually regretted by people who survive them. Furthermore, addiction and gambling are over-represented among people who attempt or die by suicide, raising the question whether their decision-making is impaired. Advances in decision neuroscience have enabled us to investigate decision processes in suicidal people and to elucidate putative neural substrates of disadvantageous decision-making. Early studies have linked attempted suicide to poor performance on gambling tasks. More recently, functional MRI augmented with a reinforcement learning computational model revealed that impaired decision-making in suicide attempters is paralleled by disrupted expected value (expected reward) signals in the ventromedial prefrontal cortex. Behavioral studies have linked increased delay discounting to low-lethality/poorly planned attempts, multiple attempts, and the co-occurrence of attempted suicide and addiction. This behavioral tendency may be related to altered integrity of the basal ganglia. By contrast, well-planned, serious suicide attempts were associated with intact/diminished delay discounting. One study has linked high-lethality suicide attempts and impaired social decision-making. This emerging literature supports the notion that various impairments in decision-making - often broadly related to impulsivity - may mark different pathways to suicide. We propose that aggressive and self-destructive responses to social stressors in people prone to suicide result from a predominance of automatic, Pavlovian processes over goal-directed computations.

Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning

Reinforcement learning (RL) theories posit that dopaminergic signals are integrated within the striatum to associate choices with outcomes. Often overlooked is that the amygdala also receives dopaminergic input and is involved in Pavlovian processes that influence choice behavior. To determine the relative contributions of the ventral striatum (VS) and amygdala to appetitive RL, we tested rhesus macaques with VS or amygdala lesions on deterministic and stochastic versions of a two-arm bandit reversal learning task. When learning was characterized with an RL model relative to controls, amygdala lesions caused general decreases in learning from positive feedback and choice consistency. By comparison, VS lesions only affected learning in the stochastic task. Moreover, the VS lesions hastened the monkeys' choice reaction times, which emphasized a speed-accuracy trade-off that accounted for errors in deterministic learning. These results update standard accounts of RL by emphasizing distinct contributions of the amygdala and VS to RL.

Erratum to: Renewal of extinguished instrumental responses: independence from Pavlovian processes and dependence on outcome value.

Erratum to: Renewal of extinguished instrumental responses: independence from Pavlovian processes and dependence on outcome value.

The Pavlovian craver: Neural and experiential correlates of single trial naturalistic food conditioning in humans

Present-day environments are replete with tempting foods and the current obesity pandemic speaks to humans' inability to adjust to this. Pavlovian processes may be fundamental to such hedonic overeating. However, a lack of naturalistic Pavlovian paradigms in humans makes translational research difficult and important parameters such as implicitness and acquisition speed are unknown. Here we present a novel naturalistic conditioning task: an image of a neutral object was conditioned to marzipan taste in a single trial procedure by asking the participant to eat the ‘object’ (made from marzipan). Relative to control objects, results demonstrate robust pre- to post-conditioning changes of both subjective ratings and early as well as late event related brain potentials, suggesting contributions of implicit (attentional) and explicit (motivational) processes. Naturalistic single-trial taste-appetitive conditioning is potent in humans and shapes attentional and motivational neural processes that might challenge self-regulation during exposure to tempting foods. Thus, appetitive conditioning processes might contribute to overweight and obesity.

Editorial: Decision-making under stress: the importance of cortico-limbic circuits

Editorial: Decision-making under stress: the importance of cortico-limbic circuits

What evolves in the evolution of social learning?

AbstractSocial learning is fundamental to social life across the animal kingdom, but we still know little about how natural selection has shaped social learning abilities on a proximate level. Sometimes, complex social learning phenomena can be entirely explained by Pavlovian processes that have little to do with the evolution of sociality. This implies that the ability to learn socially could be an exaptation, not an adaptation, to social life but not that social learning abilities have been left untouched by natural selection. I discuss new empirical evidence for associative learning in social information use, explain how natural selection might facilitate the associative learning process and discuss why such studies are changing the way that we think about social learning.