Prospect Valence Learning Research Articles

Decision-making deficits in obsessive-compulsive disorder are associated with abnormality of recency and response consistency parameter in prospect valence learning model.

Patients with obsessive-compulsive disorder (OCD) have deficits in decision-making in the Iowa Gambling Task (IGT). However, no study has investigated the parameters of the prospect valence learning (PVL) model in the IGT for OCD. This study aimed to investigate deficits in decision-making in OCD using the PVL model and identify whether the parameters of the PVL model were associated with obsessive-compulsive severity. Forty-seven medication-free patients with OCD were compared with 47 healthy controls (HCs). Decision-making was measured using the total net and block net scores of the IGT. A PVL model with a decay-reinforcement learning rule (PVL-DecayRI) was used to investigate the parameters of the model. Correlation analysis was conducted between each parameter of the PVL-DecayRL and obsessive-compulsive symptoms. The total net score of patients with OCD was significantly lower than that of the HCs. The block net scores of the OCD group did not differ across the five blocks, whereas in the HCs, the fifth block net score was significantly higher than the block net scores of the first and second blocks. The values of the recency and response consistency parameters of the PVL-DecayRI in patients with OCD were significantly lower than those in HCs. The recency parameter positively correlated with the Y-BOCS obsessive score. Meanwhile, there was no correlation between consistency parameter values and symptom severity in OCD. Our detailed analysis of the decision-making deficit in OCD suggests that the most recent outcome has a small influence on the expectancy of prospect valence, as indicated by the lower recency parameter, and is characterized by more impulsive choices, as indicated by the lower consistency parameter.

Component Processes of Decision Making in a Community Sample of Precariously Housed Persons: Associations With Learning and Memory, and Health-Risk Behaviors.

The Iowa Gambling Task (IGT) is a widely used measure of decision making, but its value in signifying behaviors associated with adverse, “real-world” consequences has not been consistently demonstrated in persons who are precariously housed or homeless. Studies evaluating the ecological validity of the IGT have primarily relied on traditional IGT scores. However, computational modeling derives underlying component processes of the IGT, which capture specific facets of decision making that may be more closely related to engagement in behaviors associated with negative consequences. This study employed the Prospect Valence Learning (PVL) model to decompose IGT performance into component processes in 294 precariously housed community residents with substance use disorders. Results revealed a predominant focus on gains and a lack of sensitivity to losses in these vulnerable community residents. Hypothesized associations were not detected between component processes and self-reported health-risk behaviors. These findings provide insight into the processes underlying decision making in a vulnerable substance-using population and highlight the challenge of linking specific decision making processes to “real-world” behaviors.

Early stages of the acute physical stress response increase loss aversion and learning on decision making: A Bayesian approach

When the cortisol peak is reached after a stressor people learn slower and make worse decisions in the Iowa Gambling Task (IGT). However, the effects of the early stress response have not received as much attention. Since physical exercise is an important neuroendocrine stressor, this study aimed to fill this gap using an acute physical stressor. We hypothesized that this stress stage would promote an alertness that may increase feedback-sensitivity and, therefore, reward-learning during IGT, leading to a greater overall decision-making. 90 participants were divided into two groups: 47 were exposed to an acute intense physical stressor (cycloergometer) and 43 to a distractor 5 min before IGT. The Prospect Valence-Learning (PVL) computational model was applied to the IGT to investigate decision-making components (feedback-sensitivity, loss aversion, learning and choice consistency). There were no differences in the overall IGT performance, but physically stressed participants showed greater loss aversion and higher learning than controls. In addition, this loss aversion was linearly related to the learning and the choice consistency. These results would support the potentially beneficial role that early stages of stress could play in decision-making and suggest the need of studying the components that underlie this cognitive skill, rather than addressing it as a single dimension.

Understanding Decisions in Collective Risk Social Dilemma Games Using Reinforcement Learning

Prior research has used reinforcement-learning models to investigate human decisions in choice games. However, research has not investigated how reinforcement-learning models expectancy valence learning (EVL) and prospect valence learning (PVL) would explain human decisions in applied judgment games where people face a collective risk social dilemma (CRSD) against societal problems such as climate change. In CRSD game, a group of players invested some part of their private incomes to a public fund over several rounds with the goal of collectively reaching a climate target, failing which climate change would occur with a certain probability making players lose their remaining incomes. In this article, we propose EVL and PVL models in the CRSD game and calibrate model parameters to aggregate and individual human decisions across four between-subjects information conditions, where half of the players in each condition possessed lesser wealth (poor) compared to the other half (rich). Results showed that model calibration to individual decisions provided a more accurate account compared to the calibration to aggregate decisions and the EVL model was better fit compared to the PVL model across most conditions. Both models outperformed the symmetric Nash model across all conditions. Overall, moderate recency, loss aversion, and exploration drove people’s decisions. We present the implications of our model results for situations involving a CRSD.

Cognitive modelling to assess decision-making impairments in patients with current depression and with/without suicide history

It has been reported that decision making is impaired in suicide attempters. Decision making is a complex process and little is known about its different components. Yet, this information would help to understand the functioning of suicidal minds. In this study, the Prospect Valence-Learning (PVL) computational model was applied to the Iowa Gambling Task (IGT) to investigate and compare decision-making components in patients with affective disorder and with/without history of suicide attempts and in healthy controls. To this aim, 116 inpatients with current major depressive episode (among whom 62 suicide attempters) and 38 healthy controls were recruited. Decision-making performance was measured using the IGT. The Bayesian computational PVL model was applied to compare the feedback sensitivity, loss aversion, learning/memory, and choice consistency components of decision making in the different groups. Depressive symptomatology was assessed using the Beck Depression Inventory short form (BDI-SF). The total IGT net score and the loss aversion and learning/memory scores were lower in suicide attempters than in healthy controls. The choice consistency score was low in all patients (with/without suicide history) compared with healthy controls. Moreover, patients with high BDI score showed a positive relationship between the choice consistency score and suicide attempt. These findings suggest that decision-making impairment in depressed patients with and without suicidal history might be the result of underlying problems in feedback processing and task learning, which influence the building of long-term strategies. All these impairments should be targeted in therapeutic strategies for suicidal patients.

폭음 대학생의 의사결정 결함: 아이오와 도박과제와 전망 유인가학습 모델을 중심으로

폭음 대학생의 의사결정 결함: 아이오와 도박과제와 전망 유인가학습 모델을 중심으로

Decision Making Profile of Positive and Negative Anticipatory Skin Conductance Responders in an Unlimited-Time Version of the IGT.

Based on the somatic marker hypothesis (Damasio, 1994), many studies have examined whether or not physiological responses are “somatic markers” that implicitly guide the decision making process. Vegetative or motor reactions that are produced by negative or positive stimuli generate a series of somatic markers. So, when a similar stimuli is encountered in the future, these somatic marks will facilitate favorable decisions and inhibit the disadvantageous ones (Martínez-Selva et al., 2006). The most widely studied physiological responses, as indicators of these markers, are heart rate and the skin conductance response (Damasio, 1994; Bechara et al., 1996). The Iowa Gambling Task (IGT) has been the most widely used tool in this research. The common IGT protocol for psychophysiological studies comprises limited inter-trial intervals, and does not distinguish participants as a function of relevant physiological traits, such as the anticipatory skin conductance response (aSCR). The objectives of this work were to determine whether “somatic markers” guide the decision making process without time restrictions and to examine the effects of opposite aSCR profiles on this process. Participants were 29 healthy subjects, divided into two groups according to positive (+) and negative (−) aSCR. Two different data analysis strategies were applied: firstly, gambling indices were computed and, secondly, we examined the parameters of the probabilistic Prospect Valence Learning (PVL) model in three versions: maximum likelihood estimation (MLE), PVL-Delta and PVL-Decay simulations with Hierarchical Bayesian analysis (HBA) for parameter estimation. The results show a significant group effect in gambling indices, with the aSCR+ group presenting lower risk in the decision making process than the aSCR− group. Significant differences were also observed in the Utility parameter of MLE-PVL, with the aSCR− group have low sensitivity to feedback outcomes, than aSRC+ group. However, data from the PVL simulations do not show significant group differences and, in both cases, the utility value denotes low sensitivity to feedback outcomes.

Decision-Making Deficits Are Associated With Learning Impairments in Female College Students at High Risk for Anorexia Nervosa: Iowa Gambling Task and Prospect Valence Learning Model.

This study investigated deficits in decision-making ability in female college students at high risk for anorexia nervosa (AN) using the Iowa Gambling Task (IGT) and the prospect valence learning (PVL) model. Based on scores on the Korean version of the Eating Attitude Test-26 (KEAT-26), participants were assigned to either the high risk for AN group (n = 42) or the control group (n = 43). The high risk for AN group exhibited significantly lower total net scores and block net scores on the third, fourth, and fifth blocks of the IGT than the control group did. The high risk for AN group selected cards significantly more often from the disadvantageous A and B decks and less often from the advantageous D deck than the control group did. In addition, the block net scores of the high risk for AN group did not differ across the five blocks, whereas those of the control group increased as the trials progressed. There was a significant negative correlation between IGT total net score and total score on the KEAT−26. The high risk for AN group had significantly lower values than the control group on the learning and response consistency parameters of the PVL model. These results indicate that female college students at high risk for AN have deficits in decision-making ability, and that these deficits are related to difficulties in remembering experience obtained from earlier trials and applying it to later trials. These difficulties further lead them to make decisions randomly.

Age of onset of cannabis use and decision making under uncertainty.

ObjectiveCannabis, like other substances, negatively affects health, inducing respiratory problems and mental and cognitive alterations. Memory and learning disorders, as well as executive dysfunctions, are also neuropsychological disorders associated to cannabis use. Recent evidence reveals that cannabis use during adolescence may disrupt the normal development of the brain. This study is aimed to analyze possible differences between early-onset and late-onset cannabis consumers.MethodWe used a task based on a card game with four decks and different programs of gains/losses. A total of 72 subjects (19 women; 53 men) participated in the study; they were selected through a purposive sampling and divided into three groups: early-onset consumers, late-onset consumers, and control (non-consumers). The task used was the “Cartas” program (computerized version based on the Iowa Gambling Task (IGT)), with two versions: direct and inverse. The computational model “Prospect Valence Learning” (PVL) was applied in order to describe the decision according to four characteristics: utility, loss aversion, recency, and consistency.ResultsThe results evidence worst performance in the IGT in the early-onset consumers as compared to late-onset consumers and control. Differences between groups were also found in the PVL computational model parameters, since the process of decision making of the early-onset consumers was more influenced by the magnitude of the gains-losses, and more determined by short-term results without loss aversion.ConclusionsEarly onset cannabis use may involve decision-making problems, and therefore intervention programs are necessary in order to reduce the prevalence and delay the onset of cannabis use among teenagers.

Computational EEG modelling of decision making under ambiguity reveals spatio-temporal dynamics of outcome evaluation

Complex human cognition, such as decision-making under ambiguity, is reflected in dynamic spatio-temporal activity in the brain. Here, we combined event-related potentials with computational modelling of the time course of decision-making and outcome evaluation during the Iowa Gambling Task. Measures of choice probability generated using the Prospect Valence Learning Delta (PVL-Delta) model, in addition to objective trial outcomes (outcome magnitude and valence), were applied as regressors in a general linear model of the EEG signal. The resulting three-dimensional spatio-temporal characterization of task-related neural dynamics demonstrated that outcome valence, outcome magnitude, and PVL-Delta choice probability were expressed in distinctly separate event related potentials. Our findings showed that the P3 component was associated with an experience-based measure of outcome expectancy.

S.22.03 - Hedonic eating and the microbiome in humans

Our everyday-life comprises a multitude of decisions that we take whilst trying to maximize advantageous outcomes, limit risks and update current needs. The cognitive processes that guide decision making as well as the brain circuits they are based on are only poorly understood. Numerous studies point to a potential role of dopamine and nicotine in decision making but less is known about their interactions. Here, 26 healthy male subjects performed the Iowa Gambling Task (IGT) in two sessions following the administration of either nicotine or placebo. Striatal dopamine transporter (DAT) binding was measured by single-photon emission computed tomography (SPECT). Results indicate that lower DAT levels were associated with better performance in the IGT (p=0.0004). Cognitive modelling analysis using the prospect valence learning (PVL) model indicated that low DAT subjects’ performance deteriorated following nicotine administration as indicated by an increased learning rate and a decreased response consistency. Our results shed light on the neurochemistry underlying reward-based decision making in humans by demonstrating a significant interaction between nicotine and the DAT. The observed interaction is consistent with the hypothesized associations between DAT expression and extracellular dopamine levels, suggestive of an inverted U-shape relationship between baseline dopamine and magnitude in response to a pro-dopaminergic compound. Our findings are of particular interest in the context of psychiatric disorders where aberrant decision making represents a part of the core symptomatology, such as addiction, schizophrenia or depression.

S.21.01 - (Epi)genetic regulation of fear sensitivity in mouse models of anxiety

Our everyday-life comprises a multitude of decisions that we take whilst trying to maximize advantageous outcomes, limit risks and update current needs. The cognitive processes that guide decision making as well as the brain circuits they are based on are only poorly understood. Numerous studies point to a potential role of dopamine and nicotine in decision making but less is known about their interactions. Here, 26 healthy male subjects performed the Iowa Gambling Task (IGT) in two sessions following the administration of either nicotine or placebo. Striatal dopamine transporter (DAT) binding was measured by single-photon emission computed tomography (SPECT). Results indicate that lower DAT levels were associated with better performance in the IGT (p=0.0004). Cognitive modelling analysis using the prospect valence learning (PVL) model indicated that low DAT subjects’ performance deteriorated following nicotine administration as indicated by an increased learning rate and a decreased response consistency. Our results shed light on the neurochemistry underlying reward-based decision making in humans by demonstrating a significant interaction between nicotine and the DAT. The observed interaction is consistent with the hypothesized associations between DAT expression and extracellular dopamine levels, suggestive of an inverted U-shape relationship between baseline dopamine and magnitude in response to a pro-dopaminergic compound. Our findings are of particular interest in the context of psychiatric disorders where aberrant decision making represents a part of the core symptomatology, such as addiction, schizophrenia or depression.

Nicotine–dopamine-transporter interactions during reward-based decision making

Our everyday-life comprises a multitude of decisions that we take whilst trying to maximize advantageous outcomes, limit risks and update current needs. The cognitive processes that guide decision making as well as the brain circuits they are based on are only poorly understood. Numerous studies point to a potential role of dopamine and nicotine in decision making but less is known about their interactions. Here, 26 healthy male subjects performed the Iowa Gambling Task (IGT) in two sessions following the administration of either nicotine or placebo. Striatal dopamine transporter (DAT) binding was measured by single-photon emission computed tomography (SPECT). Results indicate that lower DAT levels were associated with better performance in the IGT (p=0.0004). Cognitive modelling analysis using the prospect valence learning (PVL) model indicated that low DAT subjects’ performance deteriorated following nicotine administration as indicated by an increased learning rate and a decreased response consistency. Our results shed light on the neurochemistry underlying reward-based decision making in humans by demonstrating a significant interaction between nicotine and the DAT. The observed interaction is consistent with the hypothesized associations between DAT expression and extracellular dopamine levels, suggestive of an inverted U-shape relationship between baseline dopamine and magnitude in response to a pro-dopaminergic compound. Our findings are of particular interest in the context of psychiatric disorders where aberrant decision making represents a part of the core symptomatology, such as addiction, schizophrenia or depression.

Decision-making deficits in patients with chronic schizophrenia: Iowa Gambling Task and Prospect Valence Learning model.

PurposeDecision-making is the process of forming preferences for possible options, selecting and executing actions, and evaluating the outcome. This study used the Iowa Gambling Task (IGT) and the Prospect Valence Learning (PVL) model to investigate deficits in risk-reward related decision-making in patients with chronic schizophrenia, and to identify decision-making processes that contribute to poor IGT performance in these patients.Materials and methodsThirty-nine patients with schizophrenia and 31 healthy controls participated. Decision-making was measured by total net score, block net scores, and the total number of cards selected from each deck of the IGT. PVL parameters were estimated with the Markov chain Monte Carlo sampling scheme in OpenBugs and BRugs, its interface to R, and the estimated parameters were analyzed with the Mann–Whitney U-test.ResultsThe schizophrenia group received significantly lower total net scores compared to the control group. In terms of block net scores, an interaction effect of group × block was observed. The block net scores of the schizophrenia group did not differ across the five blocks, whereas those of the control group increased as the blocks progressed. The schizophrenia group obtained significantly lower block net scores in the fourth and fifth blocks of the IGT and selected cards from deck D (advantageous) less frequently than the control group. Additionally, the schizophrenia group had significantly lower values on the utility-shape, loss-aversion, recency, and consistency parameters of the PVL model.ConclusionThese results indicate that patients with schizophrenia experience deficits in decision-making, possibly due to failure in learning the expected value of each deck, and incorporating outcome experiences of previous trials into expectancies about options in the present trial.

Bayes factors for reinforcement-learning models of the Iowa gambling task.

The psychological processes that underlie performance on the Iowa gambling task (IGT) are often isolated with the help of reinforcement-learning (RL) models. The most popular method to compare RL models is the BIC post hoc fit criterion—a criterion that considers goodness-of-fit relative to model complexity. However, the current implementation of the BIC post hoc fit criterion considers only 1 dimension of model complexity, that is, the number of free parameters. A more sophisticated implementation of the BIC post hoc fit criterion, 1 that provides a coherent and complete discounting of complexity, is provided by the Bayes factor. Here we demonstrate an analysis in which Bayes factors are obtained with a Monte Carlo method, known as importance sampling, to compare 4 RL models of the IGT: the Expectancy Valence (EV), Prospect Valence Learning (PVL), PVL-Delta, and Value-Plus-Perseveration (VPP) models. We illustrate the method using a data pool of 771 participants from 11 different studies. Our results provide strong evidence for the VPP model and moderate evidence for the PVL model, but little evidence for the EV and PVL-Delta models—results that were not in line with a BIC post hoc fit analysis. We discuss how our results may be combined with results obtained from other model comparison studies to obtain a balanced and comprehensive assessment of model adequacy. (PsycINFO Database Record (c) 2016 APA, all rights reserved)

An improved cognitive model of the Iowa and Soochow Gambling Tasks with regard to model fitting performance and tests of parameter consistency.

The Iowa Gambling Task (IGT) and the Soochow Gambling Task (SGT) are two experience-based risky decision-making tasks for examining decision-making deficits in clinical populations. Several cognitive models, including the expectancy-valence learning (EVL) model and the prospect valence learning (PVL) model, have been developed to disentangle the motivational, cognitive, and response processes underlying the explicit choices in these tasks. The purpose of the current study was to develop an improved model that can fit empirical data better than the EVL and PVL models and, in addition, produce more consistent parameter estimates across the IGT and SGT. Twenty-six opiate users (mean age 34.23; SD 8.79) and 27 control participants (mean age 35; SD 10.44) completed both tasks. Eighteen cognitive models varying in evaluation, updating, and choice rules were fit to individual data and their performances were compared to that of a statistical baseline model to find a best fitting model. The results showed that the model combining the prospect utility function treating gains and losses separately, the decay-reinforcement updating rule, and the trial-independent choice rule performed the best in both tasks. Furthermore, the winning model produced more consistent individual parameter estimates across the two tasks than any of the other models.

The impact of subjective cognitive decline on Iowa Gambling Task performance.

To ascertain whether the Iowa Gambling Task (IGT) could be used to detect and identify measurable cognitive differences between older adults with subjective cognitive decline (SCD) as compared with healthy older controls (HC). Older adults with self-identified SCD and age-matched controls completed a comprehensive neuropsychological assessment battery including the clinical version of the IGT, as well as self-report measures of mood and personality. The groups did not differ on clinically normed scores on the IGT. However, the groups did differ in the specific decks chosen as they progressed through the task, with the SCD group choosing the advantageous, high loss-frequency deck (Deck C) more often toward the end of the task. Using hierarchical Bayesian parameter estimation, we show that the prospect valence learning (PVL) model outperforms the expectancy valence learning (EVL) model in parsimoniously accounting for task performance by both groups. The PVL model explains the difference in deck choices between groups as being because of an underlying difference in their learning rate, with the SCD group emphasizing the current outcome over past outcomes more than the HC group. Behavioral results indicate measureable differences in risky decision making in older adults with SCD as compared with healthy controls. Modeling results allow us to interpret this difference as potentially being because of rapid forgetting of trial-to-trial information. This work furthers our understanding of SCD, while demonstrating the use of computational modeling in the interpretation of neuropsychological data.

Decision-making in stimulant and opiate addicts in protracted abstinence: evidence from computational modeling with pure users.

Substance dependent individuals (SDI) often exhibit decision-making deficits; however, it remains unclear whether the nature of the underlying decision-making processes is the same in users of different classes of drugs and whether these deficits persist after discontinuation of drug use. We used computational modeling to address these questions in a unique sample of relatively “pure” amphetamine-dependent (N = 38) and heroin-dependent individuals (N = 43) who were currently in protracted abstinence, and in 48 healthy controls (HC). A Bayesian model comparison technique, a simulation method, and parameter recovery tests were used to compare three cognitive models: (1) Prospect Valence Learning with decay reinforcement learning rule (PVL-DecayRI), (2) PVL with delta learning rule (PVL-Delta), and (3) Value-Plus-Perseverance (VPP) model based on Win-Stay-Lose-Switch (WSLS) strategy. The model comparison results indicated that the VPP model, a hybrid model of reinforcement learning (RL) and a heuristic strategy of perseverance had the best post-hoc model fit, but the two PVL models showed better simulation and parameter recovery performance. Computational modeling results suggested that overall all three groups relied more on RL than on a WSLS strategy. Heroin users displayed reduced loss aversion relative to HC across all three models, which suggests that their decision-making deficits are longstanding (or pre-existing) and may be driven by reduced sensitivity to loss. In contrast, amphetamine users showed comparable cognitive functions to HC with the VPP model, whereas the second best-fitting model with relatively good simulation performance (PVL-DecayRI) revealed increased reward sensitivity relative to HC. These results suggest that some decision-making deficits persist in protracted abstinence and may be mediated by different mechanisms in opiate and stimulant users.

Absolute performance of reinforcement-learning models for the Iowa Gambling Task.

Decision-making deficits in clinical populations are often studied using the Iowa Gambling Task (IGT). Performance on the IGT can be decomposed in its constituent psychological processes by means of cognitive modeling analyses. However, conclusions about the hypothesized psychological processes are valid only if the model provides an adequate account of the data. In this article, we systematically assessed absolute model performance of the Expectancy Valence (EV) model, the Prospect Valence Learning (PVL) model, and a hybrid version of both models—the PVL-Delta model—using 2 different methods. These methods assess (a) whether a model provides an acceptable fit to an observed choice pattern, and (b) whether the parameters obtained from model fitting can be used to generate the observed choice pattern. Our results show that all models provided an acceptable fit to 2 stylized data sets; however, when the model parameters were used to generate choices, only the PVL-Delta model captured the qualitative patterns in the data. These findings were confirmed by fitting the models to 5 published IGT data sets. Our results highlight that a model’s ability to fit a particular choice pattern does not guarantee that the model can also generate that same choice pattern. Future applications of RL models should carefully assess absolute model performance to avoid premature conclusions about the psychological processes that drive performance on the IGT.

Strategic and non‐strategic problem gamblers differ on decision‐making under risk and ambiguity

To analyse problem gamblers' decision-making under conditions of risk and ambiguity, investigate underlying psychological factors associated with their choice behaviour and examine whether decision-making differed in strategic (e.g., sports betting) and non-strategic (e.g., electronic gaming machine) problem gamblers. Cross-sectional study. Out-patient treatment centres and university testing facilities in Victoria, Australia. Thirty-nine problem gamblers and 41 age, gender and estimated IQ-matched controls. Decision-making tasks included the Iowa Gambling Task (IGT) and a loss aversion task. The Prospect Valence Learning (PVL) model was used to provide an explanation of cognitive, motivational and response style factors involved in IGT performance. Overall, problem gamblers performed more poorly than controls on both the IGT (P = 0.04) and the loss aversion task (P = 0.01), and their IGT decisions were associated with heightened attention to gains (P = 0.003) and less consistency (P = 0.002). Strategic problem gamblers did not differ from matched controls on either decision-making task, but non-strategic problem gamblers performed worse on both the IGT (P = 0.006) and the loss aversion task (P = 0.02). Furthermore, we found differences in the PVL model parameters underlying strategic and non-strategic problem gamblers' choices on the IGT. Problem gamblers demonstrated poor decision-making under conditions of risk and ambiguity. Strategic (e.g. sports betting, poker) and non-strategic (e.g. electronic gaming machines) problem gamblers differed in decision-making and the underlying psychological processes associated with their decisions.