Representation Of Expected Value Research Articles

Computation of Conditional Expectations with Guarantees

Theoretically, the conditional expectation of a square-integrable random variable Y given a d-dimensional random vector X can be obtained by minimizing the mean squared distance between Y and f(X) over all Borel measurable functions f :mathbb {R}^d rightarrow mathbb {R}. However, in many applications this minimization problem cannot be solved exactly, and instead, a numerical method which computes an approximate minimum over a suitable subfamily of Borel functions has to be used. The quality of the result depends on the adequacy of the subfamily and the performance of the numerical method. In this paper, we derive an expected value representation of the minimal mean squared distance which in many applications can efficiently be approximated with a standard Monte Carlo average. This enables us to provide guarantees for the accuracy of any numerical approximation of a given conditional expectation. We illustrate the method by assessing the quality of approximate conditional expectations obtained by linear, polynomial and neural network regression in different concrete examples.

Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients

BackgroundReinforcement learning has been proposed to contribute to the development of amotivation in individuals with schizophrenia (SZ). Accumulating evidence suggests dysfunctional learning in individuals with SZ in Go/NoGo learning and expected value representation. However, previous findings might have been confounded by the effects of antipsychotic exposure. Moreover, reinforcement learning also rely on the learning context. Few studies have examined the learning performance in reward and loss-avoidance context separately in medication-naïve individuals with first-episode SZ. This study aimed to explore the behaviour profile of reinforcement learning performance in medication-naïve individuals with first-episode SZ, including the contextual performance, the Go/NoGo learning and the expected value representation performance.MethodsTwenty-nine medication-naïve individuals with first-episode SZ and 40 healthy controls (HCs) who have no significant difference in age and gender, completed the Gain and Loss Avoidance Task, a reinforcement learning task involving stimulus pairs presented in both the reward and loss-avoidance context. We assessed the group difference in accuracy in the reward and loss-avoidance context, the Go/NoGo learning and the expected value representation. The correlations between learning performance and the negative symptom severity were examined.ResultsIndividuals with SZ showed significantly lower accuracy when learning under the reward than the loss-avoidance context as compared to HCs. The accuracies under the reward context (90%win- 10%win) in the Acquisition phase was significantly and negatively correlated with the Scale for the Assessment of Negative Symptoms (SANS) avolition scores in individuals with SZ. On the other hand, individuals with SZ showed spared ability of Go/NoGo learning and expected value representation.ConclusionsDespite our small sample size and relatively modest findings, our results suggest possible reduced learning bias towards reward context among medication-naïve individuals with first-episode SZ. The reward learning performance was correlated with amotivation symptoms. This finding may facilitate our understanding of the underlying mechanism of negative symptoms. Reinforcement learning performance under the reward context may be important to better predict and prevent the development of schizophrenia patients’ negative symptom, especially amotivation.

A neural and behavioral trade-off between value and uncertainty underlies exploratory decisions in normative anxiety.

Exploration reduces uncertainty about the environment and improves the quality of future decisions, but at the cost of provisional uncertain and suboptimal outcomes. Although anxiety promotes intolerance to uncertainty, it remains unclear whether and by which mechanisms anxiety relates to exploratory decision-making. We use a dynamic three-armed-bandit task and find that higher trait-anxiety is associated with increased exploration, which in turn harms overall performance. We identify two distinct behavioral sources: first, decisions made by anxious individuals are guided toward reduction of uncertainty; and second, decisions are less guided by immediate value gains. These findings are similar in both loss and gain domains, and further demonstrate that an affective trait relates to exploration and results in an inverse-U-shaped relationship between anxiety and overall performance. Additional imaging data (fMRI) suggests that normative anxiety correlates negatively with the representation of expected-value in the dorsal-anterior-cingulate-cortex, and in contrast, positively with the representation of uncertainty in the anterior-insula. We conclude that a trade-off between value-gains and uncertainty-reduction entails maladaptive decision-making in individuals with higher normal-range anxiety.

A neurocomputational investigation of reinforcement-based decision making as a candidate latent vulnerability mechanism in maltreated children.

Alterations in reinforcement-based decision making may be associated with increased psychiatric vulnerability in children who have experienced maltreatment. A probabilistic passive avoidance task and a model-based functional magnetic resonance imaging analytic approach were implemented to assess the neurocomputational components underlying decision making: (a) reinforcement expectancies (the representation of the outcomes associated with a stimulus) and (b) prediction error signaling (the ability to detect the differences between expected and actual outcomes). There were three main findings. First, the maltreated group (n = 18; mean age = 13), relative to nonmaltreated peers (n = 19; mean age = 13), showed decreased activity during expected value processing in a widespread network commonly associated with reinforcement expectancies representation, including the striatum (especially the caudate), the orbitofrontal cortex, and medial temporal structures including the hippocampus and insula. Second, consistent with previously reported hyperresponsiveness to negative cues in the context of childhood abuse, the maltreated group showed increased prediction error signaling in the middle cingulate gyrus, somatosensory cortex, superior temporal gyrus, and thalamus. Third, the maltreated group showed increased activity in frontodorsal regions and in the putamen during expected value representation. These findings suggest that early adverse environments disrupt the development of decision-making processes, which in turn may compromise psychosocial functioning in ways that increase latent vulnerability to psychiatric disorder.

Decomposing the roles of perseveration and expected value representation in models of the Iowa gambling task

Models of human behavior in the Iowa Gambling Task (IGT) have played a pivotal role in accounting for behavioral differences during decision-making. One critical difference between models that have been used to account for behavior in the IGT is the inclusion or exclusion of the assumption that participants tend to persevere, or stay with the same option over consecutive trials. Models that allow for this assumption include win-stay-lose-shift (WSLS) models and reinforcement learning (RL) models that include a decay learning rule where expected values for each option decay as they are chosen less often. One shortcoming of RL models that have included decay rules is that the tendency to persevere by sticking with the same option has been conflated with the tendency to select the option with the highest expected value because a single term is used to represent both of these tendencies. In the current work we isolate the tendencies to perseverate and to select the option with the highest expected value by including them as separate terms in a Value-Plus-Perseveration (VPP) RL model. Overall the VPP model provides a better fit to data from a large group of participants than models that include a single term to account for both perseveration and the representation of expected value. Simulations of each model show that the VPP model's simulated choices most closely resemble the decision-making behavior of human subjects. In addition, we also find that parameter estimates of loss aversion are more strongly correlated with performance when perseverative tendencies and expected value representations are decomposed as separate terms within the model. The results suggest that the tendency to persevere and the tendency to select the option that leads to the best net payoff are central components of decision-making behavior in the IGT. Future work should use this model to better examine decision-making behavior.

Neural Signatures of Economic Parameters During Decision-Making: A Functional MRI (fMRI), Electroencephalography (EEG) and Autonomic Monitoring Study

Adaptive behaviour requires an ability to obtain rewards by choosing between different risky options. Financial gambles can be used to study effective decision-making experimentally, and to distinguish processes involved in choice option evaluation from outcome feedback and other contextual factors. Here, we used a paradigm where participants evaluated 'mixed' gambles, each presenting a potential gain and a potential loss and an associated variable outcome probability. We recorded neural responses using autonomic monitoring, electroencephalography (EEG) and functional neuroimaging (fMRI), and used a univariate, parametric design to test for correlations with the eleven economic parameters that varied across gambles, including expected value (EV) and amount magnitude. Consistent with behavioural economic theory, participants were risk-averse. Gamble evaluation generated detectable autonomic responses, but only weak correlations with outcome uncertainty were found, suggesting that peripheral autonomic feedback does not play a major role in this task. Long-latency stimulus-evoked EEG potentials were sensitive to expected gain and expected value, while alpha-band power reflected expected loss and amount magnitude, suggesting parallel representations of distinct economic qualities in cortical activation and central arousal. Neural correlates of expected value representation were localized using fMRI to ventromedial prefrontal cortex, while the processing of other economic parameters was associated with distinct patterns across lateral prefrontal, cingulate, insula and occipital cortices including default-mode network and early visual areas. These multimodal data provide complementary evidence for distributed substrates of choice evaluation across multiple, predominantly cortical, brain systems wherein distinct regions are preferentially attuned to specific economic features. Our findings extend biologically-plausible models of risky decision-making while providing potential biomarkers of economic representations that can be applied to the study of deficits in motivational behaviour in neurological and psychiatric patients.

Altered representation of expected value in the orbitofrontal cortex in mania

Increased responsiveness to appetitive and reduced responsiveness to aversive anticipatory cues may be associated with dysfunction of the brain reward system in mania. Here we studied neural correlates of gain and loss expectation in mania using functional magnetic resonance imaging (fMRI). Fifteen manic patients and 26 matched healthy control individuals performed a monetary incentive delay task, during which subjects anticipated to win or lose a varying amount of money. Varying both magnitude and valence (win, loss) of anticipatory cues allowed us to isolate the effects of magnitude, valence and expected value (magnitude-by-valence interaction). Response times and total gain amount did not differ significantly between groups. FMRI data indicated that the ventral striatum responded according to cued incentive magnitude in both groups, and this effect did not significantly differ between groups. However, a significant group difference was observed for expected value representation in the left lateral orbitofrontal cortex (OFC; BA 11 and 47). In this region, patients showed increasing BOLD responses during expectation of increasing gain and decreasing responses during expectation of increasing loss, while healthy subjects tended to show the inverse effect. In seven patients retested after remission OFC responses adapted to the response pattern of healthy controls. The observed alterations are consistent with a state-related affective processing bias during the expectation of gains and losses which may contribute to clinical features of mania, such as the enhanced motivation for seeking rewards and the underestimation of risks and potential punishments.

Format Effects in the Prisoner's Dilemma Game

Three different presentation formats of the same Prisoner's Dilemma game were investigated. Fifteen pairs of male subjects were assigned to each of three experimental conditions. One group played a conventional matrix representation of the game, one group played an expected value representation of the game, and one group played a simplified probabilistic version of the game called "Take-Some." Each pair of subjects played their game for fifty trials.