Rewarding Outcome Research Articles

Electrophysiological correlates of anticipating improbable but desired events

Psychological studies have emphasized that motivation is regulated by the anticipation of the emotional impact from the possible occurrence of unexpected rewarding events. Here, we scrutinized the existence of a corresponding neural signal by means of event-related potentials (ERPs) and computational modeling. In the first experiment, we designed a task that manipulated the probability of gaining a monetary reward and measured ERPs during anticipation and at reward delivery. A sustained frontocentral neural activity (i.e., the stimulus preceding negativity, SPN) was evidenced during the anticipation period. Critically, the SPN was found to increase in amplitude as the reward became more unexpected. Changes in the SPN were found to be predictive of individual differences in risk seeking, suggesting that a greater risk attitude involved a greater motivational state for receiving an improbable reward. In the second experiment, SPN results associated with unexpected monetary gains were replicated in a condition in which participants avoided monetary losses and the occurrence of unexpected rewards was also associated with an increase in the amount of self-reported pleasure. These findings support the existence of a neural ERP signature that encodes the process of tuning our motivation to the possibility of receiving a desirable but improbable rewarding outcome.

Temptation in economic decision making: effects of immediate reward and reward-cues

Temptation in economic decision making: effects of immediate reward and reward-cues Eva Woelbert, Rainer Goebel Department of Cognitive Neuroscience, Maastricht University, Maastricht, The Netherlands Abstract: Immediate exposure to reward or reward-predicting stimuli (cues) influences behavior. For example, chips placed right in front of us are likely to get eaten even if we wish to lose weight or don't actually like chips so much. In this paper we review evidence that shows that immediate exposure to reward and the presence of reward-cues can change economic behavior across various decision domains. Reward cues lead to less patient intertemporal choice, seem to increase risk aversion, and bias consumer choice. This may explain various, at first glance very different, behavioral phenomena, such as dynamic inconsistency, the certainty effect, and the endowment effect. We suggest that immediacy in time, certainty, and physical possession all create immediacy to a rewarding outcome that might bias choice in a similar way as other reward-predicting stimuli. Keywords: immediacy, certainty, proximity, valuation, choice, Pavlovian cues

Brachial Plexus Injuries in Adults: Management and Repair Strategies in our Experience. Results from the Analysis of 428 Supraclavicular Palsies

Background: Brachial plexus palsies are often devastating injuries leading to severe functional impairment of the upper limb. Their surgical management has greatly changed in the last 30 years and it is still evolving but nowadays the key role of surgery in reversing the destiny of many of these patients is clearly recognized.The Authors are presenting their guidelines of management and repair strategy in the surgical treatment of these injuries. Method: 578 post-traumatic brachial plexus injuries in adults were operated in 23 years: 428 supraclavicular and 150 infraclavicular lesions. A retrospective analysis of the surgical strategies applied in supraclavicular injuries and their outcome has been performed. A follow up of at least two years is available in 303 patients. Results: In upper brachial plexus injuries, surgery restored good shoulder and elbow function in more than 90% of cases, regardless the presence of avulsions. Total palsies still retaining some hand function (due to one lower root in continuity with the spine) also received remarkable benefit in about 60% of cases. Shoulder and elbow function were recovered in 40% of complete injuries with multiple avulsions. Conclusions: Advances in preoperative imaging techniques, electrodiagnostic studies and reconstructive surgery have made possible to provide remarkable functional improvement in brachial plexus injuries. Early diagnostic assessment, correct timing of surgery and appropriate rehabilitation are crucial to obtain a rewarding outcome.

Oncogenic osteomalacia: loss of hypophosphatemia might be the key to avoid misdiagnosis

Diagnosing oncogenic osteomalacia is still a challenge. The disorder is characterized by osteomalacia caused by renal phosphate wasting and low serum concentration of 1,25-dihydroxyvitamin D3 occurring in the presence of a tumor that produces high levels of fibroblast growth factor 23. However, it is possible that the disease is much more misdiagnosed than rare. We present the case of a 42-year-old man with a long-term history of undiagnosed progressive muscle weakness. His laboratory results mainly showed low serum phosphate. Surgical removal of a nasal hemangiopericytoma that had been diagnosed five years earlier, brought him to a symptom-free condition. Even though knowing the underlying etiology would explain his osteomalacia, the patient sought medical help from countless physicians for five consecutive years, and only after adequate treatment a rewarding outcome was achieved.

Single-Neuron Responses in the Human Nucleus Accumbens during a Financial Decision-Making Task

Linking values to actions and evaluating expectations relative to outcomes are both central to reinforcement learning and are thought to underlie financial decision-making. However, neurophysiology studies of these processes in humans remain limited. Here, we recorded the activity of single human nucleus accumbens neurons while subjects performed a gambling task. We show that the nucleus accumbens encodes two signals related to subject behavior. First, we find that under relatively predictable conditions, single neuronal activity predicts future financial decisions on a trial-by-trial basis. Interestingly, we show that this activity continues to predict decisions even under conditions of uncertainty (e.g., when the probability of winning or losing is 50/50 and no particular financial choice predicts a rewarding outcome). Furthermore, we find that this activity occurs, on average, 2 s before the subjects physically manifest their decision. Second, we find that the nucleus accumbens encodes the difference between expected and realized outcomes, consistent with a prediction error signal. We show this activity occurs immediately after the subject has realized the outcome of the trial and is present on both the individual and population neuron levels. These results provide human single neuronal evidence that the nucleus accumbens is integral in making financial decisions.

Mechanisms of attention to conditioned stimuli predictive of a cigarette outcome

Attention to stimuli associated with a rewarding outcome may be mediated by the incentive motivational properties that the stimulus acquires during conditioning. Other theories of attention state that the prediction error (the discrepancy between the expected and the actual outcome) during conditioning guides attention; once the outcome is fully predicted, attention should be abolished for the conditioned stimulus. The current study examined which of these mechanisms is dominant in conditioning when the outcome is highly rewarding. Allocation of attention to stimuli associated with cigarettes (the rewarding outcome) was tested in 16 smokers, who underwent a classical conditioning paradigm, where abstract visual stimuli were paired with a tobacco outcome. Stimuli were associated with 100% (stimulus A), 50% (stimulus B), or 0% (stimulus C) probability of receiving tobacco. Attention was measured using an eye-tracker device, and the appetitive value of the stimuli was measured with subjective pleasantness ratings during the conditioning process. Dwell time bias (duration of eye gaze) was greatest overall for the A stimulus, and increased over conditioning. Attention to stimulus A was dependent on the ratings of pleasantness that the stimulus evoked, and on the desire to smoke. These findings appear to support the theory that attention for conditioned stimuli is dominated by the incentive motivational qualities of the outcome they predict, and implicate a role for attention in the maintenance of addictive behaviours like smoking.

The Clinical Practice Collaborative Support Model for the Graduate Nurse

The collaborative support model for the graduate nurse is a proactive approach to supporting all aspects of nursing during the 18-week orientation. Weekly meetings include the unit manager, clinical learning educator, unit clinician, registered nurse (RN) resident, and preceptor. Weekly assessment tools containing goals are created and reviewed by the RN resident and preceptor. Learning needs are then addressed by the team. Because the support model was fully implemented, residents have been able to complete the training program successfully, demonstrating confidence, critical thinking, and a strong skill set. The most rewarding outcome has been the link with management, the clinical learning educator, clinicians, and staff.

Network properties of a computational model of the dorsal raphe nucleus

Serotonin (5-HT) plays an important role in regulating mood, cognition and behaviour. The midbrain dorsal raphe nucleus (DRN) is one of the primary sources of 5-HT. Recent studies show that DRN neuronal activities can encode rewarding (e.g., appetitive) and unrewarding (e.g., aversive) behaviours. Experiments have also shown that DRN neurons can exhibit heterogeneous spiking behaviours. In this work, we build and study a basic spiking neuronal network model of the DRN constrained by neuronal properties observed in experiments. We use an efficient adaptive quadratic integrate-and-fire neuronal model to capture slow afterhyperpolarization current, occasional bursting behaviours in 5-HT neurons, and fast spiking activities in the non-5-HT inhibitory neurons. Provided that our noisy and heterogeneous spiking neuronal network model adopts a feedforward inhibitory network architecture, it is able to replicate the main features of DRN neuronal activities recorded in monkeys performing a reward-based memory-guided saccade task. The model exhibits theta band oscillation, especially among the non-5-HT inhibitory neurons during the rewarding outcome of a simulated trial, thus forming a model prediction. By varying the inhibitory synaptic strengths and the afferent inputs, we find that the network model can oscillate over a range of relatively low frequencies, allow co-existence of multiple stable frequencies, and spike synchrony can spread from within a local neural subgroup to global. Our model suggests plausible network architecture, provides interesting model predictions that can be experimentally tested, and offers a sufficiently realistic multi-scale model for 5-HT neuromodulation simulations.

Coordinated activity of ventral tegmental neurons adapts to appetitive and aversive learning.

Our understanding of how value-related information is encoded in the ventral tegmental area (VTA) is based mainly on the responses of individual putative dopamine neurons. In contrast to cortical areas, the nature of coordinated interactions between groups of VTA neurons during motivated behavior is largely unknown. These interactions can strongly affect information processing, highlighting the importance of investigating network level activity. We recorded the activity of multiple single units and local field potentials (LFP) in the VTA during a task in which rats learned to associate novel stimuli with different outcomes. We found that coordinated activity of VTA units with either putative dopamine or GABA waveforms was influenced differently by rewarding versus aversive outcomes. Specifically, after learning, stimuli paired with a rewarding outcome increased the correlation in activity levels between unit pairs whereas stimuli paired with an aversive outcome decreased the correlation. Paired single unit responses also became more redundant after learning. These response patterns flexibly tracked the reversal of contingencies, suggesting that learning is associated with changing correlations and enhanced functional connectivity between VTA neurons. Analysis of LFP recorded simultaneously with unit activity showed an increase in the power of theta oscillations when stimuli predicted reward but not an aversive outcome. With learning, a higher proportion of putative GABA units were phase locked to the theta oscillations than putative dopamine units. These patterns also adapted when task contingencies were changed. Taken together, these data demonstrate that VTA neurons organize flexibly as functional networks to support appetitive and aversive learning.

Too hungry to learn? Hungry broiler breeders fail to learn a Y-maze food quantity discrimination task

AbstractChoice tests may aid determining whether qualitative dietary restriction improves the welfare of feed-restricted broiler breeder chickens(Gallus gallus domesticus). However, hunger-stress may reduce competency to choose by impairing learning. The effect of chronic feed restriction on the ability of broiler breeders to learn a hunger-relevant discrimination task was investigated using a Y-maze paradigm. The task was to associate black and white arms with large and small quantities of feed. Birds were reared to three growth curves by means of severe (n = 12), moderate (n = 12) or very mild feed restriction (n = 12). Learning the task and selecting the larger food option allowed birds to increase their feed intake. Time taken to traverse the Y-maze was also measured. Birds from all treatment groups traversed the Y-maze more quickly over time, indicating that they had learnt that running down the Y-maze arms was associated with a rewarding outcome (food). However, feed restriction significantly reduced their ability to associate the black and white cues with differences in food quantity. Consequently, average pay-offs in terms of daily feed increments disproportionately accrued to the less feed-restricted treatment groups. It is concluded that feed restriction affected the performance of broiler breeders in this task, perhaps by narrowing their attention such that they ignore potentially hunger-relevant contextual cues. However, low overall group success rates demonstrate that this task was difficult to learn even for less severely feed-restricted birds. Therefore, Y-maze choice tests may not be the most appropriate method for determining hungry broiler breeder dietary preferences.

Nicotine self-administered directly into the VTA by rats is weakly reinforcing but has strong reinforcement enhancing properties

Rats will lever press to deliver nanolitre quantities of nicotine or the muscarinic agonist carbachol directly into the ventral tegmental area (VTA). The purpose of these experiments was to investigate further the characteristics of nicotine self-administration directly into the VTA. This study aimed to confirm previous data relating to intra-VTA self-administration of nicotine and carbachol and then test two hypotheses: (a) that pre-sensitisation of nicotinic receptors is needed for robust intra-VTA self administration and (b) that rats will lever press for intra-VTA nicotine if pre-trained to associate lever pressing with a rewarding outcome. Rats were equipped with cannulae aimed at posterior VTA and allowed five sessions to self-administer nicotine or carbachol. In different experiments, rats were either pre-sensitised to nicotine by subcutaneous (s.c.) injections or pre-trained to lever press for food and a simultaneous conditioned stimulus light. We confirmed that carbachol had strong activating effects when self-administered into the VTA; selective responding for nicotine developed over five sessions by reduction in the amount of pressing on an inactive lever. Prior sensitisation did not improve responding for intra-VTA nicotine but training rats to lever press before putting them on the drug regime did potentiate pressing. The action of nicotine in the VTA might be better considered as reinforcement enhancing and that its intrinsic rewarding property here is at best weak. Identification of the VTA as a target for the reinforcement enhancing effects of nicotine is compatible with the reinforcement-related functions of VTA dopamine neurons and their cholinergic inputs.

Can unpredicted outcomes be intended? The role of outcome-beliefs in children's judgments of intention

An adult-like concept of intention includes a deliberate action to achieve a goal and a belief that one's action (if successful) will cause the desired outcome. For example, good outcomes caused by accident or by chance are not believed to be caused intentionally. In two experiments, we asked whether children understand this connection between intentions and outcomes. Children played two games in which actions could produce unintended outcomes (i.e., causes were unplanned). Children sometimes received a desirable reward independent of intention. In Experiment 1, 4- and 5-year-olds mistakenly claimed they had intended the desirable outcome even when it was unexpected. Four-year-olds judged that they had not intended a deliberate action if it did not yield a rewarding outcome. Experiment 2 demonstrates that 6-year-olds seldom make these errors. The results suggest that 4- and 5-year-old children have not yet attained an adult-like concept of intention. Their inaccurate judgments regarding their intentions, given a rewarding yet unexpected outcome, can be explained by a positivity bias.

Reward positivity elicited by predictive cues

A recent theory holds that a component of the human event-related brain potential called the reward positivity reflects a reward prediction error signal. We investigated this idea in gambling-like task in which, on each trial, a visual stimulus predicted a subsequent rewarding or nonrewarding outcome with 80% probability. Consistent with earlier results, we found that the reward positivity was larger to unexpected than to expected outcomes. In addition, we found that the predictive cues also elicited a reward positivity, as proposed by the theory. These results indicate that the reward positivity reflects the initial assessment of whether a trial will end in success or failure and the reappraisal of that information once the outcome actually occurs.

Micro-opioid receptor activation in the basolateral amygdala mediates the learning of increases but not decreases in the incentive value of a food reward.

The decision to perform, or not perform, actions known to lead to a rewarding outcome is strongly influenced by the current incentive value of the reward. Incentive value is largely determined by the affective experience derived during previous consumption of the reward-the process of incentive learning. We trained rats on a two-lever, seeking-taking chain paradigm for sucrose reward, in which responding on the initial seeking lever of the chain was demonstrably controlled by the incentive value of the reward. We found that infusion of the μ-opioid receptor antagonist, CTOP (d-Phe-Cys-Tyr-d-Trp-Orn-Thr-Pen-Thr-NH(2)), into the basolateral amygdala (BLA) during posttraining, noncontingent consumption of sucrose in a novel elevated-hunger state (a positive incentive learning opportunity) blocked the encoding of incentive value information normally used to increase subsequent sucrose-seeking responses. Similar treatment with δ [N, N-diallyl-Tyr-Aib-Aib-Phe-Leu-OH (ICI 174,864)] or κ [5'-guanidinonaltrindole (GNTI)] antagonists was without effect. Interestingly, none of these drugs affected the ability of the rats to encode a decrease in incentive value resulting from experiencing the sucrose in a novel reduced-hunger state. However, the μ agonist, DAMGO ([d-Ala2, NMe-Phe4, Gly5-ol]-enkephalin), appeared to attenuate this negative incentive learning. These data suggest that upshifts and downshifts in endogenous opioid transmission in the BLA mediate the encoding of positive and negative shifts in incentive value, respectively, through actions at μ-opioid receptors, and provide insight into a mechanism through which opiates may elicit inappropriate desire resulting in their continued intake in the face of diminishing affective experience.

Contrasting Effects of Lithium Chloride and CB1 Receptor Blockade on Enduring Changes in the Valuation of Reward

When an organism responds for a reward, its learned behavior can be characterized as goal-directed or habitual based on whether or not it is susceptible to reward devaluation. Here, we evaluated whether instrumental responding for brain stimulation reward (BSR) can be devalued using a paradigm traditionally used for natural rewards. Rats were trained to lever press for BSR; afterward, BSR was paired with either lithium chloride (LiCl, 5 mg/kg, i.p.), a pro-emetic, or AM251, a CB1 receptor antagonist (3 mg/kg, i.p.) or the vehicle of these compounds. Pairings of BSR with these compounds and their vehicles were performed in a novel environment so that only unconditional effects of BSR would be affected by the pharmacological manipulations. Subsequently, in a probe test, all rats were returned in the drug-free state to the boxes where they had received training and instrumental responding was reassessed in the absence of BSR delivery. When compared to control, LiCl produced a significant decrease in the number of responses during the test session, whereas AM251 did not. These results show that instrumental responding for BSR is susceptible to devaluation, in accord with the proposal that this behavior is supported at least in part by associations between the response and the rewarding outcome. Further, they suggest that reward modulation observed in studies involving the use of CB1 receptor antagonists arises from changes in the organism’s motivation rather than drug-induced changes in the intrinsic value of reward.

Die Nordwand der Grabkammer in der Pyramide Tetis (T/F/N). Anmerkungen und Korrekturvorschläge zur Rekonstruktion der Pyramidentexte Tetis

Abstract Already K. Sethe supposed that the north wall of Teti’s burial chamber (T/F/N) once was covered with offering spells, whose continuation is still preserved on the adjoining east wall. But it took nearly 100 years until a first provisional reconstruction was undertaken by E. Bène, which led to an overall conclusive and rewarding outcome. Nevertheless, some minor questions yet remain unanswered, which concern details of text sequences and the exact position of several fragments showing remnants of spells in columns and lines of nearly totally destructed parts of the wall T/F/N. In this article, some of these questions are treated on the one hand by including the text sequences of Teti’s corpus into the evolutionary line of Old Kingdom Pyramid Texts, on the other hand by reconsidering and correcting some details in the current proposal of reconstruction.

Theta-Band Phase Locking of Orbitofrontal Neurons during Reward Expectancy

The expectancy of a rewarding outcome following actions and cues is coded by a network of brain structures including the orbitofrontal cortex. Thus far, predicted reward was considered to be coded by time-averaged spike rates of neurons. However, besides firing rate, the precise timing of action potentials in relation to ongoing oscillations in local field potentials is thought to be of importance for effective communication between brain areas. We performed multineuron and field potential recordings in orbitofrontal cortex of rats performing olfactory discrimination learning to study the temporal structure of coding predictive of outcome. After associative learning, field potentials were marked by theta oscillations, both in advance and during delivery of reward. Orbitofrontal neurons, especially those coding information about upcoming reward with their firing rate, phase locked to these oscillations in anticipation of reward. When established associations were reversed, phase locking collapsed in the anticipatory task phase, but returned when reward became predictable again after relearning. Behaviorally, the outcome anticipation phase was marked by licking responses, but the frequency of lick responses was dissociated from the strength of theta-band phase locking. The strength of theta-band phase locking by orbitofrontal neurons robustly follows the dynamics of associative learning as measured by behavior and correlates with the rat's current outcome expectancy. Theta-band phase locking may facilitate communication of outcome-related information between reward-related brain areas and offers a novel mechanism for coding value signals during reinforcement learning.

Management of Neonatal Aortic Arch Thrombosis With Low-molecular Weight Heparin

Aortic arch thrombosis (AAT) is a potentially life threatening condition in neonates. We report 3 neonates with AAT and complete occlusion of a unilateral internal carotid artery with multiple cerebral infarcts. Multidisciplinary input led to deferral of thrombolytic therapy and surgical thrombectomy owing to the potential risk of hemorrhage within the infarcts. All the neonates were treated with low-molecular weight heparin treatment alone without any complications. Thrombus resolution was complete in 2 neonates, whereas it was near complete in one neonate. All the infants showed normal growth and development during follow-up. The rewarding outcome in these cases suggests that treatment with low-molecular weight heparin could be considered an option to treat AAT in selected patients.

Measuring and modeling the interaction among reward size, delay to reward, and satiation level on motivation in monkeys.

Motivation is usually inferred from the likelihood or the intensity with which behavior is carried out. It is sensitive to external factors (e.g., the identity, amount, and timing of a rewarding outcome) and internal factors (e.g., hunger or thirst). We trained macaque monkeys to perform a nonchoice instrumental task (a sequential red-green color discrimination) while manipulating two external factors: reward size and delay-to-reward. We also inferred the state of one internal factor, level of satiation, by monitoring the accumulated reward. A visual cue indicated the forthcoming reward size and delay-to-reward in each trial. The fraction of trials completed correctly by the monkeys increased linearly with reward size and was hyperbolically discounted by delay-to-reward duration, relations that are similar to those found in free operant and choice tasks. The fraction of correct trials also decreased progressively as a function of the satiation level. Similar (albeit noiser) relations were obtained for reaction times. The combined effect of reward size, delay-to-reward, and satiation level on the proportion of correct trials is well described as a multiplication of the effects of the single factors when each factor is examined alone. These results provide a quantitative account of the interaction of external and internal factors on instrumental behavior, and allow us to extend the concept of subjective value of a rewarding outcome, usually confined to external factors, to account also for slow changes in the internal drive of the subject.

Rat orbitofrontal cortex separately encodes response and outcome information during performance of goal-directed behavior.

Orbitofrontal cortex (OFC) neurons encode rewards and the cues that predict them, providing a neural substrate for outcome expectancy, an important component of goal-directed behavior in animals and humans. Here, we recorded and analyzed single units from rat lateral OFC during performance of a task in which the encoding of an expected rewarding outcome could be isolated from the response made in obtaining it. We found concurrent encoding of the expected outcome and the behavioral response in mostly separate populations of OFC units, in each phase of task performance: odor sampling, behavioral response, and waiting during the delay before reward delivery. Population analyses showed that outcome encoding broadly spanned across each behavioral phase, whereas response-selective firings were time-locked to the behavioral events, especially the completion of the behavioral response. A significant subset of outcome-selective units maintained selective firings from either odor sampling or response initiation until reward delivery. In contrast, response-selective units typically showed transient activation time locked to the behavioral events and were less likely to maintain selective firings across behavioral phases than outcome-selective units. These data demonstrate a broader role of OFC information processing in goal-directed behavior, beyond its widely recognized role in outcome expectancy.