Nonrewarded Stimulus Research Articles

The reduced distractibility to the low-valued and non-rewarding stimuli underlies the neural development of value-based attention

The reduced distractibility to the low-valued and non-rewarding stimuli underlies the neural development of value-based attention

Selective enhancement of neural coding in V1 underlies fine-discrimination learning in tree shrew

Visual discrimination improves with training, a phenomenon that is thought to reflect plastic changes in the responses of neurons in primary visual cortex (V1). However, the identity of the neurons that undergo change, the nature of the changes, and the consequences of these changes for other visual behaviors remain unclear. We used chronic invivo 2-photon calcium imaging to monitor the responses of neurons in the V1 of tree shrews learning a Go/No-Go fine orientation discrimination task. We observed increases in neural population measures of discriminability for task-relevant stimuli that correlate with performance and depend on a select subset of neurons with preferred orientations that include the rewarded stimulus and nearby orientations biased away from the non-rewarded stimulus. Learning is accompanied by selective enhancement in the response of these neurons to the rewarded stimulus that further increases their ability to discriminate the task stimuli. These changes persist outside of the trained task and predict observed enhancement and impairment in performance of other discriminations, providing evidence for selective and persistent learning-induced plasticity in the V1, with significant consequences for perception.

The human orbitofrontal cortex, vmPFC, and anterior cingulate cortex effective connectome: emotion, memory, and action.

The human orbitofrontal cortex, ventromedial prefrontal cortex (vmPFC), and anterior cingulate cortex are involved in reward processing and thereby in emotion but are also implicated in episodic memory. To understand these regions better, the effective connectivity between 360 cortical regions and 24 subcortical regions was measured in 172 humans from the Human Connectome Project and complemented with functional connectivity and diffusion tractography. The orbitofrontal cortex has effective connectivity from gustatory, olfactory, and temporal visual, auditory, and pole cortical areas. The orbitofrontal cortex has connectivity to the pregenual anterior and posterior cingulate cortex and hippocampal system and provides for rewards to be used in memory and navigation to goals. The orbitofrontal and pregenual anterior cortex have connectivity to the supracallosal anterior cingulate cortex, which projects to midcingulate and other premotor cortical areas and provides for action-outcome learning including limb withdrawal or flight or fight to aversive and nonreward stimuli. The lateral orbitofrontal cortex has outputs to language systems in the inferior frontal gyrus. The medial orbitofrontal cortex connects to the nucleus basalis of Meynert and the pregenual cingulate to the septum, and damage to these cortical regions may contribute to memory impairments by disrupting cholinergic influences on the neocortex and hippocampus.

The effect of dimensional reinforcement prediction on discrimination of compound visual stimuli by pigeons.

We trained eight pigeons (Columba livia) on a stagewise go/no-go visual discrimination task. A total of 16 visual stimuli were created from all possible combinations of four binary dimensions: brightness (dark/bright), size (large/small), line orientation (vertical/horizontal), and shape (circle/square). In the first stage, we presented S + and four S- stimuli: sharing one (brightness), two (brightness and orientation), three (brightness, orientation, and size), or no dimensional values with S + . In the second stage, all 16 stimuli were presented. In the first stage, stimulus discrimination was controlled by the number of dimensional disparities between non-rewarded stimuli and a rewarded one rather than by stimulus dimensional salience, whereas at the beginning of the second stage, pigeon behaviour was controlled mainly by dimensional reinforcement expectancy learned in the first stage. At the beginning of the second stage, pigeons correctly rejected 6-8 of 11 new added S- stimuli. A significant inverse correlation between the number of S- stimuli sharing dimension values with S + in the first stage and the dimensional discrimination ratios at the beginning of the second stage was found.

Noise-Correlation Is Modulated by Reward Expectation in the Primary Motor Cortex Bilaterally During Manual and Observational Tasks in Primates.

Reward modulation is represented in the motor cortex (M1) and could be used to implement more accurate decoding models to improve brain-computer interfaces (BCIs; Zhao et al., 2018). Analyzing trial-to-trial noise-correlations between neural units in the presence of rewarding (R) and non-rewarding (NR) stimuli adds to our understanding of cortical network dynamics. We utilized Pearson’s correlation coefficient to measure shared variability between simultaneously recorded units (32–112) and found significantly higher noise-correlation and positive correlation between the populations’ signal- and noise-correlation during NR trials as compared to R trials. This pattern is evident in data from two non-human primates (NHPs) during single-target center out reaching tasks, both manual and action observation versions. We conducted a mean matched noise-correlation analysis to decouple known interactions between event-triggered firing rate changes and neural correlations. Isolated reward discriminatory units demonstrated stronger correlational changes than units unresponsive to reward firing rate modulation, however, the qualitative response was similar, indicating correlational changes within the network as a whole can serve as another information channel to be exploited by BCIs that track the underlying cortical state, such as reward expectation, or attentional modulation. Reward expectation and attention in return can be utilized with reinforcement learning (RL) towards autonomous BCI updating.

Conditioning sharpens the spatial representation of rewarded stimuli in mouse primary visual cortex.

Reward is often employed as reinforcement in behavioral paradigms but it is unclear how the visuospatial aspect of a stimulus-reward association affects the cortical representation of visual space. Using a head-fixed paradigm, we conditioned mice to associate the same visual pattern in adjacent retinotopic regions with availability and absence of reward. Time-lapse intrinsic optical signal imaging under anesthesia showed that conditioning increased the spatial separation of mesoscale cortical representations of reward predicting- and non-reward predicting stimuli. Subsequent in vivo two-photon calcium imaging revealed that this improved separation correlated with enhanced population coding for retinotopic location, specifically for the trained orientation and spatially confined to the V1 region where rewarded and non-rewarded stimulus representations bordered. These results are corroborated by conditioning-induced differences in the correlation structure of population activity. Thus, the cortical representation of visual space is sharpened as consequence of associative stimulus-reward learning while the overall retinotopic map remains unaltered.

Associative Learning of Stimuli Paired and Unpaired With Reinforcement: Evaluating Evidence From Maggots, Flies, Bees, and Rats.

Finding rewards and avoiding punishments are powerful goals of behavior. To maximize reward and minimize punishment, it is beneficial to learn about the stimuli that predict their occurrence, and decades of research have provided insight into the brain processes underlying such associative reinforcement learning. In addition, it is well known in experimental psychology, yet often unacknowledged in neighboring scientific disciplines, that subjects also learn about the stimuli that predict the absence of reinforcement. Here we evaluate evidence for both these learning processes. We focus on two study cases that both provide a baseline level of behavior against which the effects of associative learning can be assessed. Firstly, we report pertinent evidence from Drosophila larvae. A re-analysis of the literature reveals that through paired presentations of an odor A and a sugar reward (A+) the animals learn that the reward can be found where the odor is, and therefore show an above-baseline preference for the odor. In contrast, through unpaired training (A/+) the animals learn that the reward can be found precisely where the odor is not, and accordingly these larvae show a below-baseline preference for it (the same is the case, with inverted signs, for learning through taste punishment). In addition, we present previously unpublished data demonstrating that also during a two-odor, differential conditioning protocol (A+/B) both these learning processes take place in larvae, i.e., learning about both the rewarded stimulus A and the non-rewarded stimulus B (again, this is likewise the case for differential conditioning with taste punishment). Secondly, after briefly discussing published evidence from adult Drosophila, honeybees, and rats, we report an unpublished data set showing that relative to baseline behavior after truly random presentations of a visual stimulus A and punishment, rats exhibit memories of opposite valence upon paired and unpaired training. Collectively, the evidence conforms to classical findings in experimental psychology and suggests that across species animals associatively learn both through paired and through unpaired presentations of stimuli with reinforcement – with opposite valence. While the brain mechanisms of unpaired learning for the most part still need to be uncovered, the immediate implication is that using unpaired procedures as a mnemonically neutral control for associative reinforcement learning may be leading analyses astray.

Long-Term Visual Training Increases Visual Acuity and Long-Term Monocular Deprivation Promotes Ocular Dominance Plasticity in Adult Standard Cage-Raised Mice

For routine behavioral tasks, mice predominantly rely on olfactory cues and tactile information. In contrast, their visual capabilities appear rather restricted, raising the question whether they can improve if vision gets more behaviorally relevant. We therefore performed long-term training using the visual water task (VWT): adult standard cage (SC)-raised mice were trained to swim toward a rewarded grating stimulus so that using visual information avoided excessive swimming toward nonrewarded stimuli. Indeed, and in contrast to old mice raised in a generally enriched environment (Greifzu et al., 2016), long-term VWT training increased visual acuity (VA) on average by more than 30% to 0.82 cycles per degree (cyc/deg). In an individual animal, VA even increased to 1.49 cyc/deg, i.e., beyond the rat range of VAs. Since visual experience enhances the spatial frequency threshold of the optomotor (OPT) reflex of the open eye after monocular deprivation (MD), we also quantified monocular vision after VWT training. Monocular VA did not increase reliably, and eye reopening did not initiate a decline to pre-MD values as observed by optomotry; VA values rather increased by continued VWT training. Thus, optomotry and VWT measure different parameters of mouse spatial vision. Finally, we tested whether long-term MD induced ocular dominance (OD) plasticity in the visual cortex of adult [postnatal day (P)162–P182] SC-raised mice. This was indeed the case: 40–50 days of MD induced OD shifts toward the open eye in both VWT-trained and, surprisingly, also in age-matched mice without VWT training. These data indicate that (1) long-term VWT training increases adult mouse VA, and (2) long-term MD induces OD shifts also in adult SC-raised mice.

A Simple Computational Model of the Bee Mushroom Body Can Explain Seemingly Complex Forms of Olfactory Learning and Memory

Honeybees are models for studying how animals with relatively small brains accomplish complex cognition, displaying seemingly advanced (or "non-elemental") learning phenomena involving multiple conditioned stimuli. These include "peak shift" [1-4]-where animals not only respond to entrained stimuli, but respond even more strongly to similar ones that are farther away from non-rewarding stimuli. Bees also display negative and positive patterning discrimination [5], responding in opposite ways to mixtures of two odors than to individual odors. Since Pavlov, it has often been assumed that such phenomena are more complex than simple associate learning. We present a model of connections between olfactory sensory input and bees' mushroom bodies [6], incorporating empirically determined properties of mushroom body circuitry (random connectivity [7], sparse coding [8], and synaptic plasticity [9, 10]). We chose not to optimize the model's parameters to replicate specific behavioral phenomena, because we were interested in the emergent cognitive capacities that would pop out of a network constructed solely based on empirical neuroscientific information and plausible assumptions for unknown parameters. We demonstrate that the circuitry mediating "simple" associative learning can also replicate the various non-elemental forms of learning mentioned above and can effectively multi-task by replicating a range of different learning feats. We found that PN-KC synaptic plasticity is crucial in controlling the generalization-discrimination trade-off-it facilitates peak shift and hinders patterning discrimination-and that PN-to-KC connection number can affect this trade-off. These findings question the notion that forms of learning that have been regarded as "higher order" are computationally more complex than "simple" associative learning.

Neural correlates of sample-coding and reward-coding in the delay activity of neurons in the entopallium and nidopallium caudolaterale of pigeons (Columba livia)

We recorded neuronal activity from the nidopallium caudolaterale, the avian equivalent of mammalian prefrontal cortex, and the entopallium, the avian equivalent of the mammalian visual cortex, in four birds trained on a differential outcomes delayed matching-to-sample procedure in which one sample stimulus was followed by reward and the other was not. Despite similar incidence of reward-specific and reward-unspecific delay cell types across the two areas, overall entopallium delay activity occurred following both rewarded and non-rewarded stimuli, whereas nidopallium caudolaterale delay activity tended to occur following the rewarded stimulus but not the non-rewarded stimulus. These findings are consistent with the view that delay activity in entopallium represents a code of the sample stimulus whereas delay activity in nidopallium caudolaterale represents a code of the possibility of an upcoming reward. However, based on the types of delay cells encountered, cells in NCL also code the sample stimulus and cells in ENTO are influenced by reward. We conclude that both areas support the retention of information, but that the activity in each area is differentially modulated by factors such as reward and attentional mechanisms.

Lesions of the ventral midline thalamus produce deficits in reversal learning and attention on an odor texture set shifting task

The nucleus reuniens (RE) of the ventral midline thalamus is strongly reciprocally connected with the hippocampus (HF) and the medial prefrontal cortex (mPFC) and has been shown to mediate the transfer of information between these structures. It has become increasingly well established that RE serves a critical role in mnemonic tasks requiring the interaction of the HF and mPFC, but essentially not tasks relying solely on the HF. Very few studies have addressed the independent actions of RE on prefrontal executive functioning. The present report examined the effects of lesions of the ventral midline thalamus, including RE and the dorsally adjacent rhomboid nucleus (RH) in rats on attention and behavioral flexibility using the attentional set shifting task (AST). The task uses odor and tactile stimuli to test for attentional set formation, attentional set shifting, behavioral flexibility and reversal learning. By comparison with sham controls, lesioned rats were significantly impaired on reversal learning and intradimensional (ID) set shifting. Specifically, RE/RH lesioned rats were impaired on the first reversal stage of the task which required a change in response strategy to select a previously non-rewarded stimulus for reward. RE/RH lesioned rats also exhibited deficits in the ability to transfer or generalize rules of the task which requires making the same modality-based choices (e.g., odor vs. tactile) to different sets of stimuli in the ID stage of the task. These results demonstrate that in addition to its role in tasks dependent on HF-mPFC interactions, nucleus reuniens is also critically involved cognitive/executive functions associated with the medial prefrontal cortex. As such, the deficits in the AST task produced by RE/RH lesions suggest the ventral midline thalamus directly contributes to flexible goal directed behavior.

Orbitofrontal cortical neurons encode expectation-driven initiation of reward-seeking.

Adaptive execution and inhibition of behavior are guided by the activity of neuronal populations across multiple frontal cortical areas. The rodent medial prefrontal cortex has been well studied with respect to these behaviors, influencing behavioral execution/inhibition based on context. Other frontal regions, in particular the orbitofrontal cortex (OFC), are critical in directing behavior to obtain rewards, but the relationship between OFC neuronal activity and response execution or inhibition has been poorly characterized. In particular, little is known about OFC with respect to extinction learning, an important example of context-guided response inhibition. Here, we recorded the activity of OFC neurons while rats performed a discriminative-stimulus (DS)-driven sucrose-seeking task followed by multiple days of extinction of the DS. OFC neuronal activity was maximally responsive (1) to reward-predicting stimuli (RS) that triggered a lever press (i.e., lever-response initiation) and (2) during reward-well approach in pursuit of sucrose (i.e., well-response initiation). RS presentation that was not followed by a lever press or RS presentation during extinction produced weak activation, as did nonrewarded stimulus (NS) presentation regardless of response (press or withhold) or session (DS-sucrose or extinction). Activity related to nonrewarded well entry was minor, and activity was significantly inhibited during reward consumption. Finally, OFC neuronal activity switched selectivity to track rewarded behaviors when the RS/NS contingencies were reversed. Thus, rather than signaling variables related to extinction or response inhibition, activity in OFC was strongest at the initiation of multiple components of reward-seeking behavior, most prominently when valid reward-predicting cues drove these behaviors.

Temperament trait Harm Avoidance associates with μ-opioid receptor availability in frontal cortex: A PET study using [11C]carfentanil

Harm Avoidance is a temperament trait that associates with sensitivity to aversive and non-rewarding stimuli, higher anticipated threat and negative emotions during stress as well as a higher risk for affective disorders. The neurobiological correlates of interindividual differences in Harm Avoidance are largely unknown. We hypothesized that variability in Harm Avoidance trait would be explained by differences in the activity of μ-opioid system as the opioid system is known to regulate affective states and stress sensitivity. Brain μ-opioid receptor availability was measured in 22 healthy subjects using positron emission tomography and [(11)C]carfentanil, a selective μ-opioid receptor agonist. The subjects were selected from a large Finish population-based cohort (N=2075) on the basis of their pre-existing Temperament and Character Scores. Subjects scoring consistently in the upper (10) and lower (12) quartiles for the Harm Avoidance trait were studied. High Harm Avoidance score associated with high μ-opioid receptor availability (i.e. lower endogenous μ-opioid drive) in anterior cingulate cortex, ventromedial and dorsolateral prefrontal cortices and anterior insular cortex. These associations were driven by two subscales of Harm Avoidance; Shyness with Strangers and Fatigability and Asthenia. In conclusion, higher Harm Avoidance score in healthy subjects is associated with higher μ-opioid availability in regions involved in the regulation of anxiety as well as in the control of emotions, affective component of pain and interoceptive awareness. The results have relevance in the research of vulnerability factors for affective disorders.

Modulation of neural activity by motivational and spatial biases

Motivational biases and spatial attention both modulate neural activity and influence behavioural performance. The time course of motivational bias effects, as well as the relationship between motivation and attention across the time course of information processing, however, are relatively unknown. In the present study, event-related potentials (ERPs) were recorded whilst individuals performed a modified Posner task, in which cue stimuli indicated the reward stakes of a given trial and the probable spatial location of a subsequent target stimulus. Reaction times (RTs) were sensitive to motivation and to attention, with faster responses produced on valid and on rewarded trials. In addition, motivation modulated neural activity from the visual analysis of stimuli, with an earlier N1 peak for rewarded compared with non-rewarded stimuli. Effects of motivation were relatively independent from those of attention until late cognitive processing and response production, where motivation and attention interacted to enhance P300-like potentials and the lateralised readiness potential (LRP). The results suggest that multiple sources of modulatory influences may exist, with motivation and attention exerting independent influences over early stimulus and cognitive processing, followed by a late interaction allowing the construction of a comprehensive stimulus representation that contains information pertaining to both motivational and spatial expectations.

Roles of Nucleus Accumbens Core and Shell in Incentive-Cue Responding and Behavioral Inhibition

The nucleus accumbens (NAc) is involved in many reward-related behaviors. The NAc has two major components, the core and the shell. These two areas have different inputs and outputs, suggesting that they contribute differentially to goal-directed behaviors. Using a discriminative stimulus (DS) task in rats and inactivating the NAc by blocking excitatory inputs with glutamate antagonists, we dissociated core and shell contributions to task performance. NAc core but not shell inactivation decreased responding to a reward-predictive cue. In contrast, inactivation of either subregion induced a general behavioral disinhibition. This reveals that the NAc actively suppresses actions inappropriate to the DS task. Importantly, selective inactivation of the shell but not core significantly increased responding to the nonrewarded cue. To determine whether the different contributions of the NAc core and shell depend on the information encoded in their constituent neurons, we performed electrophysiological recording in rats performing the DS task. Although there was no firing pattern unique to either core or shell, the reward-predictive cue elicited more frequent and larger magnitude responses in the NAc core than in the shell. Conversely, more NAc shell neurons selectively responded to the nonrewarded stimulus. These quantitative differences might account for the different behavioral patterns that require either core or shell. Neurons with similar firing patterns could also have different effects on behavior due to their distinct projection targets.

Dopamine signals for reward value and risk: basic and recent data

BackgroundPrevious lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli.MethodsThe described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks.ResultsDopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.ConclusionsNeurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

The emergence of saliency and novelty responses from Reinforcement Learning principles

Recent attempts to map reward-based learning models, like Reinforcement Learning [Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An introduction. Cambridge, MA: MIT Press], to the brain are based on the observation that phasic increases and decreases in the spiking of dopamine-releasing neurons signal differences between predicted and received reward [Gillies, A., & Arbuthnott, G. (2000). Computational models of the basal ganglia. Movement Disorders, 15(5), 762-770; Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1-27]. However, this reward-prediction error is only one of several signals communicated by that phasic activity; another involves an increase in dopaminergic spiking, reflecting the appearance of salient but unpredicted non-reward stimuli [Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4-6), 495-506; Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience, 96(4), 651-656; Redgrave, P., & Gurney, K. (2006). The short-latency dopamine signal: A role in discovering novel actions? Nature Reviews Neuroscience, 7(12), 967-975], especially when an organism subsequently orients towards the stimulus [Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1-27]. To explain these findings, Kakade and Dayan [Kakade, S., & Dayan, P. (2002). Dopamine: Generalization and bonuses. Neural Networks, 15(4-6), 549-559.] and others have posited that novel, unexpected stimuli are intrinsically rewarding. The simulation reported in this article demonstrates that this assumption is not necessary because the effect it is intended to capture emerges from the reward-prediction learning mechanisms of Reinforcement Learning. Thus, Reinforcement Learning principles can be used to understand not just reward-related activity of the dopaminergic neurons of the basal ganglia, but also some of their apparently non-reward-related activity.

Discrimination: From behaviour to brain

Discrimination is a skill needed by many organisms for survival: decisions about food, shelter, and mate selection all require the ability to distinguish among stimuli. This article reviews the how and why of discrimination and how researchers may exploit this natural skill in the laboratory to learn more about what features of stimuli animals use to discriminate. The paper then discusses the possible neurophysiological basis of discrimination and proposes a model, based on one of stimulus-association put forth by Beninger and Gerdjikov (2004) [Beninger, R.J., Gerdjikov, T.V., 2004. The role of signaling molecules in reward-related incentive learning. Neurotox. Res., 6, 91-104], to account for the role of dopamine in how an animal learns to discriminate rewarded from non-rewarded stimuli.

Perception of the stereokinetic illusion by the common marmoset (Callithrix jacchus)

Stereokinetic illusions have never been investigated in non-human primates, nor in other mammalian species. These illusions consist in the perception of a 3D solid object when certain 2D stimuli are rotated slowly in the plane perpendicular to the line of sight. The ability to perceive the stereokinetic illusion was investigated in the common marmoset (Callithrix jacchus). Four adult marmosets were trained to discriminate between a solid cylinder and a solid cone for food reward. Once learning criterion was reached, the marmosets were tested in sets of eight probe trials in which the two solid objects used at training were replaced by two rotating 2D stimuli. Only one of these stimuli produced, at least to the human observer, the stereokinetic illusion corresponding to the solid object previously reinforced. At test, the general behaviour and the total time spent by the marmosets observing each stimulus were recorded. The subjects stayed longer near the stimulus producing the stereokinetic illusion corresponding to the solid object reinforced at training than they did near the illusion corresponding to the previously non-rewarded stimulus. Hence, the common marmosets behaved as if they could perceive stereokinetic illusions.

Differential cingulate and caudate activation following unexpected nonrewarding stimuli

This study examined the effects of varying the predictability of nonrewarding events on behavior and neural activation using a rapid mixed-trial functional magnetic resonance imagery (fMRI) design. Twelve adult subjects were scanned with echo planar imaging during performance of a visual detection task where the probability of events (target and nontarget) varied. This task included expected and unexpected nonrewarding events (expected target, unexpected nontarget, and omission of target) in a design that closely parallels studies of dopamine function and reward processing in the alert monkey. We predicted that activation in dopamine-rich areas of the forebrain would behave like the animal literature shows that dopamine neurons in the midbrain behave. Specifically, we predicted increased activity in these regions when an unexpected event occurred and decreased activity when an expected event was omitted. Two main regions, the anterior cingulate and dorsal striatum, showed this pattern. The response in these regions was distinguished by enhanced anterior cingulate activity following the occurrence of an unexpected event and greater suppression of caudate activity following the omission of an expected event. These results suggest that neural activity within specific dopamine-rich brain regions can be modulated by violations in the expectation of nonrewarding events and that the direction of the modulation depends on the nature of the violations.