Encoding Of Prediction Errors Research Articles

Anatomical inputs to sulcal portions of areas 9m and 8Bm in the macaque monkey.

Neuronal activities recorded from the dorsal bank of the anterior cingulate sulcus have suggested that this cortical area is involved in control of search vs. repetition, goal-based action selection and encoding of prediction error regarding action value. In this study, to explore potential anatomical bases for these neuronal activities, we injected retrograde tracers (CTB-Alexa-488 and CTB-gold) into the dorsal bank of the anterior cingulate sulcus and examined the distribution of labeled cell bodies in macaque monkey brains. The Nissl staining showed that the cortex in the dorsal bank of the anterior cingulate sulcus has consistent layer 4 which means that the cortical region is a part of the granular prefrontal cortex. The injection site belonged to the sulcal portion of area 9m in two cases and the sulcal portion of area 8Bm in one case. In addition to the continuous distribution of labeled cells in the two areas (areas 9m and 8Bm) around the injection site, the labeled cells were densely distributed in the cingulate areas (areas 24, 32, and 23) in all the cases. The dense labeling of cells was also found in other prefrontal areas (areas 46, 10, 11, and 12) in the two cases with injection into the sulcal portion of area 9m, whereas the dense labeling of cells was found in pre-motor areas (F6 and F7) in the case with injection into the sulcal portion of area 8Bm. The dense labeling of cells in the prefrontal and premotor areas was more similar to those previously found after injections into dorsal parts of areas 9 and 8B. Subcortical distribution of labeled cells was found in the mediodorsal nucleus of thalamus, claustrum, and substantia nigra pars compacta in all the cases.

Altered neural reward and loss processing and prediction error signalling in depression.

Dysfunctional processing of reward and punishment may play an important role in depression. However, functional magnetic resonance imaging (fMRI) studies have shown heterogeneous results for reward processing in fronto-striatal regions. We examined neural responsivity associated with the processing of reward and loss during anticipation and receipt of incentives and related prediction error (PE) signalling in depressed individuals. Thirty medication-free depressed persons and 28 healthy controls performed an fMRI reward paradigm. Regions of interest analyses focused on neural responses during anticipation and receipt of gains and losses and related PE-signals. Additionally, we assessed the relationship between neural responsivity during gain/loss processing and hedonic capacity. When compared with healthy controls, depressed individuals showed reduced fronto-striatal activity during anticipation of gains and losses. The groups did not significantly differ in response to reward and loss outcomes. In depressed individuals, activity increases in the orbitofrontal cortex and nucleus accumbens during reward anticipation were associated with hedonic capacity. Depressed individuals showed an absence of reward-related PEs but encoded loss-related PEs in the ventral striatum. Depression seems to be linked to blunted responsivity in fronto-striatal regions associated with limited motivational responses for rewards and losses. Alterations in PE encoding might mirror blunted reward- and enhanced loss-related associative learning in depression.

Effective Connectivity of the Human Cerebellum during Visual Attention

Insights from both lesion and neuroimaging studies increasingly substantiate the view that the human cerebellum not only serves motor control but also supports various cognitive processes. Higher cognitive functions like working memory or executive control have been associated with the phylogenetically younger parts of the cerebellum, crus I and crus II. Functional connectivity studies corroborate this notion as activation of the cerebellum correlates with activity in numerous areas of the cerebral cortex. Moreover, these cerebrocerebellar loops were shown to be topographically organized. We used an attention-to-motion paradigm to elaborate on the effective connectivity of cerebellar crus I during visual attention. Psychophysiological interaction analyses demonstrated enhanced connectivity of the cerebellum--during attention--with dorsal visual stream regions including posterior parietal cortex (PPC) and left secondary visual cortex (V5). Dynamic causal modeling revealed a modulation of the connections from V5 to PPC and from crus I to V5 by attention. Remarkably, the influence which V5 exerted on PPC was reduced during attention, resulting in a suppression of the sensitivity of PPC to bottom-up information. Moreover, the sensitivity of V5 populations to inputs from crus I was increased under attention. This might underscore the presumed role of the cerebellum as a state estimator that provides hierarchically lower regions (V5) with top-down predictions, which in turn might be based on endogenous inputs from PPC to the cerebellum. These results are in line with formulations of attention in predictive coding, where attention increases the precision or sensitivity of hierarchically lower neuronal populations that may encode prediction error.

Human Dorsal Striatum Encodes Prediction Errors during Observational Learning of Instrumental Actions

The dorsal striatum plays a key role in the learning and expression of instrumental reward associations that are acquired through direct experience. However, not all learning about instrumental actions require direct experience. Instead, humans and other animals are also capable of acquiring instrumental actions by observing the experiences of others. In this study, we investigated the extent to which human dorsal striatum is involved in observational as well as experiential instrumental reward learning. Human participants were scanned with fMRI while they observed a confederate over a live video performing an instrumental conditioning task to obtain liquid juice rewards. Participants also performed a similar instrumental task for their own rewards. Using a computational model-based analysis, we found reward prediction errors in the dorsal striatum not only during the experiential learning condition but also during observational learning. These results suggest a key role for the dorsal striatum in learning instrumental associations, even when those associations are acquired purely by observing others.

Mismatch Negativity Encoding of Prediction Errors Predicts S-ketamine-Induced Cognitive Impairments

Psychotomimetics like the N-methyl-D-aspartate receptor (NMDAR) antagonist ketamine and the 5-hydroxytryptamine2A receptor (5-HT(2A)R) agonist psilocybin induce psychotic symptoms in healthy volunteers that resemble those of schizophrenia. Recent theories of psychosis posit that aberrant encoding of prediction errors (PE) may underlie the expression of psychotic symptoms. This study used a roving mismatch negativity (MMN) paradigm to investigate whether the encoding of PE is affected by pharmacological manipulation of NMDAR or 5-HT(2A)R, and whether the encoding of PE under placebo can be used to predict drug-induced symptoms. Using a double-blind within-subject placebo-controlled design, S-ketamine and psilocybin, respectively, were administrated to two groups of healthy subjects. Psychological alterations were assessed using a revised version of the Altered States of Consciousness (ASC-R) questionnaire. As an index of PE, we computed changes in MMN amplitudes as a function of the number of preceding standards (MMN memory trace effect) during a roving paradigm. S-ketamine, but not psilocybin, disrupted PE processing as expressed by a frontally disrupted MMN memory trace effect. Although both drugs produced positive-like symptoms, the extent of PE processing under placebo only correlated significantly with the severity of cognitive impairments induced by S-ketamine. Our results suggest that the NMDAR, but not the 5-HT(2A)R system, is implicated in PE processing during the MMN paradigm, and that aberrant PE signaling may contribute to the formation of cognitive impairments. The assessment of the MMN memory trace in schizophrenia may allow detecting early phases of the illness and might also serve to assess the efficacy of novel pharmacological treatments, in particular of cognitive impairments.

Expected value and prediction error abnormalities in depression and schizophrenia

The dopamine system has been linked to anhedonia in depression and both the positive and negative symptoms of schizophrenia, but it remains unclear how dopamine dysfunction could mechanistically relate to observed symptoms. There is considerable evidence that phasic dopamine signals encode prediction error (differences between expected and actual outcomes), with reinforcement learning theories being based on prediction error-mediated learning of associations. It has been hypothesized that abnormal encoding of neural prediction error signals could underlie anhedonia in depression and negative symptoms in schizophrenia by disrupting learning and blunting the salience of rewarding events, and contribute to psychotic symptoms by promoting aberrant perceptions and the formation of delusions. To test this, we used model based functional magnetic resonance imaging and an instrumental reward-learning task to investigate the neural correlates of prediction errors and expected-reward values in patients with depression (n=15), patients with schizophrenia (n=14) and healthy controls (n=17). Both patient groups exhibited abnormalities in neural prediction errors, but the spatial pattern of abnormality differed, with the degree of abnormality correlating with syndrome severity. Specifically, reduced prediction errors in the striatum and midbrain were found in depression, with the extent of signal reduction in the bilateral caudate, nucleus accumbens and midbrain correlating with increased anhedonia severity. In schizophrenia, reduced prediction error signals were observed in the caudate, thalamus, insula and amygdala-hippocampal complex, with a trend for reduced prediction errors in the midbrain, and the degree of blunting in the encoding of prediction errors in the insula, amygdala-hippocampal complex and midbrain correlating with increased severity of psychotic symptoms. Schizophrenia was also associated with disruption in the encoding of expected-reward values in the bilateral amygdala-hippocampal complex and parahippocampal gyrus, with the degree of disruption correlating with psychotic symptom severity. Neural signal abnormalities did not correlate with negative symptom severity in schizophrenia. These findings support the suggestion that a disruption in the encoding of prediction error signals contributes to anhedonia symptoms in depression. In schizophrenia, the findings support the postulate of an abnormality in error-dependent updating of inferences and beliefs driving psychotic symptoms. Phasic dopamine abnormalities in depression and schizophrenia are suggested by our observation of prediction error abnormalities in dopamine-rich brain areas, given the evidence for dopamine encoding prediction errors. The findings are consistent with proposals that psychiatric syndromes reflect different disorders of neural valuation and incentive salience formation, which helps bridge the gap between biological and phenomenological levels of understanding.

A Lossless Embedded Compression Using Significant Bit Truncation for HD Video Coding

Increasing the image size of a video sequence aggravates the memory bandwidth problem of a video coding system. Despite many embedded compression (EC) algorithms proposed to overcome this problem, no lossless EC algorithm able to handle high-definition (HD) size video sequences has been proposed thus far. In this paper, a lossless EC algorithm for HD video sequences and related hardware architecture is proposed. The proposed algorithm consists of two steps. The first is a hierarchical prediction method based on pixel averaging and copying. The second step involves significant bit truncation (SBT) which encodes prediction errors in a group with the same number of bits so that the multiple prediction errors are decoded in a clock cycle. The theoretical lower bound of the compression ratio of the SBT coding was also derived. Experimental results have shown a 60% reduction of memory bandwidth on average. Hardware implementation results have shown that a throughput of 14.2 pixels/cycle can be achieved with 36 K gates, which is sufficient to handle HD-size video sequences in real time.

PL02-01 - Neurobiology of reward systems: relevance for learning, addiction and psychosis

Neurotransmitter systems such as dopamine and glutamate play a major role in encoding prediction errors and in learning of cue-triggered and goal-directed behaviour. Dysfunction of these neurotransmitter systems affect prefrontal-limbic neurocircuits and thus contribute to the formation of delusions, and to motivational and affective deficits. For example, reduced activation of the ventral striatum during reward anticipation has been associated with craving for drugs of abuse and with motivational deficits such as apathy, a so-called negative symptom in schizophrenia. Dopamine-dependent learning dysfunctions in alcohol dependence impairs learning of alternative, non substance-associated behaviour. On the other hand, aberrant activation during feedback of negative outcomes correlated with delusion formation in schizophrenic psychoses. Multimodal imaging studies implied glutamatergic and dopaminergic dysfunction to be associated with altered functional brain activation. Human genetic and imaging studies revealed that candidate genes, which affect monoamine metabolism, interact with neurotransmission and task-relevant activation in these brain areas; furthermore, first and second-generation neuroleptics differentially interact with functional brain activation in these neurocircuits. Together, these findings suggest that the combination of genetic and imaging studies may help to elucidate the neurobiologcial correlates of dysfunctional learning processes in psychosis and addiction and to identify new targets for behavioural and pharmacological interventions.

A Class-Conditioned Lossless Wavelet-Based Predictive Multispectral Image Compressor

We present a nonlinear lossless compressor designed for multispectral images consisting of few bands and having greater spatial than spectral correlation. Our compressor is based on a 2-D integer wavelet transform that reduces spatial correlation. Different models for the statistical dependences of wavelet detail coefficients are analyzed and tested to perform linear inter/intraband predictions. Band, class, scale, and orientation are used as conditioning contexts to calculate predictions, as well as to encode prediction errors with an adaptive arithmetic coder. A new mechanism is proposed for band ordering, based on wavelet fine detail coefficients. Our compressor CLWP outperforms state-of-the-art lossless compressors. It has random access capability and can be applied to compress volumetric data having similar characteristics.

Subliminal Instrumental Conditioning Demonstrated in the Human Brain

SummaryHow the brain uses success and failure to optimize future decisions is a long-standing question in neuroscience. One computational solution involves updating the values of context-action associations in proportion to a reward prediction error. Previous evidence suggests that such computations are expressed in the striatum and, as they are cognitively impenetrable, represent an unconscious learning mechanism. Here, we formally test this by studying instrumental conditioning in a situation where we masked contextual cues, such that they were not consciously perceived. Behavioral data showed that subjects nonetheless developed a significant propensity to choose cues associated with monetary rewards relative to punishments. Functional neuroimaging revealed that during conditioning cue values and prediction errors, generated from a computational model, both correlated with activity in ventral striatum. We conclude that, even without conscious processing of contextual cues, our brain can learn their reward value and use them to provide a bias on decision making.

Differential Encoding of Losses and Gains in the Human Striatum

Studies on human monetary prediction and decision making emphasize the role of the striatum in encoding prediction errors for financial reward. However, less is known about how the brain encodes financial loss. Using Pavlovian conditioning of visual cues to outcomes that simultaneously incorporate the chance of financial reward and loss, we show that striatal activation reflects positively signed prediction errors for both. Furthermore, we show functional segregation within the striatum, with more anterior regions showing relative selectivity for rewards and more posterior regions for losses. These findings mirror the anteroposterior valence-specific gradient reported in rodents and endorse the role of the striatum in aversive motivational learning about financial losses, illustrating functional and anatomical consistencies with primary aversive outcomes such as pain.

Recent Developments in Context-Based Predictive Techniques for Lossless Image Compression

In this paper we describe some recent developments that have taken place in context-based predictive coding, in response to the JPEG/JBIG committee’s recent call for proposals for a new international standard on lossless compression of continuous-tone images. We describe the different prediction techniques that were proposed and give a performance comparison. We describe the notion of context-based bias cancellation, which represents one of the key ideas that was proposed and incorporated in the final standard. We also describe the different error modelling and entropy coding techniques that were proposed for encoding prediction errors, the most important development here being an ingeniously simple and effective technique for adaptive Golomb‐Rice coding. We conclude with a short discussion on avenues for future research.