Perceptual Inference Research Articles

Perceptual inference of dynamic emotion in natural movies

Perceptual inference of dynamic emotion in natural movies

Feeling stiffness in the back: a protective perceptual inference in chronic back pain

Does feeling back stiffness actually reflect having a stiff back? This research interrogates the long-held question of what informs our subjective experiences of bodily state. We propose a new hypothesis: feelings of back stiffness are a protective perceptual construct, rather than reflecting biomechanical properties of the back. This has far-reaching implications for treatment of pain/stiffness but also for our understanding of bodily feelings. Over three experiments, we challenge the prevailing view by showing that feeling stiff does not relate to objective spinal measures of stiffness and objective back stiffness does not differ between those who report feeling stiff and those who do not. Rather, those who report feeling stiff exhibit self-protective responses: they significantly overestimate force applied to their spine, yet are better at detecting changes in this force than those who do not report feeling stiff. This perceptual error can be manipulated: providing auditory input in synchrony to forces applied to the spine modulates prediction accuracy in both groups, without altering actual stiffness, demonstrating that feeling stiff is a multisensory perceptual inference consistent with protection. Together, this presents a compelling argument against the prevailing view that feeling stiff is an isomorphic marker of the biomechanical characteristics of the back.

Sensory noise predicts divisive reshaping of receptive fields.

In order to respond reliably to specific features of their environment, sensory neurons need to integrate multiple incoming noisy signals. Crucially, they also need to compete for the interpretation of those signals with other neurons representing similar features. The form that this competition should take depends critically on the noise corrupting these signals. In this study we show that for the type of noise commonly observed in sensory systems, whose variance scales with the mean signal, sensory neurons should selectively divide their input signals by their predictions, suppressing ambiguous cues while amplifying others. Any change in the stimulus context alters which inputs are suppressed, leading to a deep dynamic reshaping of neural receptive fields going far beyond simple surround suppression. Paradoxically, these highly variable receptive fields go alongside and are in fact required for an invariant representation of external sensory features. In addition to offering a normative account of context-dependent changes in sensory responses, perceptual inference in the presence of signal-dependent noise accounts for ubiquitous features of sensory neurons such as divisive normalization, gain control and contrast dependent temporal dynamics.

Visual Hallucinations Are Characterized by Impaired Sensory Evidence Accumulation: Insights From Hierarchical Drift Diffusion Modeling in Parkinson’s Disease

BackgroundModels of hallucinations emphasize imbalance between sensory input and top-down influences over perception, as false perceptual inference can arise when top-down predictions are afforded too much precision (certainty) relative to sensory evidence. Visual hallucinations in Parkinson’s disease (PD) are associated with lower-level visual and attentional impairments, accompanied by overactivity in higher-order association brain networks. PD therefore provides an attractive framework to explore contributions of bottom-up versus top-down disturbances in hallucinations. MethodsWe characterized sensory processing during perceptual decision making in patients with PD with (n = 20) and without (n = 25) visual hallucinations and control subjects (n = 12), by fitting a hierarchical drift diffusion model to an attentional task. The hierarchical drift diffusion model uses Bayesian estimates to decompose task performance into parameters reflecting drift rates of evidence accumulation, decision thresholds, and nondecision time. ResultsWe observed slower drift rates in patients with hallucinations, which were less sensitive to changes in task demand. In contrast, wider decision boundaries and shorter nondecision times relative to control subjects were found in patients with PD regardless of hallucinator status. Inefficient and less flexible sensory evidence accumulation emerges as a unique feature of PD hallucinators. ConclusionsWe integrate these results with evidence accumulation and predictive coding models of hallucinations, suggesting that in PD sensory evidence is less informative and may therefore be down-weighted, resulting in overreliance on top-down influences. Considering impaired drift rates as an approximation of reduced sensory precision, our findings provide a novel computational framework to specify impairments in sensory processing that contribute to development of visual hallucinations.

Investigating audition with a generative model of impact sounds

When objects collide they vibrate and emit sound. Physical laws govern these collisions and subsequent vibrations. As a result, sound contains information about objects (density/hardness/size/shape), and the manner in which they collide (bouncing/rolling/scraping). Everyday experience suggests that human listeners have some ability to discern material and kinematics from impact sounds. However, the accuracy of these perceptual inferences remains unclear, and the underlying mechanisms are uncharacterized. Listeners could rely on stored templates for particular familiar objects. Alternatively, we could infer generative parameters for a sound via probabilistic inference in an internal model of the generative process. To explore these possibilities we constructed a generative model of impact sounds, modeling sounds as the convolution of a time-varying impact force with the impulse responses (IRs) of two objects. The force was modeled as a function of mass, hardness and impact velocity. IRs were measured from a range of objects using contact speakers and microphones to measure an object’s effect on vibrational input. IRs for arbitrary objects were generated via interpolation between recorded examples. The model generates compelling renditions of impact sounds. Physically motivated alterations to the force and/or IRs produced physically plausible synthetic impact sounds that can be used in perceptual experiments.

First, you need a Gestalt: An interaction of bottom-up and top-down streams during the perception of the ambiguously rotating human walker

Our visual system combines sensory evidence with prior knowledge to produce a representation of an outside world. Here, we explored the limits of the feedforward computation using an ambiguously rotating human biological motion. Specifically, we investigated whether an overall rotation, which was added to all the displays used in the study, would be perceived when the point-light walker was presented upside-down, a condition that typically obliterates perception of a human Gestalt. We report that inversion of the point-light walker or the absence of an identifiable Gestalt abolished the perception of an overall rotation. Perception of rotation was restored if the human walker Gestalt could be identified (an upright walker), if observers were informed about the nature of the motion display, or if observers expected to see the rotation of an unknown dynamic object. This implies that a mathematically more complex human motion was accounted for before the remaining motion components could be used to infer an overall rotation. Our results indicate that the perceptual inference does not proceed in a hierarchical manner with the simpler components being identified first. Instead, prior knowledge acts as a starting point for the decomposition of an even relatively simple combination of two motions.

The interaction of perceptual biases in bistable perception

When viewing ambiguous stimuli, people tend to perceive some interpretations more frequently than others. Such perceptual biases impose various types of constraints on visual perception, and accordingly, have been assumed to serve distinct adaptive functions. Here we demonstrated the interaction of two functionally distinct biases in bistable biological motion perception, one regulating perception based on the statistics of the environment – the viewing-from-above (VFA) bias, and the other with the potential to reduce costly errors resulting from perceptual inference – the facing-the-viewer (FTV) bias. When compatible, the two biases reinforced each other to enhance the bias strength and induced less perceptual reversals relative to when they were in conflict. Whereas in the conflicting condition, the biases competed with each other, with the dominant percept varying with visual cues that modulate the two biases separately in opposite directions. Crucially, the way the two biases interact does not depend on the dominant bias at the individual level, and cannot be accounted for by a single bias alone. These findings provide compelling evidence that humans robustly integrate biases with different adaptive functions in visual perception. It may be evolutionarily advantageous to dynamically reweight diverse biases in the sensory context to resolve perceptual ambiguity.

Perceptual distortions and deceptions: what computers can teach us

The nature of perception has fascinated philosophers for centuries, and has more recently been the focus of research in psychology and neuroscience. Many psychiatric disorders are characterised by perceptual abnormalities, ranging from sensory distortions to illusions and hallucinations. The distinction between normal and abnormal perception is, however, hard to articulate. In this article we argue that the distinction between normal perception and abnormal perception is best seen as a quantitative one, resting on the degree to which the observer's prior expectations influence perceptual inference. We illustrate this point with an example taken from researchers at Google working on computer vision.

FMRI-based decoding of reward effects in binocular rivalry.

Binocular rivalry is a phenomenon where the simultaneous presentation of two different stimuli to the two eyes leads to alternating perception of the two stimuli. The temporary dominance of one stimulus over the other is influenced by several factors. Here, we studied the influence of reward on binocular rivalry dynamics and its neural representation in visual cortex. Orthogonal rotating grating stimuli were shown continuously, while monetary reward was given during the conscious perception of one stimulus but not the other. Periods of perceptual dominance were assessed both through participants’ subjective report and objectively using functional magnetic resonance imaging and multi-voxel pattern analysis. Results did not confirm previous evidence for an effect of reward on perceptual dominance durations. Exploratory post-hoc analyses indicated that knowledge regarding both the reward contingency and the subjective nature of perceptual alternations may have interfered with potential reward effects on perceptual phase durations, suggesting a moderating role of meta-cognitive awareness in reward-based perceptual inference. Future studies of top-down influences on bistable perception should carefully consider the methodological challenges related to meta-cognitive awareness.

A Hierarchical Predictive Coding Model of Object Recognition in Natural Images

Predictive coding has been proposed as a model of the hierarchical perceptual inference process performed in the cortex. However, results demonstrating that predictive coding is capable of performing the complex inference required to recognise objects in natural images have not previously been presented. This article proposes a hierarchical neural network based on predictive coding for performing visual object recognition. This network is applied to the tasks of categorising hand-written digits, identifying faces, and locating cars in images of street scenes. It is shown that image recognition can be performed with tolerance to position, illumination, size, partial occlusion, and within-category variation. The current results, therefore, provide the first practical demonstration that predictive coding (at least the particular implementation of predictive coding used here; the PC/BC-DIM algorithm) is capable of performing accurate visual object recognition.

Semantic Image Segmentation Method with Multiple Adjacency Trees and Multiscale Features

Semantic image segmentation is the basis of image understanding, which is one of the most important human cognitive activities. Cognitive studies have shown that human neocortical information transmission depends on cognitive processing at multiple scales, and contextual information aids the human cognitive system in solving perceptual inference tasks. Inspired by multiscale cognitive mechanisms and contextual effects, in this paper, we propose a semantic image segmentation method addressing multiscale features and contextual information. To integrate multiscale features, after over-segmenting an image into small-scale segments, we employ a segment-based classifier and a CRF (conditional random field) model to generate large-scale regions. We then use the features of regions to train a region-based classifier. To capture context, we propose a multiple adjacency tree model where each tree represents one type of region relevance and can be generated by the adjacency graph corresponding to that relevance metric. Using the multiple tree model instead of a general graph model, we can perform exact inference with some simple assumptions and capture multiple types of regional context dependency. Experiments on the MSRC-21 and Stanford background datasets show advantages of our method over a segment-based CRF model using single-scale features. The results demonstrate the importance of multiscale features and contextual information.

Task modulations and clinical manifestations in the brain functional connectome in 1615 fMRI datasets

ObjectiveAn abundance of experimental studies have motivated a range of models concerning the cognitive underpinnings of severe mental disorders, yet the conception that cognitive and brain dysfunction is confined to specific cognitive domains and contexts has limited ecological validity. Schizophrenia and bipolar spectrum disorders have been conceptualized as disorders of brain connectivity; yet little is known about the pervasiveness across cognitive tasks. MethodsTo address this outstanding issue of context specificity, we estimated functional network connectivity from fMRI data obtained during five cognitive tasks (0-back, 2-back, go/no-go, recognition of positive faces, negative faces) in 90 patients with schizophrenia spectrum, 97 patients with bipolar spectrum disorder, and 136 healthy controls, including 1615 fMRI datasets in total. We tested for main effects of task and group, and their interactions, and used machine learning to classify task labels and predict cognitive domain scores from brain connectivity. ResultsConnectivity profiles were positively correlated across tasks, supporting the existence of a core functional connectivity backbone common to all tasks. However, 76.2% of all network links also showed significant task-related alterations, robust on the single subject level as evidenced by high machine-learning performance when classifying task labels. Independent of such task-specific modulations, 9.5% of all network links showed significant group effects, particularly including sensory (sensorimotor, visual, auditory) and cognitive (frontoparietal, default-mode, dorsal attention) networks. A lack of group by task interactions revealed that the pathophysiological sensitivity remained across tasks. Such pervasiveness across tasks was further supported by significant predictions of cognitive domain scores from the connectivity backbone obtained across tasks. ConclusionsThe high accuracies obtained when classifying cognitive tasks support that brain connectivity indices provide sensitive and specific measures of cognitive states. Importantly, we provide evidence that brain network dysfunction in severe mental disorders is not confined to specific cognitive tasks and show that the connectivity backbone common to all tasks is predictive of cognitive domain traits. Such pervasiveness across tasks may support a generalization of pathophysiological models from different domains, thereby reducing their complexity and increasing their ecological validity. Future research incorporating a wider range of cognitive tasks, involving other sensory modalities (auditory, somatosensory, motor) and requirements (learning, perceptual inference, decision making, etc.), is needed to assess if under certain circumstances, context dependent aberrations may evolve. Our results provide further evidence from a large sample that fMRI based functional network connectivity can be used to reveal both, state and trait effects in the connectome.

Why Harmless Sensations Might Hurt in Individuals with Chronic Pain: About Heightened Prediction and Perception of Pain in the Mind.

In individuals with chronic pain harmless bodily sensations can elicit anticipatory fear of pain resulting in maladaptive responses such as taking pain medication. Here, we aim to broaden the perspective taking into account recent evidence that suggests that interoceptive perception is largely a construction of beliefs, which are based on past experience and that are kept in check by the actual state of the body. Taking a Bayesian perspective, we propose that individuals with chronic pain display a heightened prediction of pain [prior probability p(pain)], which results in heightened pain perception [posterior probability p(pain|sensation)] due to an assumed link between pain and a harmless bodily sensation [p(sensation|pain)]. This pain perception emerges because their mind infers pain as the most likely cause for the sensation. When confronted with a mismatch between predicted pain and a (harmless bodily) sensation, individuals with chronic pain try to minimize the mismatch most likely by active inference of pain or alternatively by an attentional shift away from the sensation. The active inference results in activities that produce a stronger sensation that will match with the prediction, allowing subsequent perceptual inference of pain. Here, we depict heightened pain perception in individuals with chronic pain by reformulating and extending the assumptions of the interoceptive predictive coding model from a Bayesian perspective. The review concludes with a research agenda and clinical considerations.

Seeing a straight line on a curved surface: decoupling of patterns from surfaces by single IT neurons.

We have no difficulty seeing a straight line on a curved piece of paper, but in fact, doing so requires decoupling the shape of the surface from the pattern itself. Here we report a novel form of invariance in the visual cortex: single neurons in monkey inferior temporal cortex respond similarly to congruent transformations of patterns and surfaces, in effect decoupling patterns from the surface on which they are overlaid.

Blocking serotonin but not dopamine reuptake alters neural processing during perceptual decision making.

Dopamine and serotonin have opponent interactions on aspects of impulsivity. Therefore we wanted to test the hypothesis that dopamine and serotonin would have opposing effects on speed-accuracy trade offs in a perceptual decision making task. Unlike other behavioral measures of impulsivity, perceptual decision making allows us to determine whether decreasing premature responses, often interpreted as decreased impulsivity, corresponds to increased behavioral performance. We administered GBR-12909 (a dopamine transporter blocker), escitalopram (a serotonin transporter blocker), or saline in separate sessions to 3 rhesus macaques. We found that animals had slower reaction times (RTs) on escitalopram than on GBR-12909 or saline. However, they were also least accurate on escitalopram. Animals were faster, although nonsignificantly, on GBR than saline and had equivalent accuracy. Administration of GBR-12909 did cause animals to be faster in error trials than correct trials. Therefore, from the point of view of RTs the animals were less impulsive on escitalopram. However, the decreased accuracy of the monkeys shows that they were not able to make use of their slower response times to make more accurate decisions. Therefore, impulsivity was reduced on escitalopram, but at the expense of a slower information-processing rate in the perceptual inference task. (PsycINFO Database Record

Perceptual Inferences in Schizophrenia: A preliminary study in healthy participants

Perceptual Inferences in Schizophrenia: A preliminary study in healthy participants

Perceptual Inference: A Matter of Predictions and Errors

A recent study finds that separate populations of neurons in inferotemporal cortex code for perceptual predictions and prediction errors, supporting predictive coding theories of perception.

Prediction error, ketamine and psychosis: An updated model.

In 2007, we proposed an explanation of delusion formation as aberrant prediction error-driven associative learning. Further, we argued that the NMDA receptor antagonist ketamine provided a good model for this process. Subsequently, we validated the model in patients with psychosis, relating aberrant prediction error signals to delusion severity. During the ensuing period, we have developed these ideas, drawing on the simple principle that brains build a model of the world and refine it by minimising prediction errors, as well as using it to guide perceptual inferences. While previously we focused on the prediction error signal per se, an updated view takes into account its precision, as well as the precision of prior expectations. With this expanded perspective, we see several possible routes to psychotic symptoms – which may explain the heterogeneity of psychotic illness, as well as the fact that other drugs, with different pharmacological actions, can produce psychotomimetic effects. In this article, we review the basic principles of this model and highlight specific ways in which prediction errors can be perturbed, in particular considering the reliability and uncertainty of predictions. The expanded model explains hallucinations as perturbations of the uncertainty mediated balance between expectation and prediction error. Here, expectations dominate and create perceptions by suppressing or ignoring actual inputs. Negative symptoms may arise due to poor reliability of predictions in service of action. By mapping from biology to belief and perception, the account proffers new explanations of psychosis. However, challenges remain. We attempt to address some of these concerns and suggest future directions, incorporating other symptoms into the model, building towards better understanding of psychosis.

Perceptual Inference in Chronic Pain: An Investigation Into the Economy of Action Hypothesis.

The experience of chronic pain critically alters one's ability to interact with their environment. One fundamental issue that has received little attention, however, is whether chronic pain disrupts how one perceives their environment in the first place. The Economy of Action hypothesis purports that the environment is spatially scaled according to the ability of the observer. Under this hypothesis it has been proposed that the perception of the world is different between those with and without chronic pain. Such a possibility has profound implications for the investigation and treatment of pain. The present investigation tested the application of this hypothesis to a heterogenous chronic pain population. Individuals with chronic pain (36; 27F) and matched pain-free controls were recruited. Each participant was required to judge the distance to a series of target cones, to which they were to subsequently walk. In addition, at each distance, participants used Numerical Rating Scales to indicate their perceived effort and perceived pain associated with the distance presented. Our findings do not support the Economy of Action hypothesis: there were no significant differences in distance estimates between the chronic pain and pain-free groups (F1,60=0.927; P=0.340). In addition, we found no predictive relationship in the chronic pain group between anticipated pain and estimated distance (F1,154=0.122, P=0.727), nor anticipated effort (1.171, P=0.281) and estimated distance (F1,154=1.171, P=0.281). The application of the Economy of Action hypothesis and the notion of spatial perceptual scaling as a means to assess and treat the experience of chronic pain are not supported by the results of this study.

Bayesian inference, dysconnectivity and neuromodulation in schizophrenia.

This scientific commentary refers to ‘Estimating changing contexts in schizophrenia’, by Kaplan et al. (doi:10.1093/brain/aww095). The paper by Kaplan et al. in this issue of Brain addresses one of the most interesting questions in contemporary schizophrenia research: the role of uncertainty during perception (Kaplan et al. , 2016). Uncertainty enjoys much interest in schizophrenia research as it may provide a crucial link between core clinical symptoms of schizophrenia—aberrant perceptual inference (e.g. hallucinations) and abnormal beliefs (delusions)—and long-standing neurobiological findings that patients with schizophrenia display widespread alterations in structural and functional brain connectivity (dysconnectivity). These two cardinal features of schizophrenia have been integrated in disease theories, which have developed in three waves. A first influential proposal was that dysconnectivity in schizophrenia arises from abnormal regulation of NMDA receptor (NMDAR)-dependent transmission by neuromodulatory (dopaminergic and cholinergic) influences (Friston, 1998). Given the critical role of NMDARs for synaptic plasticity and myelination, this suggested that both neurodevelopmental aspects of schizophrenia ( cf . abnormal pruning of connections by altered experience-dependent plasticity) and structural dysconnectivity might arise from a primary disturbance of NMDAR-dependent plasticity due to aberrant neuromodulatory control. Second, these putatively abnormal NMDAR-neuromodulator interactions (NNI) were proposed to cause a central computational impairment in schizophrenia: abnormal hierarchical Bayesian inference in the cortex (Stephan et al. , 2006). This proposal was inspired by the notion that the brain constructs a hierarchical and probabilistic model of the world in order to infer the environmental causes of its sensory inputs (predictive coding), and by the increasingly discernible importance of NMDAR-neuromodulator interactions for implementing hierarchical Bayesian inference in the brain (Fig. 1). Under generic conditions, belief updates in Bayesian inference are driven by prediction errors (the difference between actual and predicted inputs) but, critically, weighted by how uncertain or precise both predictions and sensory inputs are. While …