Prediction of Perception

Predictive processing models have recently flourished in neuroscience1–9. Feedforward and feedback modulation are at the heart of these hierarchical predictive processing models. Previous experimental studies using fMRI, EEG/MEG, and LFP1,10,11 could not reliably resolve feedback modulation from local computations and feedforward outputs. Here, using open-science9, multi-species, multi-area, high-density12, laminar neurophysiology13, we empirically test whether predictive processing is a key component shaping sensation. To isolate sensory information processing and eliminate motor/reward confounders1,10,11, we use a no-report task. Our task leveraged so-called global oddballs (GO) as unpredictable, deviant stimuli that circumvent low-level adaptation. We examined their responses relative to local oddballs (LO) that we habituated into highly predictable priors. Four surprising findings in this dataset challenge many existing predictive processing models. First, passively evoked GO responses were exclusive to higher-order, more cognitive areas rather than early-to-mid sensory cortex. Second, interneuron-targeted optogenetics revealed no evidence for a subtractive mechanism in both primates and mice. Third, highly predictable LO responses dominated in over 50% of all neurons, including in higher-order cortex which should have anticipated them, indicating limited evidence for predictive suppression. Lastly, prediction errors followed a feedback, rather than a feedforward signature. These results reveal circuit dynamics that govern the shaping of sensory processing by prediction, which will motivate new, neurally-constrained predictive processing models.

A critical question in speech perception is the relative independence of perceptual and semantic processing. Answering this requires addressing two issues. First, does perceptual processing complete before semantic processing (discrete stages vs. continuous cascades)? Second, do semantic expectations affect perceptual processing (feedback)? These questions are difficult to address as there are few measures of early perceptual processing for speech. We extend a recent electroencephalography (EEG) paradigm which has shown sensitivity to pre-categorical encoding of Voice-Onset Time (VOT; Toscano et al., 2010). By measuring the timecourse over which perceptual and semantic factors affect the neural signal, we quantify how these processes interact. Participants (N = 31) heard sentences (Good dogs also sometimes—) which biased them to expect a target word (bark rather than park). We manipulated VOT of the target word and coarticulation leading to it. A component-independent analysis determined when each cue affects the continuous EEG signal every 2 msec. This revealed an early window (125–225 msec) sensitive exclusively to bottom-up information, a late window (400–575 msec) sensitive to semantic information, and a critical intermediate window (225–350 msec) during which VOT and coarticulation are processed simultaneously with semantic expectations. This suggests continuous cascades and early interactions between perceptual and semantic processes.

Long-term music training can positively impact speech processing. A recent framework developed to explain such cross-domain plasticity posits that music training-related advantages in speech processing are due to shared cognitive and perceptual processes between music and speech. Although perceptual and cognitive processing advantages due to music training have been independently demonstrated, to date no study has examined perceptual and cognitive processing within the context of a single task. The present study examines the impact of long-term music training on speech learning from a rigorous, computational perspective derived from signal detection theory. Our computational models provide independent estimates of cognitive and perceptual processing in native English-speaking musicians (n = 15, mean age = 25 years) and non-musicians (n = 15, mean age = 23 years) learning to categorize non-native lexical pitch patterns (Mandarin tones). Musicians outperformed non-musicians in this task. Model-based analyses suggested that musicians shifted from simple unidimensional decision strategies to more optimal multidimensional (MD) decision strategies sooner than non-musicians. In addition, musicians used optimal decisional strategies more often than non-musicians. However, musicians and non-musicians who used MD strategies showed no difference in performance. We estimated parameters that quantify the magnitude of perceptual variability along two dimensions that are critical for tone categorization: pitch height and pitch direction. Both musicians and non-musicians showed a decrease in perceptual variability along the pitch height dimension, but only musicians showed a significant reduction in perceptual variability along the pitch direction dimension. Notably, these advantages persisted during a generalization phase, when no feedback was provided. These results provide an insight into the mechanisms underlying the musician advantage observed in non-native speech learning.

Hearing How Smooth It Looks: Selective Attention and Crossmodal Perception in the Arts

Author response: A unified neural account of contextual and individual differences in altruism

Perceptual processing demands influence voluntary task choice

The brain represents the state of the world around us based upon a stream of ambiguous sensory data. This requires the integration of different sources of information, including information received from different sense modalities, and information drawn from past experience. Bayesian theories of perception provide an approach to characterising how systematic differences in the probabilistic integration of information in the brain may underlie the differences that occur between individuals in their perceptual experience. The present thesis explores how perceptual, motor and social aspects of autism spectrum disorder (ASD) may emerge from variation in the neurocognitive processes described by Bayesian theories. A focus is on the predictive processing model of brain function, which links Bayesian theories of perception to neural mechanisms. The thesis includes empirical studies that examine perceptual, sensorimotor and cognitive aspects of ASD. ASD and nonclinical autistic traits are first examined in the context of body perception; concerning, specifically, how the brain represents the state of the limbs based on visual, tactile and proprioceptive information, and how this sensory information is integrated with prior expectations regarding the body. These processes are investigated using a multisensory perceptual illusion: the rubber hand illusion. ASD is associated with a largely typical perceptual experience of this illusion, indicating intact multisensory integration in the context of body representation. A reduced influence of the illusion on motor function, however, supports a difference in ASD in the integration of expectations for limb position (influenced by the illusion) with conflicting sensory (proprioceptive) signals. The integration of incoming information with existing expectations about the environment is also examined in the context of statistical learning. A computerised task is developed to assess the integration of iterative feedback with prior information as participants predict the location of a noisy set of visual markers. The hypothesis tested is that features of ASD are associated with a persistently higher weighting of incoming information in driving inference, at the expense of expectations developed from recent experience; in this more cognitive context, these data provide evidence against this hypothesis, suggesting that a more context-dependent atypicality in information processing in ASD is more likely than that initially hypothesised. The thesis also develops theoretical and philosophically-relevant treatments of how the symptoms of ASD may emerge from differences in brain function characterised within a Bayesian framework. This includes drawing on recent theoretical developments regarding the role of volatility processing and action in maintaining optimal inference on the environment. In addition, models of Bayesian inference in the brain can be extended to the social domain. For instance, the process of representing the mental states of other people (i.e., theory of mind) can be cast in terms of implicit inference on the external causes of sensory signals. In light of these expanded models of predictive processing, the differences in information processing hypothesised to occur in ASD have implications not just for perception, but also for motor behaviours, social cognition and social interaction. The overall view that emerges is that diverse aspects of ASD may be captured in terms of how incoming sensory signals are integrated, in a probabilistic manner, with the brain’s hierarchical and multimodal model of its environment. A promising direction for this field is in developing this idea in the context of action (i.e., active inference) and more recent models of how the brain estimates the optimal weighting of sensory information. This area of research has the potential to provide a nuanced perspective on the neurocognitive basis of ASD and the relationship between sensory mechanisms and autistic behaviours.

Predictive processing models and affective neuroscience

Grounded cognition suggests that conceptual processing shares cognitive resources with perceptual processing. Hence, conceptual processing should be affected by perceptual processing, and vice versa. The current study explored the relationship between conceptual and perceptual processing of size. Within a pair of words, we manipulated the font size of each word, which was either congruent or incongruent with the actual size of the referred object. In Experiment 1a, participants compared object sizes that were referred to by word pairs. Higher accuracy was observed in the congruent condition (e.g., word pairs referring to larger objects in larger font sizes) than in the incongruent condition. This is known as the size-congruency effect. In Experiments 1b and 2, participants compared the font sizes of these word pairs. The size-congruency effect was not observed. In Experiments 3a and 3b, participants compared object and font sizes of word pairs depending on a task cue. Results showed that perceptual processing affected conceptual processing, and vice versa. This suggested that the association between conceptual and perceptual processes may be bidirectional but further modulated by semantic processing. Specifically, conceptual processing might only affect perceptual processing when semantic information is activated. The current study

Representation of the body is a nuclear aspect of self-image. Body representation includes both subjective and intersubjective experiences. The importance of visual body representation in our social life is demonstrated by the time we spend to take care of our physical appearance, including use of plastic surgery, as well as by the severe mental disorders associated to its alteration, including eating disorders (EDs) and body dysmorphic disorders. In spite of these issues, (neuro)psychological research on body representation has so far mainly focused on the body as a motor device, devoted to the perception of and interaction with objects. The complexity of factors involved in the development, maintenance, and plasticity of body image representations requires an integrated approach that facilitates the evaluation and treatment of EDs with novel, evidence-based, and more efficacious protocols. This special issue of European Psychologist aims to advance such an integrative approach, with a collection of papers that gather different perspectives on the phenomenological, cognitive, (neuro)psychological, and cultural aspects of body representation disorders.

An intriguing finding of research on emotional processing is a discrepancy between perception and behavior. Perceptually, a robust finding is that negative stimuli are processed faster and more efficiently than positive stimuli. Behaviorally, a similarly robust finding is that response times are slower for negative than for positive stimuli. We proposed and tested a novel account to explain this still unexplained discrepancy, on the basis of the assumption that negative valence narrows perceptual processes to the benefit of speeded perception, but broadens motor processes at the cost of slowed responding. Participants performed a valence judgment task in which they responded with their left or right hand to negative and positive stimuli that were presented on the left or right, and we measured the activation of relevant/deactivation of irrelevant perceptual and motor processes, as revealed by the lateralization of electroencephalographic brain oscillations. Stimulus-related lateralization of alpha activity (8-12Hz) over perceptual areas was increased for negative stimuli, indicating more efficient perceptual processing. By contrast, response-related lateralization of beta activity (20-25Hz) over motor areas was decreased for negative stimuli, indicating less efficient response activation. Consistent with our predictions, more detailed analyses showed that both lateralization effects were caused by dynamics at the level of inhibiting irrelevant processes. For negative as compared to positive stimuli, the inhibition of irrelevant perceptual processes was increased, but the inhibition of irrelevant motor processes was decreased. These findings indicate that the discrepancy between perception and behavior in emotional processing may stem from asymmetrical effects of emotional valence on the breadth of cortical activations in perceptual and motor networks.

Research on multitasking harkens back to the beginnings of cognitive psychology. The central question has always been how we manage to perform multiple actions at the same time. Here, we highlight the role of specific inputand output-modalities involved in coordinating multiple action demands (i.e., crossmodal action). For a long time, modalityand content-blind models of multitasking have dominated theory, but a variety of recent findings indicate that modalities and content substantially determine performance. Typically, the term ‘‘input modality’’ refers to sensory channels (e.g., visual input is treated differently from auditory input), and the term ‘‘output modality’’ is closely associated with effector systems (e.g., hand vs. foot movements). However, this definition may be too narrow. The term ‘‘input modality’’ sometimes refers to a dimension within a sensory channel (e.g., shape/color in vision). Furthermore, the linkage between output-modalities and effector systems may not be specific enough to illuminate some notorious twilight zones (e.g., to distinguish between hand and wrist movements). As a consequence, we will use ‘‘modality’’ as an umbrella term here to capture various sources of stimulus variability used to differentiate the task-relevant information and sources of motor variability used to differentiate responses. Many of the pioneering studies involved the observation of dual-task performance in two continuous tasks that typically consisted of complex action sequences (e.g., reading and writing, see Solomons & Stein, 1896; Spelke, Hirst, & Neisser, 1976). However, it soon became apparent that tighter experimental control was necessary to pinpoint the specific cognitive mechanisms supporting multitasking. The PRP paradigm: an experimental breakthrough. The development of the psychological refractory period (PRP) paradigm (Telford, 1931; Welford, 1952) provided a methodological breakthrough that allowed researchers to exactly control the flow of information in both tasks. The PRP paradigm involves two elementary tasks with a limited set of clearly defined stimuli and responses. The mechanisms underlying multitasking are studied by systematically manipulating the temporal overlap of the two tasks, which is achieved by varying the delay between the presentations of the stimuli for the two tasks (stimulus onset asynchrony, SOA). The PRP effect refers to the typical finding that reaction times (RTs) for the second task increase with decreasing SOA, an effect that has been replicated in numerous studies with a variety of stimulus and response modalities (see Bertelson, 1966; Pashler, 1994; Smith, 1967). The RSB model: a powerful explanatory concept? The most influential and elegant account of the PRP effect has been the response selection bottleneck (RSB) model (Telford, 1931; Welford, 1952). A starting assumption of the RSB model is that tasks at hand can be divided into three successive cognitive processing steps, namely perceptual processing (i.e., stimulus encoding/categorization), response selection (i.e., deciding which response corresponds to the stimulus according to the task rules), and response execution processes. In a number of experiments, the duration of each of these processing stages was systematically manipulated for each of the two tasks (see Pashler, 1994). As a result, the most convincing hypothesis to accommodate the corresponding findings was the assumption that perceptual processing and response L. Huestegge (&) RWTH Aachen University, Aachen, Germany e-mail: lynn.huestegge@psych.rwth-aachen.de

It has been well established in previous research that predictive information has a large impact on decision-making in a variety of cognitive paradigms. Less is known about how predictive cues influence the sensory, perceptual and affective processes toward a subjective choice. Here, we designed an evaluative decision-making paradigm, with naturalistic food images as stimuli, using predictive cues to measure pupil dilation during information processing toward a subjective rating. In each trial, we used a predictive cue to generate an expectation about the upcoming target image. The color of the cue indicated the predictive validity (either 100% or 50% reliability); the shape of the cue indicated the predicted valence of the target image (either appetitive or aversive). We also varied the length of the delay between the predictive cue and the target image (either 1s or 9s). The participants were asked to rate the target food images on a continuous scale from -10 to 10 using a joystick. The results showed a difference in pupil dilation in response to the predictive cue as a function of the predicted valence, with more constriction following negative cues. The dilation was unaffected by color. There was also a notable difference in pupil dilation during the target image viewing as a function of the actual valence, with more constriction for appetitive images. The apparent contradiction in dilation as a function of item category may be due to the level of arousal, with complementary or opposing functions for valence during expectation versus actual sensory and perceptual processing. Future research is required to establish the exact relationship with arousal in order to uncover the underlying mechanisms of this novel finding.

Implicit Learning and Memory: Psychological and Neural Aspects

Researchers in the field of active perception study how sensory processes coalesce with motor actions to extract information from the world. Such actions intrinsically alter perceptual processing and have intended sensory outcomes, but also lead to incidental sensory consequences, which are side effects of moving the sensory surface to its intended goal. These incidental consequences of actions are generally considered a nuisance to perception that needs to be attenuated or suppressed during movement execution. In this Perspective, we propose instead that incidental sensory consequences of actions shape perceptual processes through action–perception couplings and we review evidence from the domain of active vision. We propose four hallmarks representing the degrees to which actions are an integral part of a perceptual processing architecture. Finally, we outline a research strategy for probing these hallmarks in active perceptual systems and conclude that researchers of perception should embrace the study of action kinematics in pursuit of their questions. Studying action is key for understanding perception because perceptual processes are shaped by movements of the sensory surface. In this Perspective, Rolfs and Schweitzer propose four hallmarks of action–perception coupling and outline a research strategy for the study of action kinematics in perception.