Coherent Perception Research Articles

The Power of Trial History: How Previous Trial Shapes Audiovisual Integration

Abstract Combining information from visual and auditory modalities to form a unified and coherent perception is known as audiovisual integration. Audiovisual integration is affected by many factors. However, it remains unclear whether the trial history can influence audiovisual integration. We used a target–target paradigm to investigate how the target modality and spatial location of the previous trial affect audiovisual integration under conditions of divided-modalities attention (Experiment 1) and modality-specific selective attention (Experiment 2). In Experiment 1, we found that audiovisual integration was enhanced in the repeat locations compared with switch locations. Audiovisual integration was the largest following the auditory targets compared to following the visual and audiovisual targets. In Experiment 2, where participants were asked to focus only on visual, we found that the audiovisual integration effect was larger in the repeat location trials than switch location trials only when the audiovisual target was presented in the previous trial. The present results provide the first evidence that trial history can have an effect on audiovisual integration. The mechanisms of trial history modulating audiovisual integration are discussed. Future examining of audiovisual integration should carefully manipulate experimental conditions based on the effects of trial history.

Inverse relation between motion perception and postural responses induced by motion of a touched object

Self vs. external attribution of motions based on vestibular cues is suggested to underlie our coherent perception of object motion and self-motion. However, it remains unclear whether such attribution also underlies sensorimotor responses. Here, we examined this issue in the context of touch. We asked participants to lightly touch a moving object with their thumb while standing still on an unstable surface. We measured both the accuracy of judging the object motion direction and the postural response. If the attribution underlies both object-motion perception and posture control, sensitivity of posture to object motion should decrease with motion speed since high speed motion is unlikely to reflect self-motion. Furthermore, when motion perception is erroneous, there should be a corresponding increase in postural responses. Our results are consistent with these predictions and suggest that self-external attribution of somatosensory motion underlies both object motion perception and postural responses.

Understanding the role of illness perceptions in the relationship between cognitive and emotional difficulties after stroke

ABSTRACT Emotional difficulties are common after stroke and up to one third of stroke-survivors develop post-stroke depression. Psychological distress in this population remains poorly understood, despite high prevalence and secondary implications. One established predictor of depressive symptoms after stroke is cognitive impairment, however, the mechanism underlying this relationship is unclear. This research investigated the potential role of stroke-related illness appraisals as a mediating factor to this known association. Seventy-seven participants, aged 45–94, were consecutively recruited from inpatient stroke units in Oxfordshire over 15-months and completed assessments of mood, cognition and illness appraisals, which were analyzed cross-sectionally. As expected, cognitive impairment significantly predicted depressive symptoms. Importantly, this relationship was shown to be mediated by perceptions of threat and control. Higher levels of cognitive impairment were significantly associated with lower perceived control and higher perceived threat, which partially explained the relationship between cognitive impairment and depressive symptoms. Perceptions of illness coherence were predictive of depressive symptoms but not associated with degree of cognitive impairment. This research has implications for the management of cognitive impairment in the early stages after stroke and suggests that illness appraisals may be an important intervention target for reducing depressive symptoms in patients with post-stroke cognitive impairments.

Dissociating prosodic from syntactic delta activity during natural speech comprehension

Decoding human speech requires the brain to segment the incoming acoustic signal into meaningful linguistic units, ranging from syllables and words to phrases. Integrating these linguistic constituents into a coherent percept sets the root of compositional meaning and hence understanding. One important cue for segmentation in natural speech is prosodic cues, such as pauses, but their interplay with higher-level linguistic processing is still unknown. Here, we dissociate the neural tracking of prosodic pauses from the segmentation of multi-word chunks using magnetoencephalography (MEG). We find that manipulating the regularity of pauses disrupts slow speech-brain tracking bilaterally in auditory areas (below 2Hz) and in turn increases left-lateralized coherence of higher-frequency auditory activity at speech onsets (around 25-45Hz). Critically, we also find that multi-word chunks-defined as short, coherent bundles of inter-word dependencies-are processed through the rhythmic fluctuations of low-frequency activity (below 2Hz) bilaterally and independently of prosodic cues. Importantly, low-frequency alignment at chunk onsets increases the accuracy of an encoding model in bilateral auditory and frontal areas while controlling for the effect of acoustics. Our findings provide novel insights into the neural basis of speech perception, demonstrating that both acoustic features (prosodic cues) and abstract linguistic processing at the multi-word timescale are underpinned independently by low-frequency electrophysiological brain activity in the delta frequency range.

Top-down biological motion perception does not differ between adults scoring high versus low on autism traits

The perception of biological motion is an important social cognitive ability. Models of biological motion perception recognize two processes that contribute to the perception of biological motion: a bottom-up process that binds optic-flow patterns into a coherent percept of biological motion and a top-down process that binds sequences of body-posture ‘snapshots’ over time into a fluent percept of biological motion. The vast majority of studies on autism and biological motion perception have used point-light figure stimuli, which elicit biological motion perception predominantly via bottom-up processes. Here, we investigated whether autism is associated with deviances in the top-down processing of biological motion. For this, we tested a sample of adults scoring low vs high on autism traits on a recently validated EEG paradigm in which apparent biological motion is combined with frequency tagging (Cracco et al., 2022) to dissociate between two percepts: 1) the representation of individual body postures, and 2) their temporal integration into movements. In contrast to our hypothesis, we found no evidence for a diminished temporal body posture integration in the high-scoring group. We did, however, find a group difference that suggests that adults scoring high on autism traits have a visual processing style that focuses more on a single percept (i.e. either body postures or movements, contingent on saliency) compared to adults scoring low on autism traits who instead seemed to represent the two percepts included in the paradigm in a more balanced manner. Although unexpected, this finding aligns well with the autism literature on perceptual stability.

Variations in the coherence and engagement in students’ experience of blended learning

We report a study which examines coherence of engagement of 344 first year engineering students’ blended learning experience. Using self-report and observational data sources, we demonstrate that student perceptions of the blended learning environment, academic learning outcomes, and engagement with the online learning activities are logically related at the variable level as shown by correlation analyses; and at the level of student groupings of similar learning experience and behaviors, as revealed by cluster, ANOVA, and 2 x 2 contingency analyses. Using self-report data, we found that when students perceived the learning activities in the f2f and online environments were integrated, they were more likely to be engaged with the online learning and to perform relatively higher on the assessment tasks than students who perceived disintegration between f2f and online learning. Using the observational data, students who were more engaged with the online learning tended to perceive that the online learning was well integrated with the f2f learning, that the online contributions were valuable for the whole learning experience, and achieved relatively higher than less engaged students. A 2 x 2 contingency table revealed a logical relationship between the groupings of students based on the selfreport and observational data: moderate and positive association was found between students with coherent perceptions and more engagement; and between students with fragmented perceptions and less engagement. The use of multiple data sources and methods enabled triangulation, strengthened analysis power, and offered a more comprehensive picture of students’ blended learning experience.

“I Can’t Control MS Entirely, But at Least I Feel a Sense of Control”: A Qualitative Analysis of Individuals’ Perceptions of Control, Illness Coherence, and Self-Efficacy Following a Web-Based Lifestyle Program for Multiple Sclerosis (Preprint)

“I Can’t Control MS Entirely, But at Least I Feel a Sense of Control”: A Qualitative Analysis of Individuals’ Perceptions of Control, Illness Coherence, and Self-Efficacy Following a Web-Based Lifestyle Program for Multiple Sclerosis (Preprint)

Aligning Human and Computational Coherence Evaluations

Abstract Automated coherence metrics constitute an efficient and popular way to evaluate topic models. Previous works present a mixed picture of their presumed correlation with human judgment. This work proposes a novel sampling approach to mine topic representations at a large-scale while seeking to mitigate bias from sampling, enabling the investigation of widely-used automated coherence metrics via large corpora. Additionally, this article proposes a novel user study design, an amalgamation of different proxy tasks, to derive a finer insight into the human decision-making processes. This design subsumes the purpose of simple rating and outlier-detection user studies. Similar to the sampling approach, the user study conducted is very extensive, comprising forty study participants split into eight different study groups tasked with evaluating their respective set of one hundred topic representations. Usually, when substantiating the use of these metrics, human responses are treated as the golden standard. This article further investigates the reliability of human judgment by flipping the comparison and conducting a novel extended analysis of human response at the group and individual level against a generic corpus. The investigation results show a moderate to good correlation between these metrics and human judgment, especially for generic corpora, and derive further insights into the human perception of coherence. Analysing inter-metric correlations across corpora shows moderate to good correlation amongst these metrics. As these metrics depend on corpus statistics, this article further investigates the topical differences between corpora revealing nuances in applications of these metrics.

The role of pattern coherence in interocular grouping during binocular rivalry: Insights from individual differences

Interocular grouping during binocular rivalry occurs when two images presented to each eye combine into a coherent pattern. The experience of interocular grouping is thought to be influenced by both eye-of-origin, which involves excitatory lateral connections among monocular neurons, and pattern coherence, which results from top-down intervention from higher visual areas. However, it remains unclear which factor plays a more significant role in the interocularly-grouped percepts during binocular rivalry. The current study employed an individual difference approach to investigate whether grouping dynamics are mainly determined by eye-of-origin or pattern coherence. We found that participants who perceived interocularly-driven coherent percepts for a longer duration also tended to experience longer periods of monocularly-driven coherent percepts. In contrast, participants who experienced non-coherent piecemeal percepts for an extended duration in conventional rivalry also had longer duration of non-coherent percepts in the interocular coherence setting. This individual differences in experiencing interocular grouping suggest that pattern coherence exerts a stronger influence on grouping dynamics during binocular rivalry compared to eye-of-origin factors.

Abnormal multisensory temporal discrimination in Parkinson’s disease

Cognitive deficits are prevalent in Parkinson’s disease (PD), ranging from mild deficits in perception and executive function to severe dementia. Multisensory integration (MSI), the ability to pool information from different sensory modalities to form a combined, coherent perception of the environment, is known to be impaired in PD. This study investigated the disruption of audiovisual MSI in PD patients by evaluating temporal discrimination ability between auditory and visual stimuli with different stimulus onset asynchronies (SOAs). The experiment was conducted with Fifteen PD patients and fifteen age-matched healthy controls where participants were requested to report whether the audiovisual stimuli pairs were temporal simultaneous. The temporal binding window (TBW), the time during which sensory modalities are perceived as synchronous, was adapted as the comparison index between PD patients and healthy individuals. Our results showed that PD patients had a significantly wider TBW than healthy controls, indicating abnormal audiovisual temporal discrimination. Furthermore, PD patients had more difficulty in discriminating temporal asynchrony in visual-first, but not in auditory-first stimuli, compared to healthy controls. In contrast, no significant difference was observed for auditory-first stimuli. PD patients also had shorter reaction times than healthy controls regardless of stimulus priority. Together, our findings point to abnormal audiovisual temporal discrimination, a major component of MSI irregularity, in PD patients. These results have important implications for future models of MSI experiments and models that aim to uncover the underlying mechanism of MSI in patients afflicted with PD.

Trajectories of illness perceptions in paediatric cancer patients and their parents and associations with health-related quality of life: Results of a prospective-longitudinal study.

In paediatric oncology, little is known about trajectories of illness perceptions and their longitudinal associations with health-related quality of life (HRQoL). Therefore, the aim of this study was to investigate changes in illness perceptions in children and parents over a one-year-period and to investigate predictive value of child's and parent's illness perceptions during acute treatment for child's HRQoL 1year later. N=65 child-parent-dyads participated in a longitudinal study (retention rate: 80.2%). Children were 4-18years of age and underwent acute cancer treatment at baseline. Children and parents reported on their own illness perceptions (Illness-Perception-Questionnaire-Revised), as well as on the child's HRQoL (KINDL-R) at baseline and one-year-follow-up. Paired-samples t-tests were calculated to investigate changes over time. A hierarchical multiple regression analysis was performed to investigate predictive value of child's and parent's illness perceptions for child's HRQoL. Child's HRQoL t(63)=-6.73, p<0.001, their perceptions of coherence (i.e. understanding; t(54)=-2.36, p=0.022) and consequences of their illness (t(54)=2.86, p=0.006), and parent's perception of cyclical trajectory (t(61)=2.06, p=0.044) improved from baseline to 1-year-follow-up. All other illness perceptions remained stable. Exploratory post-hoc analyses showed differences in the pattern of change in age-, gender-, and diagnosis-specific subgroups. After controlling for baseline levels of HRQoL, child's perceptions of symptoms and consequences were independent predictors of their HRQoL 1year later (R2=0.396, F(2,52)=10.782, p<0.001), whereas no parent's illness perceptions added predictive value. In paediatrics, child's and parent's illness perceptions should be assessed. Our findings highlight the importance of illness perceptions as potential modifiable variables in interventions to improve child's HRQoL.

Where visual fields coming from? ‐ A short introduction from neurophysiology point of view

Visual fields refer to the area of vision that can be seen by each eye when it is fixed in a certain position. The visual fields are created by the combination of the images received by each eye, which are then processed by the brain to create a single, unified image. The visual fields are divided into four quadrants: superior, inferior, nasal, and temporal. Each eye sees a slightly different portion of the visual field, and the brain combines these two images to create a three‐dimensional perception of the world.The visual fields are outcome of the interaction of the ocular structures, optic nerve, and brain centres. The eye captures images of the visual field through the cornea and lens, which focus the light onto the retina. The retina contains specialized photoreceptors, which convert the light into electrical signals that are transmitted through the optic nerve to the brain. The images are first integrated and processed within lateral geniculate nuclei of the thalamus. In this important structure visual stimuli are first filtered, to eliminate irrelevant or unnecessary information, allowing only important visual signals to pass on to the visual cortex, then integrated from both eyes, allowing for depth perception and 3D vision, processed, based on features such as colour, contrast, and movement, which are then relayed to the visual cortex and synchronized with other sensory information in the brain, allowing for a coherent perception of the environment. The visual cortex in the brain then processes these integrated signals to create the perception of the visual field. This complex process allows us to see and interpret the world around us, including objects, people, and scenery, and to respond to them in a meaningful way.

Study Program Leaders’ Perceptions of Coherence and Strategies for Creating Coherent Teacher Education Programs

Scholars and policymakers have consistently argued for the importance of coherence in teacher education (TE). Despite this attention to coherence, challenges of fragmentation and disconnect remain, and little research exists on how study program leaders (SPLs) in TE work to achieve coherence. This article explores how SPLs in two selected TE institutions in Norway perceive coherence and what strategies they use to create coherent TE programs. The two case studies use an ethnographic approach and report on repeated qualitative interviews with seven SPLs, supported by short-term observations over a period of one academic year. The analysis contributes to the small but emergent literature on how SPLs conceptualize coherence and identifies six strategies that SPLs use to navigate persistent barriers within a diverse and autonomous faculty in their ongoing efforts to strengthen coherence in TE programs. Implications for study program leadership in TE are discussed.

Alpha-frequency feedback to early visual cortex orchestrates coherent naturalistic vision.

During naturalistic vision, the brain generates coherent percepts by integrating sensory inputs scattered across the visual field. Here, we asked whether this integration process is mediated by rhythmic cortical feedback. In electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) experiments, we experimentally manipulated integrative processing by changing the spatiotemporal coherence of naturalistic videos presented across visual hemifields. Our EEG data revealed that information about incoherent videos is coded in feedforward-related gamma activity while information about coherent videos is coded in feedback-related alpha activity, indicating that integration is indeed mediated by rhythmic activity. Our fMRI data identified scene-selective cortex and human middle temporal complex (hMT) as likely sources of this feedback. Analytically combining our EEG and fMRI data further revealed that feedback-related representations in the alpha band shape the earliest stages of visual processing in cortex. Together, our findings indicate that the construction of coherent visual experiences relies on cortical feedback rhythms that fully traverse the visual hierarchy.

Cross-Modal Plasticity during Self-Motion Perception.

To maintain stable and coherent perception in an ever-changing environment, the brain needs to continuously and dynamically calibrate information from multiple sensory sources, using sensory and non-sensory information in a flexible manner. Here, we review how the vestibular and visual signals are recalibrated during self-motion perception. We illustrate two different types of recalibration: one long-term cross-modal (visual-vestibular) recalibration concerning how multisensory cues recalibrate over time in response to a constant cue discrepancy, and one rapid-term cross-modal (visual-vestibular) recalibration concerning how recent prior stimuli and choices differentially affect subsequent self-motion decisions. In addition, we highlight the neural substrates of long-term visual-vestibular recalibration, with profound differences observed in neuronal recalibration across multisensory cortical areas. We suggest that multisensory recalibration is a complex process in the brain, is modulated by many factors, and requires the coordination of many distinct cortical areas. We hope this review will shed some light on research into the neural circuits of visual-vestibular recalibration and help develop a more generalized theory for cross-modal plasticity.

Atypical development of causal inference in autism inferred through a neurocomputational model.

In everyday life, the brain processes a multitude of stimuli from the surrounding environment, requiring the integration of information from different sensory modalities to form a coherent perception. This process, known as multisensory integration, enhances the brain's response to redundant congruent sensory cues. However, it is equally important for the brain to segregate sensory inputs from distinct events, to interact with and correctly perceive the multisensory environment. This problem the brain must face, known as the causal inference problem, is strictly related to multisensory integration. It is widely recognized that the ability to integrate information from different senses emerges during the developmental period, as a function of our experience with multisensory stimuli. Consequently, multisensory integrative abilities are altered in individuals who have atypical experiences with cross-modal cues, such as those on the autistic spectrum. However, no research has been conducted on the developmental trajectories of causal inference and its relationship with experience thus far. Here, we used a neuro-computational model to simulate and investigate the development of causal inference in both typically developing children and those in the autistic spectrum. Our results indicate that higher exposure to cross-modal cues accelerates the acquisition of causal inference abilities, and a minimum level of experience with multisensory stimuli is required to develop fully mature behavior. We then simulated the altered developmental trajectory of causal inference in individuals with autism by assuming reduced multisensory experience during training. The results suggest that causal inference reaches complete maturity much later in these individuals compared to neurotypical individuals. Furthermore, we discuss the underlying neural mechanisms and network architecture involved in these processes, highlighting that the development of causal inference follows the evolution of the mechanisms subserving multisensory integration. Overall, this study provides a computational framework, unifying causal inference and multisensory integration, which allows us to suggest neural mechanisms and provide testable predictions about the development of such abilities in typically developed and autistic children.

Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement

Texture, a fundamental object attribute, is perceived through multisensory information including touch and auditory cues. Coherent perceptions may rely on shared texture representations across different senses in the brain. To test this hypothesis, we delivered haptic textures coupled with a sound synthesizer to generate real-time textural sounds. Participants completed roughness estimation tasks with haptic, auditory, or bimodal cues in an MRI scanner. Somatosensory, auditory, and visual cortices were all activated during haptic and auditory exploration, challenging the traditional view that primary sensory cortices are sense-specific. Furthermore, audio-tactile integration was found in secondary somatosensory (S2) and primary auditory cortices. Multivariate analyses revealed shared spatial activity patterns in primary motor and somatosensory cortices, for discriminating texture across both modalities. This study indicates that primary areas and S2 have a versatile representation of multisensory textures, which has significant implications for how the brain processes multisensory cues to interact more efficiently with our environment.

Divergences in color perception between deep neural networks and humans

Deep neural networks (DNNs) are increasingly proposed as models of human vision, bolstered by their impressive performance on image classification and object recognition tasks. Yet, the extent to which DNNs capture fundamental aspects of human vision such as color perception remains unclear. Here, we develop novel experiments for evaluating the perceptual coherence of color embeddings in DNNs, and we assess how well these algorithms predict human color similarity judgments collected via an online survey. We find that state-of-the-art DNN architectures — including convolutional neural networks and vision transformers — provide color similarity judgments that strikingly diverge from human color judgments of (i) images with controlled color properties, (ii) images generated from online searches, and (iii) real-world images from the canonical CIFAR-10 dataset. We compare DNN performance against an interpretable and cognitively plausible model of color perception based on wavelet decomposition, inspired by foundational theories in computational neuroscience. While one deep learning model — a convolutional DNN trained on a style transfer task — captures some aspects of human color perception, our wavelet algorithm provides more coherent color embeddings that better predict human color judgments compared to all DNNs we examine. These results hold when altering the high-level visual task used to train similar DNN architectures (e.g., image classification versus image segmentation), as well as when examining the color embeddings of different layers in a given DNN architecture. These findings break new ground in the effort to analyze the perceptual representations of machine learning algorithms and to improve their ability to serve as cognitively plausible models of human vision. Implications for machine learning, human perception, and embodied cognition are discussed.

Impact of Illness Perceptions on Poststroke Activity Engagement and the Moderating Role of Gender.

Beyond existing knowledge of demographic and performance skill factors, establishing relationships between poststroke activity engagement and illness perceptions sets the stage for the development of more effective intervention strategies. To describe the illness perceptions of community-dwelling people with stroke in the first 2 yr poststroke; specifically, to examine whether illness perceptions are associated with activity engagement and to explore the moderating role of gender in these relationships. Cross-sectional study. Participants were recruited from eight rehabilitation settings in Beijing, China. 202 community dwellers with stroke. Activity engagement and illness perceptions were measured with the Mandarin version of the Assessment of Life Habits and the Chinese version of the Stroke-Specific Illness Perceptions Questionnaire-Revised, respectively. Participants' demographic information, cognitive status, and motor function were also collected. Stronger perceptions of consequences and controllability were related to better performance in activity engagement at the personal level, and stronger perceptions of illness coherence were related to better performance in activity engagement at the societal level. In addition, gender differences in the relationship between illness perceptions and activity engagement were described. How people with stroke perceived their conditions dictated their levels of activity engagement in their community of residence. The findings suggest that understanding clients' illness perceptions may assist practitioners in developing comprehensive, targeted interventions to improve activity engagement and maximize recovery after stroke. Future studies are needed to explore the gender effect of illness perceptions on activity engagement in people with stroke. What This Article Adds: This study identified the relationships between illness perceptions and level of activity engagement in real-life environments in people with stroke. In addition to motor and cognitive interventions, providing opportunities for clients to gain a better understanding of stroke would facilitate their activity engagement in their real-life environment.

Duplex perception reveals brainstem auditory representations are modulated by listeners' ongoing percept for speech.

So-called duplex speech stimuli with perceptually ambiguous spectral cues to one ear and isolated low- versus high-frequency third formant "chirp" to the opposite ear yield a coherent percept supporting their phonetic categorization. Critically, such dichotic sounds are only perceived categorically upon binaural integration. Here, we used frequency-following responses (FFRs), scalp-recorded potentials reflecting phase-locked subcortical activity, to investigate brainstem responses to fused speech percepts and to determine whether FFRs reflect binaurally integrated category-level representations. We recorded FFRs to diotic and dichotic stop-consonants (/da/, /ga/) that either did or did not require binaural fusion to properly label along with perceptually ambiguous sounds without clear phonetic identity. Behaviorally, listeners showed clear categorization of dichotic speech tokens confirming they were heard with a fused, phonetic percept. Neurally, we found FFRs were stronger for categorically perceived speech relative to category-ambiguous tokens but also differentiated phonetic categories for both diotically and dichotically presented speech sounds. Correlations between neural and behavioral data further showed FFR latency predicted the degree to which listeners labeled tokens as "da" versus "ga." The presence of binaurally integrated, category-level information in FFRs suggests human brainstem processing reflects a surprisingly abstract level of the speech code typically circumscribed to much later cortical processing.