Deficits In Audiovisual Integration Research Articles

Stable organization of the early lexical-semantic network in 18- and 24-month-old preterm and full-term infants: an eye-tracker study.

An organized mental lexicon determines new information acquisition by orienting attention during language processing. Adult-like lexical-semantic knowledge organization has already been demonstrated in 24-month-olds. However, the outcomes of earlier studies have been contradictory in terms of the organizational capacities of 18-month-olds, thus our aim was to examine lexical-semantic organization in this younger age group. In prematurely born infants, audiovisual integration deficits have been found alongside disruptions in language perception. By including late preterm infants with corrected ages in our study, we aimed to test whether maturational differences influence lexical-semantic organization when vocabulary is growing rapidly. We tested 47 late preterm and full-term 18- and 24-month-old infants by means of an infant-adapted target-absent task using a slightly modified version of the original visual world paradigm for eye tracker. We found a longer fixation duration for the lexical and semantic distractors compared to the neutral pictures. Neither language proficiency nor age affected the looking time results. We found a dissociation by age between taxonomic and associative semantic relations. Maturational differences were detectable in the initial processing of taxonomic relations, as processing in the preterm group was slightly delayed and qualitatively different in the first half of the looking time. The size and composition of the expressive vocabulary differed only by age. In general, our study demonstrated a stable lexical-semantic organization between 18 and 24 months of age, regardless of maturational differences.

Individuals With Mild Cognitive Impairment and Alzheimer's Disease Benefit From Audiovisual Speech Cues and Supportive Sentence Context

Purpose Speech perception in noise becomes difficult with age but can be facilitated by audiovisual (AV) speech cues and sentence context in healthy older adults. However, individuals with Alzheimer's disease (AD) may present with deficits in AV integration, potentially limiting the extent to which they can benefit from AV cues. This study investigated the benefit of these cues in individuals with mild cognitive impairment (MCI), individuals with AD, and healthy older adult controls. Method This study compared auditory-only and AV speech perception of sentences presented in noise. These sentences had one of two levels of context: high (e.g., "Stir your coffee with a spoon") and low (e.g., "Bob didn't think about the spoon"). Fourteen older controls (M age = 72.71 years, SD = 9.39), 13 individuals with MCI (M age = 79.92 years, SD = 5.52), and nine individuals with probable Alzheimer's-type dementia (M age = 79.38 years, SD = 3.40) completed the speech perception task and were asked to repeat the terminal word of each sentence. Results All three groups benefited (i.e., identified more terminal words) from AV and sentence context. Individuals with MCI showed a smaller AV benefit compared to controls in low-context conditions, suggesting difficulties with AV integration. Individuals with AD showed a smaller benefit in high-context conditions compared to controls, indicating difficulties with AV integration and context use in AD. Conclusions Individuals with MCI and individuals with AD do benefit from AV speech and semantic context during speech perception in noise (albeit to a lower extent than healthy older adults). This suggests that engaging in face-to-face communication and providing ample context will likely foster more effective communication between patients and caregivers, professionals, and loved ones.

An audiovisual integration deficit underlies reading failure in nontransparent writing systems: An fMRI study of Chinese children with dyslexia

Establishing the associations between speech sounds and visual scripts is an essential step for reading acquisition. Audiovisual integration deficits have been proposed to be a cause of developmental dyslexia, and this has been evidenced by behavioral and neural findings from alphabetic languages with relatively transparent orthographies. Here, using functional magnetic resonance imaging (fMRI) in a lexical decision task, this study examined the neural correlates of audiovisual integration deficits in Chinese children with dyslexia and age-matched controls. The results indicated that the children with dyslexia showed a reduced audiovisual congruency effect in the left angular gyrus, left superior temporal gyrus and left middle and superior frontal gyrus. Moreover, dyslexics showed reduced functional connectivity for the congruency effect between the left angular gyrus and both the left lingual gyrus and left cerebellum. The regional activation and functional connectivity abnormalities might underlie the ill-developed correspondence between orthographic and phonological information in Chinese dyslexia. This study, for the first time, illustrates the neural mechanisms of the audiovisual integration deficit in dyslexia in a nontransparent logographic writing system, extending our understanding of the neural basis of dyslexia.

Dysregulated left inferior parietal activity in schizophrenia and depression: functional connectivity and characterization

The inferior parietal cortex (IPC) is a heterogeneous region that is known to be involved in a multitude of diverse different tasks and processes, though its contribution to these often-complex functions is yet poorly understood. In a previous study we demonstrated that patients with depression failed to deactivate the left IPC during processing of congruent audiovisual information. We now found the same dysregulation (same region and condition) in schizophrenia. By using task-independent (resting state) and task-dependent meta-analytic connectivity modeling (MACM) analyses we aimed at characterizing this particular region with regard to its connectivity and function. Across both approaches, results revealed functional connectivity of the left inferior parietal seed region with bilateral IPC, precuneus and posterior cingulate cortex (PrC/PCC), medial orbitofrontal cortex (mOFC), left middle frontal (MFG) as well as inferior frontal (IFG) gyrus. Network-level functional characterization further revealed that on the one hand, all interconnected regions are part of a network involved in memory processes. On the other hand, sub-networks are formed when emotion, language, social cognition and reasoning processes are required. Thus, the IPC-region that is dysregulated in both depression and schizophrenia is functionally connected to a network of regions which, depending on task demands may form sub-networks. These results therefore indicate that dysregulation of left IPC in depression and schizophrenia might not only be connected to deficits in audiovisual integration, but is possibly also associated to impaired memory and deficits in emotion processing in these patient groups.

An acquired deficit of intermodal temporal processing for audiovisual speech: A case study

Integrating sensory information across modalities is necessary for a cohesive experience of the world; disrupting the ability to bind the multisensory stimuli arising from an event leads to a disjointed and confusing percept. We previously reported (Hamilton et al., 2006) a patient, AWF, who suffered an acute neural incident after which he displayed a distinct inability to integrate auditory and visual speech information. While our prior experiments involving AWF suggested that he had a deficit of audiovisual speech processing, they did not explore the hypothesis that his deficits in audiovisual integration are restricted to speech. In order to test this notion, we conducted a series of experiments aimed at testing AWF’s ability to integrate cross-modal information from both speech and non-speech events. AWF was tasked with making temporal order judgments (TOJs) for videos of object noises (such as hands clapping) or speech, wherein the onsets of auditory and visual information were manipulated. Results from the experiments show that while AWF performed worse than controls in his ability to accurately judge even the most salient onset differences for speech videos, he did not differ significantly from controls in his ability to make TOJs for the object videos. These results illustrate the possibility of disruption of intermodal binding for audiovisual speech events with spared binding for real-world, non-speech events.

Audiovisual integration of speech is disturbed in schizophrenia: An fMRI study

Speech perception is an essential part of social interaction. Visual information (lip movements, facial expression) may supplement auditory information in particular under inadvertent listening situations. Schizophrenia patients have been shown to have a deficit in integrating articulatory motions with the auditory speech input. The goal of this study was to investigate the neural basis of this deficit in audiovisual speech processing in schizophrenia patients by using fMRI. Disyllabic nouns were presented in congruent (audio matches visual information) and incongruent conditions in a slow event related fMRI design. Schizophrenia patients ( n = 15) were compared to age and gender matched control participants. The statistical examination was conducted by analysis of variance with main factors: audiovisual congruency and group membership. The patients' brain activity differed from the control group as evidenced by congruency by group interaction effects. The pertinent brain sites were located predominantly in the right hemisphere and comprised the pars opercularis, middle frontal sulcus, and superior temporal gyrus. In addition, we observed interactions bilaterally in the fusiform gyrus and the nucleus accumbens. We suggest that schizophrenia patients' deficits in audiovisual integration during speech perception are due to a dysfunction of the speech motor system in the right hemisphere. Furthermore the results can be also seen as a reflection of reduced lateralization of language functions to the left hemisphere in schizophrenia.

Multisensory integration of emotional faces and voices in schizophrenics

In their natural environment, organisms receive information through multiple sensory channels and these inputs from different sensory systems are routinely combined into integrated percepts. Previously, we reported that in a population of schizophrenics, deficits in audiovisual integration were observed for complex stimuli (auditory and visual syllables), but not for more simple ones (beeps and light flashes). Here, we investigated multisensory integration of emotional information in a group of schizophrenic patients. In Experiment 1, we found a reduced effect of an emotional voice on the categorization of a facial expression. In Experiment 2, the reverse test situation was presented, and, here, we observed an exaggerated effect of a face expression on the categorization of an emotional voice. Results are discussed in the light of current models of multisensory integration and their relevance for schizophrenia.