Spatial modulation of audiovisual integration in the sound-induced flash illusion: A hierarchical drift diffusion modeling approach.

Although there is some evidence suggesting that audiovisual integration is inefficient in older adults, and that such inefficiency is associated with age-related functions such as mild cognitive impairment, falls, and balance maintenance, these associations have yet to be demonstrated in a population-representative study of ageing. Based on a sample of 3,955 adults aged over 50 years, we investigated the role of age, cognitive status, and sex on susceptibility to the sound-induced flash illusion (SIFI) as a measure of audiovisual temporal integration, while controlling for a range of covariates. We developed a hierarchical Bayesian, ordinal-regression model to determine which variables predicted audiovisual integration. Higher susceptibility to the SIFI was predicted by older age, female sex (at larger temporal asynchronies), and a lower score on the Montreal Cognitive Assessment (MoCA). Our results confirm, in a population-representative sample, that enhanced audiovisual integration is associated with ageing and extend the association between multisensory integration and mild cognitive impairment to global cognitive status. Importantly, the findings also highlight the role of the sex of the participant as a previously overlooked factor in studying multisensory perception in ageing. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

IntroductionSubjective cognitive decline (SCD) refers to individuals’ perceived decline in memory and/or other cognitive abilities relative to their previous level of performance. Sensory decline is one of the main manifestations of decline in older adults with SCD. The efficient integration of visual and auditory information, known as audiovisual integration, is a crucial perceptual process. This study aims to evaluate audiovisual integration in older adults with SCD.MethodsWe adopted the audiovisual detection task, the Colavita task, and the Sound‐Induced Flash Illusion (SIFI) task to evaluate the audiovisual integration by examining both redundant and illusory effects. Older adults diagnosed with SCD (N = 50, mean age = 67.8 years) and a control group of non‐SCD older adults (N = 51, mean age = 66.5 years) were recruited. All participants took part in the three aforementioned experiments.ResultsThe outcomes showed that a redundant effect occurred in both SCD and non‐SCD older adults, with SCD older adults gaining more benefits in audiovisual detection task. Moreover, an equivalent amount of the visual dominance effect was observed among both SCD and non‐SCD older adults in Colavita task. In addition, older adults with SCD perceived an equal fission illusion but a bigger fusion illusion compared with non‐SCD older adults in SIFI task.ConclusionsOverall, older adults with SCD exhibit increased audiovisual redundant effects and stronger fusion illusion susceptibility compared to non‐SCD older adults. Besides, visual dominance was observed in both groups via the Colavita task, with no significant difference between non‐SCD and SCD older adults. These findings implied that audiovisual integration might offer a potential way for the identification of SCD.

Visual Hallucinations Are Characterized by Impaired Sensory Evidence Accumulation: Insights From Hierarchical Drift Diffusion Modeling in Parkinson’s Disease

TOO MUCH DELIBERATION? CAUTIOUS DECISION-MAKING IN OCD

Integrating information across different senses is a central feature of human perception. Previous research suggests that multisensory integration is shaped by a context-dependent and largely adaptive interplay between stimulus-driven bottom-up and top-down endogenous influences. One critical question concerns the extent to which this interplay is sensitive to the amount of available cognitive resources. In the present study, we investigated the influence of limited cognitive resources on audiovisual integration by measuring high-density electroencephalography (EEG) in healthy participants performing the sound-induced flash illusion (SIFI) and a verbal n-back task (0-back, low load and 2-back, high load) in a dual-task design. In the SIFI, the integration of a flash with two rapid beeps can induce the illusory perception of two flashes. We found that high compared with low load increased illusion susceptibility and modulated neural oscillations underlying illusion-related crossmodal interactions. Illusion perception under high load was associated with reduced early β power (18-26 Hz, ∼70 ms) in auditory and motor areas, presumably reflecting an early mismatch signal and subsequent top-down influences including increased frontal θ power (7-9 Hz, ∼120 ms) in mid-anterior cingulate cortex (ACC) and a later β power suppression (13-22 Hz, ∼350 ms) in prefrontal and auditory cortex. Our study demonstrates that integrative crossmodal interactions underlying the SIFI are sensitive to the amount of available cognitive resources and that multisensory integration engages top-down θ and β oscillations when cognitive resources are scarce.SIGNIFICANCE STATEMENT The integration of information across multiple senses, a remarkable ability of our perceptual system, is influenced by multiple context-related factors, the role of which is highly debated. It is, for instance, poorly understood how available cognitive resources influence crossmodal interactions during multisensory integration. We addressed this question using the sound-induced flash illusion (SIFI), a phenomenon in which the integration of two rapid beeps together with a flash induces the illusion of a second flash. Replicating our previous work, we demonstrate that depletion of cognitive resources through a working memory (WM) task increases the perception of the illusion. With respect to the underlying neural processes, we show that when available resources are limited, multisensory integration engages top-down θ and β oscillations.

Multisensory integration in hemianopia and unilateral spatial neglect: Evidence from the sound induced flash illusion

Cognitive deficits in depressed adults may reflect impaired decision-making. To investigate this possibility, we analyzed data from unmedicated adults with Major Depressive Disorder (MDD) and healthy controls as they performed a probabilistic reward task. The Hierarchical Drift Diffusion Model (HDDM) was used to quantify decision-making mechanisms recruited by the task, to determine if any such mechanism was disrupted by depression. Data came from two samples (Study 1: 258 MDD, 36 controls; Study 2: 23 MDD, 25 controls). On each trial, participants indicated which of two similar stimuli was presented; correct identifications were rewarded. Quantile-probability plots and the HDDM quantified the impact of MDD on response times (RT), speed of evidence accumulation (drift rate), and the width of decision thresholds, among other parameters. RTs were more positively skewed in depressed v. healthy adults, and the HDDM revealed that drift rates were reduced-and decision thresholds were wider-in the MDD groups. This pattern suggests that depressed adults accumulated the evidence needed to make decisions more slowly than controls did. Depressed adults responded slower than controls in both studies, and poorer performance led the MDD group to receive fewer rewards than controls in Study 1. These results did not reflect a sensorimotor deficit but were instead due to sluggish evidence accumulation. Thus, slowed decision-making-not slowed perception or response execution-caused the performance deficit in MDD. If these results generalize to other tasks, they may help explain the broad cognitive deficits seen in depression.

It is well established that physical activity leads to numerous health, cognitive, and psychological benefits. However, to date, very few studies have investigated the impact of physical activity on multisensory perception, that is, the brain’s capacity to integrate information across sensory modalities. Furthermore, it is unknown what level of long-term physical activity is associated with multisensory integration in adults. We explored the relationship between multisensory integration and a ten-year physical activity trajectory in 2,974 adults aged 50+ from The Irish Longitudinal Study on Ageing by measuring susceptibility to the Sound Induced Flash Illusion (SIFI) at multiple audio-visual temporal asynchronies. Physical activity was measured using the International Physical Activity Questionnaire (IPAQ-SF) at 2 years intervals over ten years. We used latent class trajectory modelling to identify latent growth classes of individuals following a similar trajectory of physical activity over time. We analysed the association of this trajectory with performance accuracy to the illusion trials of the SIFI task with generalized logistic mixed effects regression models, adjusted for several covariates. Results showed that more precise integration (i.e., lower SIFI susceptibility with larger temporal asynchronies) was associated with a higher level of sustained physical activity across ten years. Although the use of self-reported physical activity and a short version of the SIFI task limit our conclusions to some extent, nonetheless, the results suggest that sustained physical activity is associated with more precise multisensory integration, which in turn is linked to better balance and a lower risk of falling in older adults.

Many natural events generate both visual and auditory signals, and humans are remarkably adept at integrating information from those sources. However, individuals appear to differ markedly in their ability or propensity to combine what they hear with what they see. Individual differences in audiovisual integration have been established using a range of materials, including speech stimuli (seeing and hearing a talker) and simpler audiovisual stimuli (seeing flashes of light combined with tones). Although there are multiple tasks in the literature that are referred to as "measures of audiovisual integration," the tasks themselves differ widely with respect to both the type of stimuli used (speech versus non-speech) and the nature of the tasks themselves (e.g., some tasks use conflicting auditory and visual stimuli whereas others use congruent stimuli). It is not clear whether these varied tasks are actually measuring the same underlying construct: audiovisual integration. This study tested the relationships among four commonly-used measures of audiovisual integration, two of which use speech stimuli (susceptibility to the McGurk effect and a measure of audiovisual benefit), and two of which use non-speech stimuli (the sound-induced flash illusion and audiovisual integration capacity). We replicated previous work showing large individual differences in each measure but found no significant correlations among any of the measures. These results suggest that tasks that are commonly referred to as measures of audiovisual integration may be tapping into different parts of the same process or different constructs entirely.

Stage-specific computational mechanisms of working memory deficits in first-episode and chronic schizophrenia.

People display systematic priorities to self-related stimuli. As the self is not a unified entity, however, it remains unclear which aspects of the self are crucial to producing this stimulus prioritization. To explore this issue, we manipulated the valence of the self-concept (good me vs. bad me) — a core identity-based facet of the self — using a standard shape-label association task in which participants initially learned the associations (e.g., circle/good-self, triangle/good-other, diamond/bad-self, square/bad-other), after which they completed shape-label matching and shape-categorization tasks, such that attention was directed to different aspects of the stimuli (i.e., self-relevance and valence). The results revealed that responses were more efficient to the good-self shape (vs. other shapes), regardless of the task that was undertaken. A hierarchical drift diffusion model (HDDM) analysis indicated that this good-self prioritization effect was underpinned by differences in the rate of information uptake. These findings demonstrate that activation of the good-self representation exclusively facilitates perceptual decision-making, thereby furthering understanding of the self-prioritization effect.

The modulatory role of pre-SMA in speed-accuracy tradeoff: A bi-directional TMS study

When exposed to complementary features of information across sensory modalities, our brains formulate cross-modal associations between features of stimuli presented separately to multiple modalities. For example, auditory pitch-visual size associations map high-pitch tones with small-size visual objects, and low-pitch tones with large-size visual objects. Preferential, or congruent, cross-modal associations have been shown to affect behavioural performance, i.e. choice accuracy and reaction time (RT) across multisensory decision-making paradigms. However, the neural mechanisms underpinning such influences in perceptual decision formation remain unclear. Here, we sought to identify when perceptual improvements from associative congruency emerge in the brain during decision formation. In particular, we asked whether such improvements represent ‘early’ sensory processing benefits, or ‘late’ post-sensory changes in decision dynamics. Using a modified version of the Implicit Association Test (IAT), coupled with electroencephalography (EEG), we measured the neural activity underlying the effect of auditory stimulus-driven pitch-size associations on perceptual decision formation. Behavioural results showed that participants responded significantly faster during trials when auditory pitch was congruent, rather than incongruent, with its associative visual size counterpart. We used multivariate Linear Discriminant Analysis (LDA) to characterise the spatiotemporal dynamics of EEG activity underpinning IAT performance. We found an ‘Early’ component (∼100–110 ms post-stimulus onset) coinciding with the time of maximal discrimination of the auditory stimuli, and a ‘Late’ component (∼330–340 ms post-stimulus onset) underlying IAT performance. To characterise the functional role of these components in decision formation, we incorporated a neurally-informed Hierarchical Drift Diffusion Model (HDDM), revealing that the Late component decreases response caution, requiring less sensory evidence to be accumulated, whereas the Early component increased the duration of sensory-encoding processes for incongruent trials. Overall, our results provide a mechanistic insight into the contribution of ‘early’ sensory processing, as well as ‘late’ post-sensory neural representations of associative congruency to perceptual decision formation.

Ongoing neural oscillations reflect fluctuations of cortical excitability. A growing body of research has underlined the role of neural oscillations for stimulus processing. Neural oscillations in the alpha band have gained special interest in electrophysiological research on perception. Recent studies proposed the idea that neural oscillations provide temporal windows in which sensory stimuli can be perceptually integrated. This also includes multisensory integration. In the current high-density EEG-study we examined the relationship between the individual alpha frequency (IAF) and cross-modal audiovisual integration in the sound-induced flash illusion (SIFI). In 26 human volunteers we found a negative correlation between the IAF and the SIFI illusion rate. Individuals with a lower IAF showed higher audiovisual illusions. Source analysis suggested an involvement of the visual cortex, especially the calcarine sulcus, for this relationship. Our findings corroborate the notion that the IAF affects the cross-modal integration of auditory on visual stimuli in the SIFI. We integrate our findings with recent observations on the relationship between audiovisual integration and neural oscillations and suggest a multifaceted influence of neural oscillations on multisensory processing.

It has been suggested that patients with schizophrenia (SZs) show impairment in multisensory integration (De Gelder et al., 2003; Stevenson et al., 2017; Williams et al., 2010). A work in our group examined the audio-visual temporal integration with sound-induced flash illusion and found a lengthened temporal binding window for the bisensory information in SZs (Kim et al., IMRF 2017). The present study investigated the audio-visual spatial integration in schizophrenia by using the ventriloquist illusion paradigm. Ventriloquist illusion indicates biased perception of the spatial position of an auditory stimulus toward the position of a temporally synchronized visual stimulus. A group of 22 SZs and a group of 22 healthy controls performed an auditory localization task when a beep sounded with or without a synchronized LED flash. Eight loudspeakers with attached LEDs were arranged in a semicircular array, half of which were on the left, and the other half were on the right side of the central fixation LED. Each loudspeaker was 14-deg away from its adjacent one. The audio-visual stimuli were presented 0-deg, 14-deg or 28-deg apart. The ventriloquist effect was defined as the ratio of biased sound localization towards the visual stimulus to the spatial disparity between audio-visual stimuli. Following the experiment, SZs were given a structured psychiatric interview to assess the severity of positive and negative symptoms with SAPS and SANS respectively. Results showed a typical ventriloquist effect in both groups with no statistically significant difference. However, in SZs, the ventriloquist effect showed a negative correlation with the severity of visual hallucinations, and a marginally significant negative correlation with the global ratings of hallucinations. There was no correlation between hallucinations and unisensory auditory localization performance. These results demonstrate that hallucinations were implicated in reduced audio-visual spatial integration, which might be related to impaired multisensory processing, not unisensory processing.