Auditory Modality Research Articles

Modality-Attention Promotes the Neural Effects of Precise Timing Prediction in Early Sensory Processing.

Precise timing prediction (TP) enables the brain to accurately predict the occurrence of upcoming events in millisecond timescale, which is fundamental for adaptive behaviors. The neural effect of the TP within a single sensory modality has been widely studied. However, less is known about how precise TP works when the brain is concurrently faced with multimodality sensory inputs. Modality attention (MA) is a crucial cognitive function for dealing with the overwhelming information induced by multimodality sensory inputs. Therefore, it is necessary to investigate whether and how the MA influences the neural effects of the precise TP. This study designed a visual-auditory temporal discrimination task, in which the MA was allocated to visual or auditory modality, and the TP was manipulated into no timing prediction (NTP), matched timing prediction (MTP), and violated timing prediction (VTP) conditions. Behavioral and electroencephalogram (EEG) data were recorded from 27 subjects, event-related potentials (ERP), time-frequency distributions of inter-trial coherence (ITC), and event-related spectral perturbation (ERSP) were analyzed. In the visual modality, precise TP led to N1 amplitude and 200-400 ms theta ITC variations. Such variations only emerged when the MA was attended. In auditory modality, the MTP had the largest P2 amplitude and delta ITC than other TP conditions when the MA was attended, whereas the distinctions disappeared when the MA was unattended. The results suggest that the MA promoted the neural effects of the precise TP in early sensory processing, which provides more neural evidence for better understanding the interactions between the TP and MA.

Informing Healthcare Alarm Design and Use: A Human Factors Cross-Industry Perspective

Background: Alarms are signals intended to capture and direct human attention to a potential issue that may require monitoring, assessment, or intervention and play a critical safety role in high-risk industries. Healthcare relies heavily on auditory and visual alarms. While there are some guidelines to inform alarm design and use, alarm fatigue and other alarm issues are challenges in the healthcare setting. Automotive, aviation, and nuclear industries have used the science of human factors to develop alarm design and use guidelines. These guidelines may provide important insights for advancing patient safety in healthcare. Methods: We identified documents containing alarm design and use guidelines from the automotive, aviation, and nuclear industries that have been endorsed by oversight agencies. These guidelines were reviewed by human factors and clinical experts to identify those most relevant to healthcare, qualitatively analyze the relevant guidelines to identify meaningful topics, synthesize the guidelines under each topic to identify key commonalities and differences, and describe how the guidelines might be considered by healthcare stakeholders to improve alarm design and use. Results: A total of 356 guidelines were extracted from industry documents (2012–present) and 327 (91.9%) were deemed relevant to healthcare. A qualitative analysis of relevant guidelines resulted in nine distinct topics: Alarm Reduction, Appropriateness, Context-Dependence, Design Characteristics, Mental Model, Prioritization, Specificity, Urgency, and User Control. There were several commonalities, as well as some differences, across industry guidelines. The guidelines under each topic were found to inform the auditory or visual modality, or both. Certain guidelines have clear considerations for healthcare stakeholders, especially technology developers and healthcare facilities. Conclusion: Numerous guidelines from other high-risk industries can inform alarm design and use in healthcare. Healthcare facilities can use the information presented as a framework for working with their technology developers to appropriately design and modify alarming technologies and can evaluate their clinical environments to see how alarming technologies might be improved.

General tau theory as a model to evaluate audiovisual interplay in interceptive actions

When interacting with the environment, sensory information is essential to guide movements. Picking up the appropriate sensory information (both visual and auditory) about the progression of an event is required to reach the right place at the right time. In this study, we aimed to see if general tau theory could explain the audiovisual guidance of movement in interceptive action (an interception task). The specific contributions of auditory and visual sensory information were tested by timing synchronous and asynchronous audiovisual interplays in successful interceptive trials. The performance was computed by using the tau-coupling model for information-movement guidance. Our findings revealed that while the auditory contribution to movement guidance did change across conditions, the visual contribution remained constant. In addition, when comparing the auditory and visual contributions, the results revealed a significant decrease in the auditory compared to the visual contribution in just one of the asynchronous conditions where the visual target was presented after the sound. This may be because more attention was drawn to the visual information, resulting in a decrease in the auditory guidance of movement. To summarize, our findings reveal how tau-coupling can be used to disentangle the relative contributions of the visual and auditory sensory modalities in movement planning.

Auditory, tactile, and multimodal noise reduce balance variability

Auditory and somatosensory white noise can stabilize standing balance. However, the differential effects of auditory and tactile noise stimulation on balance are unknown. Prior work on unimodal noise stimulation showed gains in balance with white noise through the auditory and tactile modalities separately. The current study aims to examine whether multimodal noise elicits similar responses to unimodal noise. We recorded the postural sway of healthy young adults who were presented with continuous white noise through the auditory or tactile modalities and through a combination of both (multimodal condition) using a wearable device. Our results replicate previous work that showed that auditory or tactile noise reduces sway variability with and without vision. Additionally, we show that multimodal noise also reduces the variability of sway. Analysis of different frequency bands of sway is typically used to separate open-loop exploratory (< 0.3 Hz) and feedback-driven (> 0.3 Hz) sway. We performed this analysis and showed that unimodal and multimodal white noise affected postural sway variability similarly in both timescales. These results support that the sensory noise effects on balance are robust across unimodal and multimodal conditions and can affect both mechanisms of sway represented in the frequency spectrum. In future work, the parameters of acoustic/tactile manipulation should be optimized for the most effective balance stabilization, and multimodal therapies should be explored for older adults with typical age-related balance instabilities.

Salient omissions-pupil dilation in response to unexpected omissions of sound and touch.

Recent theories describe perception as an inferential process based on internal predictive models adjusted by means of prediction violations (prediction error). To study and demonstrate predictive processing in the brain the use of unexpected stimulus omissions has been suggested as a promising approach as the evoked brain responses are uncontaminated by responses to stimuli. Here, we aimed to investigate the pupil's response to unexpected stimulus omissions in order to better understand surprise and orienting of attention resulting from prediction violation. So far only few studies have used omission in pupillometry research and results have been inconsistent. This study adapted an EEG paradigm that has been shown to elicit omission responses in auditory and somatosensory modalities. Healthy adults pressed a button at their own pace, which resulted in the presentation of sounds or tactile stimuli in either 88%, 50% or 0% (motor-control) of cases. Pupil size was recorded continuously and averaged to analyze the pupil dilation response associated with each condition. Results revealed that omission responses were observed in both modalities in the 88%-condition compared to motor-control. Similar pupil omission responses were observed between modalities, suggesting modality-unspecific activation of the underlying brain circuits. In combination with previous omission studies using EEG, the findings demonstrate predictive models in brain processing and point to the involvement of subcortical structures in the omission response. Our pupillometry approach is especially suitable to study sensory prediction in vulnerable populations within the psychiatric field.

Pre-stimulus gamma power in human posteromedial cortex shows supra-modal mechanisms in predicting the amplitude and latency of task-induced suppression.

Upon repetitively performing the same well-practiced task on identical bottom-up stimuli, our performance still varies. Although it has been well documented that elevated pre-stimulus baseline activity in the human default-mode network impairs the subsequent task performance, it remains unknown (i) the fine-grained temporal dynamics and (ii) whether the underlying neural dynamics are supra-modal or modality-specific. We utilized intracranial recordings in the human posteromedial cortex (PMC) during a simple visual and an auditory detection task. Our findings suggested that the pre-stimulus gamma power in PMC predicted the subsequent task performance. Critically, the higher the pre-stimulus gamma power, the longer it took for it to be suppressed, and the less suppressed it was during the task performance, which eventually resulted in deleterious effects on task performance, i.e. longer reaction times. These fine-grained temporal dynamics were consistent between the visual and auditory simple detection task. In addition, a direct comparison between the visual and auditory modality showed that the between-modality difference emerged during the recovery period from the maximal gamma suppression back to the baseline. Taken together, the present results contribute novel spatio-temporal mechanisms in human PMC on how simple detection performance varies across multiple repetitions, irrespective of the sensory modality involved.

The multimodal emotion perception in codependent individuals

The emotional disturbances of individuals with codependency are often ignored. This study aimed to investigate the emotional perception of codependent individuals in four modalities – visual, auditory, tactile and olfactory. An EEG study was performed and presented pleasant and unpleasant stimuli selected by a panel of experts for each modality. Participants (fifteen codependent individuals and fifteen healthy volunteers) were instructed to assess the emotional impact and pleasantness of stimuli. The method of EEG spaces was used to visualize how close perceived stimuli were according to EEG data. The results showed ambivalence of emotional response to emotional stimuli with social component and lack of recognition of emotional tone detected in EEG and behavioral levels. The empathy feeling in codependent individuals was detected for fewer emotional stimuli. The group differences were associated with evolutionary newer modalities (auditory and visual). The lack of emotional perception in codependent subjects was determined by social factors and was detected in visual and auditory modalities, which were more involved in social interactions than olfactory and tactile modalities.

The Functional Brain Network of Subcortical and Cortical Regions Underlying Time Estimation: An Functional MRI Study

Time estimation is fundamental for human survival. There have been increasing studies suggesting that distributed brain regions, such as the basal ganglia, cerebellum and the parietal cortex, may contribute to a dedicated neural mechanism of time estimation. However, evidence on the specific function of the subcortical and cortical brain regions and the interplay of them is scare. In this work, we explored how the subcortical and cortical networks function in time estimation during a time reproduction task using functional MRI (fMRI). Thirty healthy participants performed the time reproduction task in both auditory and visual modalities. Results showed that time estimation in visual and auditory modality recruited a subcortical-cortical brain network including the left caudate, left cerebellum, and right precuneus. Besides, the superior temporal gyrus (STG) was found essential in the difference between time estimation in visual and auditory modality. Using psychophysiological interaction (PPI) analysis, we observed an increase in the connection between left caudate and left precuneus using the left caudate as the seed region in temporal reproduction task than control task. This suggested that the left caudate is the key region connecting and transmitting information to other brain regions in the dedicated brain network of time estimation.

A Cross-Modal Investigation of Statistical Learning in Developmental Dyslexia

ABSTRACT Structural patterns existing in language can be exploited for implicit prediction of sequences in speech and visual input via a process termed statistical learning (SL). Despite extensive examination of SL in dyslexia, whether SL problems arise from modality-constrained learning processes or from global learning processes is still unknown, nor is it clear how SL can be supported. Purpose The present study used the triplet paradigm to explore SL among young adults with dyslexia and among typical readers across auditory and visual modalities and tested whether information from one sensory modality can assist SL in a different sensory modality. Method Participants performed auditory and visual SL tasks under conditions in which a consistent visual/auditory cue respectively accompanied the auditory/visual triplets or under conditions in which no cross-modal information was presented. Results SL performance was poorer in the dyslexia group than among typical readers across visual and auditory modalities. Furthermore, both groups improved their SL abilities under conditions in which cues were consistent with triplet boundaries compared to under conditions lacking cross-modal information Conclusions These findings suggest that SL impairments observed in dyslexia stem from a domain-general deficiency and that cross-modal information can be recruited to support SL in dyslexia.

Dynamic causal modeling analysis reveals the modulation of motor cortex and integration in superior temporal gyrus during multisensory speech perception.

The online version contains supplementary material available at 10.1007/s11571-023-09945-z.

Caressed by music: Related preferences for velocity of touch and tempo of music?

Given that both hearing and touch are 'mechanical senses' that respond to physical pressure or mechanical energy and that individuals appear to have a characteristic internal or spontaneous tempo, individual preferences in musical and touch rhythms might be related. We explored this in two experiments probing individual preferences for tempo in the tactile and auditory modalities. Study 1 collected ratings of received stroking on the forearm and measured the velocity the participants used for stroking a fur. Music tempo preferences were assessed as mean beats per minute of individually selected music pieces and via the adjustment of experimenter-selected music to a preferred tempo. Heart rate was recorded to measure levels of physiological arousal. We found that the preferred tempo of favorite (self-selected) music correlated positively with the velocity with which each individual liked to be touched. In Study 2, participants rated videos of repeated touch on someone else's arm and videos of a drummer playing with brushes on a snare drum, both at a variety of tempos. We found that participants with similar rating patterns for the different stroking speeds did not show similar rating patterns for the different music beats. The results suggest that there may be a correspondence between preferences for favorite music and felt touch, but this is either weak or it cannot be evoked effectively with vicarious touch and/or mere drum beats. Thus, if preferences for touch and music are related, this is likely to be dependent on the specific type of stimulation.

How do human newborns come to understand the multimodal environment?

For a long time, newborns were considered as human beings devoid of perceptual abilities who had to learn with effort everything about their physical and social environment. Extensive empirical evidence gathered in the last decades has systematically invalidated this notion. Despite the relatively immature state of their sensory modalities, newborns have perceptions that are acquired, and are triggered by, their contact with the environment. More recently, the study of the fetal origins of the sensory modes has revealed that in utero all the senses prepare to operate, except for the vision mode, which is only functional starting from the first minutes after birth. This discrepancy between the maturation of the different senses leads to the question of how human newborns come to understand our multimodal and complex environment. More precisely, how the visual mode interacts with the tactile and auditory modes from birth. After having defined the tools that newborns use to interact with other sensory modalities, we review studies across different fields of research such as the intermodal transfer between touch and vision, auditory-visual speech perception, and the existence of links between the dimensions of space, time, and number. Overall, evidence from these studies supports the idea that human newborns are spontaneously driven, and cognitively equipped, to link information collected by the different sensory modes in order to create a representation of a stable world.

Sensory modality defines the relation between EEG Lempel–Ziv diversity and meaningfulness of a stimulus

Diversity of brain activity is a robust neural correlate of global states of consciousness. It has been proposed that diversity measures specifically reflect the temporal variability of conscious experience. Previous studies supported this hypothesis by showing that perception of meaningful visual stimuli causes richer, more-variable experiences than perception of meaningless stimuli, and this is reflected in greater brain signal diversity. To investigate whether this relation is consistent across sensory modalities, to participants we presented three versions of naturalistic visual and auditory stimuli (videos and audiobooks) that varied in the amount of meaning (original, scrambled, and noise), while recording electroencephalographic signals. We report three main findings. First, greater meaningfulness of visual stimuli was related to higher Lempel–Ziv diversity of EEG signals, but the opposite effect was found in the auditory modality. Second, visual perception was related to generally higher EEG diversity than auditory perception. Third, perception of meaningful visual stimuli and auditory stimuli respectively resulted in higher and lower EEG diversity in comparison to the resting state. In conclusion, the signal diversity of continuous brain signals depends on the stimulated sensory modality, therefore it is not a generic index of the variability of conscious experience.

β-Bursts over Frontal Cortex Track the Surprise of Unexpected Events in Auditory, Visual, and Tactile Modalities.

One of the fundamental ways in which the brain regulates and monitors behavior is by making predictions about the sensory environment and adjusting behavior when those expectations are violated. As such, surprise is one of the fundamental computations performed by the human brain. In recent years, it has been well established that one key aspect by which behavior is adjusted during surprise is inhibitory control of the motor system. Moreover, because surprise automatically triggers inhibitory control without much proactive influence, it can provide unique insights into largely reactive control processes. Recent years have seen tremendous interest in burst-like β frequency events in the human (and nonhuman) local field potential-especially over (p)FC-as a potential signature of inhibitory control. To date, β-bursts have only been studied in paradigms involving a substantial amount of proactive control (such as the stop-signal task). Here, we used two cross-modal oddball tasks to investigate whether surprise processing is accompanied by increases in scalp-recorded β-bursts. Indeed, we found that unexpected events in all tested sensory domains (haptic, auditory, visual) were followed by low-latency increases in β-bursting over frontal cortex. Across experiments, β-burst rates were positively correlated with estimates of surprise derived from Shannon's information theory, a type of surprise that represents the degree to which a given stimulus violates prior expectations. As such, the current work clearly implicates frontal β-bursts as a signature of surprise processing. We discuss these findings in the context of common frameworks of inhibitory and cognitive control after unexpected events.

Global and local priming in a multi-modal context.

Perceptual information can be processed at many different scales, from featural details to entire scenes. Attentional selection of different scales has been studied using hierarchical stimuli, with research elucidating a variety of biases in local and global attentional selection (due to, e.g., stimulus properties, brain injury, and experience). In this study, the emphasis is on biases produced through recent experience, or level-specific priming effects, which have been demonstrated within both the visual and auditory modalities. Namely, when individuals attend to local information, they are subsequently biased to attend locally (and similarly so with global attention). Here, these level-specific priming effects are investigated in a multi-modal context to determine whether cross-modal interactions occur between visual and auditory modalities during hierarchical processing. Specifically, the study addresses if attentional selection of local or global information in the visual modality subsequently biases auditory attentional selection to that level, and vice versa (i.e., level-priming). Though expected identity priming effects emerged in the study, no cross-modal level-priming effects manifested. Furthermore, the multi-modal context eliminated the well-established within-modality level-specific priming effects. Thus, though the study does reveal a multi-modal effect, it was not a level-based effect. Instead, paradoxically, the multi-modal context eliminated attentional scope biases (i.e., level-priming) within uni-modal transitions. In other words, when visual and auditory information are equally likely require attention, no persistence emerges for processing local or global information over time, even within a single modality.

Dynamics of alpha suppression index both modality specific and general attention processes

EEG alpha power varies under many circumstances requiring visual attention. However, mounting evidence indicates that alpha may not only serve visual processing, but also the processing of stimuli presented in other sensory modalities, including hearing. We previously showed that alpha dynamics during an auditory task vary as a function of competition from the visual modality (Clements et al., 2022) suggesting that alpha may be engaged in multimodal processing. Here we assessed the impact of allocating attention to the visual or auditory modality on alpha dynamics at parietal and occipital electrodes, during the preparatory period of a cued-conflict task. In this task, bimodal precues indicated the modality (vision, hearing) relevant to a subsequent reaction stimulus, allowing us to assess alpha during modality-specific preparation and while switching between modalities. Alpha suppression following the precue occurred in all conditions, indicating that it may reflect general preparatory mechanisms. However, we observed a switch effect when preparing to attend to the auditory modality, in which greater alpha suppression was elicited when switching to the auditory modality compared to repeating. No switch effect was evident when preparing to attend to visual information (although robust suppression did occur in both conditions). In addition, waning alpha suppression preceded error trials, irrespective of sensory modality. These findings indicate that alpha can be used to monitor the level of preparatory attention to process both visual and auditory information, and support the emerging view that alpha band activity may index a general attention control mechanism used across modalities.

Transcranial high-frequency random noise stimulation does not modulate Nogo N2 and Go/Nogo reaction times in somatosensory and auditory modalities

Transcranial random noise stimulation (tRNS) of the primary sensory or motor cortex can improve sensorimotor functions by enhancing circuit excitability and processing fidelity. However, tRNS is reported to have little effect on higher brain functions, such as response inhibition when applied to associated supramodal regions. These discrepancies suggest differential effects of tRNS on the excitability of the primary and supramodal cortex, although this has not been directly demonstrated. This study examined the effects of tRNS on supramodal brain regions on somatosensory and auditory Go/Nogo task performance, a measure of inhibitory executive function, while simultaneously recording event-related potentials (ERPs). Sixteen participants received sham or tRNS stimulation of the dorsolateral prefrontal cortex in a single-blind crossover design study. Neither sham nor tRNS altered somatosensory and auditory Nogo N2 amplitudes, Go/Nogo reaction times, or commission error rates. The results suggest that current tRNS protocols are less effective at modulating neural activity in higher-order cortical regions than in the primary sensory and motor cortex. Further studies are required to identify tRNS protocols that effectively modulate the supramodal cortex for cognitive enhancement.

Sensorimotor synchronization with visual, auditory, and tactile modalities.

While it is well known that humans are highly responsive to rhythm, the factors that influence our ability to synchronize remain unclear. In the current study, we examined how stimulus modality and rhythmic deviation, along with the synchronizer's level of musicality, impacted sensorimotor synchronization (SMS). Utilizing a finger-tapping task and three sensory modalities (visual, auditory, and tactile), we manipulated rhythmic deviation by varying the temporal position, intensity, and availability of cues across four deviation levels. Additionally, to determine our participants' musical familiarity and aptitude, we administered the Goldsmiths Musical Sophistication Index (Gold-MSI) questionnaire. We found that SMS to external rhythmic stimuli was significantly more precise for auditory and tactile than for visual sequences. Further, we found SMS consistency significantly decreased in all modalities with increased rhythmic deviation, suggesting rhythmic deviation directly relates to SMS difficulty. Moreover, a significant correlation was found between Gold-MSI scores and SMS consistency in the most rhythmically deviant level, such that the higher one's musical general sophistication score, the greater one's SMS ability. This held for all three modalities. Combined, these findings suggest that rhythmic synchronization performance is affected not only by the modality and rhythmic deviation of the stimuli but also by the musical general sophistication of the synchronizer.

Bringing the Tranquillity Rating Prediction Tool (TRAPT) Indoors - A Case Study at The Warren, Hull

Tranquil environments are considered quiet, peaceful places where people can get away from everyday life, and for many years the University of Bradford's Tranquillity Rating Prediction Tool (TRAPT) has been used to successfully predict the perceived tranquillity of outdoor spaces as a function of their auditory and visual modalities. The Warren, Hull, established 1983, is a citywide free project for marginalised and vulnerable young people aged 14-25 that helps to encourage and support them to take control of their lives and futures by engaging them in all of the decision-making that has impact upon the strategic and day to day running of project. Staff at The Warren have noticed that the young people often gather around a large public fountain to socialise and relax; the same fountain is due to undergo major work in 2022 and will be sectioned off from the general public for months. This paper addresses the challenges associated with bringing tranquillity indoors via considered and experimental application of the TRAPT tool to indoor spaces, and via empowerment of the young people to introduce tranquil auditory and visual changes to multiple spaces in The Warren.

Best practice recommendations for using music with children and young people with disorders of consciousness

Children and youth with Disorders of Consciousness (DoC) present with highly complex medical, physical, and behavioral needs. Additionally, involving such children and young people in meaningful activities with caregivers can be challenging due to the combined motor, sensory, cognitive and communication impairments. Authoritative guidelines for engaging children and youth in meaningful sensory experiences that can enhance social and emotional experiences are lacking due to the minimal research with this population. Music is a medium that optimizes sensory stimulation using salient stimuli in the auditory modality. Furthermore, music experiences require minimal physical demands and offer opportunities for social interactions in line with developmental milestones. The primary objective of this paper is to offer best-practice recommendations for musical stimulation with children and youth with DoC, including its use in leisure and relaxation activities. These recommendations are synthesized from the existing evidence and combined with expert opinion. They are intended for caregivers of children and young people with DoC, which include paid healthcare professionals and providers, as well as non-paid caregivers such as family, friends, and volunteers. These practical suggestions may be applicable in a variety of settings, including hospitals, educational and residential care settings, family homes, and rehabilitation units.