Speech Stimuli Research Articles

On the Relative Importance of Visual and Spatial Audio Rendering on VR Immersion

A study was performed using a virtual environment to investigate the relative importance of spatial audio fidelity and video resolution on perceived audio-visual quality and immersion. Subjects wore a head-mounted display and headphones and were presented with a virtual environment featuring music and speech stimuli using three levels each of spatial audio quality and video resolution. Spatial audio was rendered monaurally, binaurally with head-tracking, and binaurally with head-tracking and room acoustic rendering. Video was rendered at resolutions of 0.5 megapixels per eye, 1.5 megapixels per eye, and 2.5 megapixels per eye. Results showed that both video resolution and spatial audio rendering had a statistically significant effect on both immersion and audio-visual quality. Most strikingly, the results showed that under the conditions that were tested in the experiment, the addition of room acoustic rendering to head-tracked binaural audio had the same improvement on immersion as increasing the video resolution five-fold, from 0.5 megapixels per eye to 2.5 megapixels per eye.

Heritability of Sound Processing Deficits in Autism: Neural Insights

Heritability of Sound Processing Deficits in Autism: Neural Insights

Do Voice-Based Judgments of Socially Relevant Speaker Traits Differ Across Speech Types?

The human voice is a powerful and evolved social tool, with hundreds of studies showing that nonverbal vocal parameters robustly influence listeners' perceptions of socially meaningful speaker traits, ranging from perceived gender and age to attractiveness and trustworthiness. However, these studies have utilized a wide variety of voice stimuli to measure listeners' voice-based judgments of these traits. Here, in the largest scale study known to date, we test whether listeners judge the same unseen speakers differently depending on the complexity of the neutral speech stimulus, from single vowel sounds to a full paragraph. In a playback experiment testing 2,618 listeners, we examine whether commonly studied voice-based judgments of attractiveness, trustworthiness, dominance, likability, femininity/masculinity, and health differ if listeners hear isolated vowels, a series of vowels, single words, single sentences (greeting), counting from 1 to 10, or a full paragraph recited aloud (Rainbow Passage), recorded from the same 208 men and women. Data were collected using a custom-designed interface in which vocalizers and traits were randomly assigned to raters. Linear-mixed models show that the type of voice stimulus does indeed consistently affect listeners' judgments. Overall, ratings of attractiveness, trustworthiness, dominance, likability, health, masculinity among men, and femininity among women increase as speech duration increases. At the same time, speaker-level regression analyses show that interindividual differences in perceived speaker traits are largely preserved across voice stimuli, especially among those of a similar duration. Socially relevant perceptions of speakers are not wholly changed but rather moderated by the length of their speech. Indeed, the same vocalizer is perceived in a similar way regardless of which neutral statements they speak, with the caveat that longer utterances explain the most shared variance in listeners' judgments and elicit the highest ratings on all traits, possibly by providing additional nonverbal information to listeners. https://doi.org/10.23641/asha.21158890.

Focus perception in Japanese: Effects of lexical accent and focus location.

This study explored the contexts in which native Japanese listeners have difficulty identifying prosodic focus. Using a 4AFC identification task, we compared native Japanese listeners’ focus identification accuracy in different lexical accent × focus location conditions using resynthesised speech stimuli, which varied only in fundamental frequency. Experiment 1 compared the identification accuracy in lexical accent × focus location conditions using both natural and resynthesised stimuli. The results showed that focus identification rates were similar with the two stimulus types, thus establishing the reliability of the resynthesised stimuli. Experiment 2 explored these conditions further using only resynthesised stimuli. Narrow foci bearing the lexical pitch accent were always more correctly identified than unaccented ones, whereas the identification rate for final focus was the lowest among all focus locations. From these results, we argue that the difficulty of focus perception in Japanese is attributed to (i) the blocking of PFC by unaccented words, and (ii) similarity in F0 contours between lexical pitch accent and narrow focus, including in particular the similarity between downstep and PFC. Focus perception is therefore contingent on other concurrent communicative functions which may sometimes take precedence in a +PFC language.

Short-Term Habituation of Auditory N1 in Spoken Word-Forms Is Modulated by Phonological Information

Short-term auditory habituation is typically reflected by decreased but recoverable amplitudes of the N1 component of event-related potentials to repeated stimuli. It remains less well understood whether and how N1 habituation is modulated by the human cognition. The current study aims to further test for the potential modulatory roles of phonological information carried by spoken word-forms. Two phonological variables, namely lexicality (real versus pseudoword-form) and usage frequency (high versus low frequency), are considered and combined factorially, yielding four types of monosyllabic Mandarin spoken word-forms. Each type consists of 10 items (i.e., word-forms). The stimuli were passively presented to native Mandarin speakers in trains of five (S1–S5), while their EEG was recorded. The peak amplitudes of N1 to the same type of speech stimuli were calculated for each position by averaging the trains extracted from the EEG recording. Then, the N1 habituation was quantified for the two electrodes of interest (C3 and C4) in each repetitive presentation position (S2–S5). The results showed that the N1 habituation in low-frequency pseudo word-forms was consistently greater than in low-frequency real word-forms and high-frequency pseudo word-forms, respectively, at the fourth presentation (S4). The results suggest the first evidence that different types of phonological information (i.e., lexicality and usage frequency) modulate N1 habituation, interactively. Sensory filtering is proposed as a candidate mechanism for mediating between the processing of phonological information and the short-term habituation of auditory N1.

Development of a Binaural Masking Level Differences Test Application Using Tonal and Speech Stimuli in Persian and Assessing its Test-Retest Reliability in Normal-Hearing Young People

Background and Aim: The masking level difference (MLD) test is a behavioral test that assesses binaural interactions and central auditory processing. This study aims to design a binaural MLD (BMLD) test application using tonal and Speech Stimuli in Persian.
 Methods: In the first stage, the BMLD test application was designed in MATLAB software. Then, the spondaic words were recorded and added to the sound processing panel of the application. In the next stage, the MLD of tonal and speech stimuli was measured on 20 subjects aged 18–25 years. To examine the test-retest reliability of the application, all steps were repeated after one week.
 Results: The MLD values for tonal and speech stimuli under the SoNo-SπNo condition were significantly different from those under the SoNo-SoNπ condition (p<0.05). These values for tonal and speech stimuli were 10.50 and 6.12 dB under the SoNo- SπNo condition, and 6.85 and 4.43 under the SoNo- SoNπ condition. There was no significant difference in the MLD values between males and females. The intraclass correlation coefficient value for different phase conditions was more than 0.8, indicating the high reliability of the BMLD test application for tonal and speech stimuli in Persian (p<0.05).
 Conclusion: The designed BMLD test application using tonal and speech stimuli in Persian has similar results compared to the conventional methods of MLD test for other languages, therefore, this application can be used in audiology clinics in Iran to perform the MLD test.
 Keywords: Release from masking; masking; binaural hearing; software

Auditory Discrimination Elicited by Nonspeech and Speech Stimuli in Children With Congenital Hearing Loss.

Congenital deafness not only delays auditory development but also hampers the ability to perceive nonspeech and speech signals. This study aimed to use auditory event-related potentials to explore the mismatch negativity (MMN), P3a, negative wave (Nc), and late discriminative negativity (LDN) components in children with and without hearing loss. Nineteen children with normal hearing (CNH) and 17 children with hearing loss (CHL) participated in this study. Two sets of pure tones (1 kHz vs. 1.1 kHz) and lexical tones (/ba2/ vs. /ba4/) were used to examine the auditory discrimination process. MMN could be elicited by the pure tone and the lexical tone in both groups. The MMN latency elicited by nonspeech and speech was later in CHL than in CNH. Additionally, the MMN latency induced by speech occurred later in the left than in the right hemisphere in CNH, and the MMN amplitude elicited by speech in CHL produced a discriminative deficiency compared with that in CNH. Although the P3a latency and amplitude elicited by nonspeech in CHL and CNH were not significantly different, the Nc amplitude elicited by speech performed much lower in CHL than in CNH. Furthermore, the LDN latency elicited by nonspeech was later in CHL than in CNH, and the LDN amplitude induced by speech showed higher dominance in the right hemisphere in both CNH and CHL. By incorporating nonspeech and speech auditory conditions, we propose using MMN, Nc, and LDN as potential indices to investigate auditory perception, memory, and discrimination.

The role of informational content of visual speech in an audiovisual cocktail party: Evidence from cortical oscillations in young and old participants.

Age-related differences in the processing of audiovisual speech in a multi-talker environment were investigated analysing event-related spectral perturbations (ERSPs), focusing on theta, alpha and beta oscillations that are assumed to reflect conflict processing, multisensory integration and attentional mechanisms, respectively. Eighteen older and 21 younger healthy adults completed a two-alternative forced-choice word discrimination task, responding to audiovisual speech stimuli. In a cocktail-party scenario with two competing talkers (located at -15° and 15° azimuth), target words (/yes/or/no/) appeared at a pre-defined (attended) position, distractor words at the other position. In two audiovisual conditions, acoustic speech was combined either with informative or uninformative visual speech. While a behavioural benefit for informative visual speech occurred for both age groups, differences between audiovisual conditions in the theta and beta band were only present for older adults. A stronger increase in theta perturbations for stimuli containing uninformative visual speech could be associated with early conflict processing, while a stronger suppression in beta perturbations for informative visual speech could be associated to audiovisual integration. Compared to the younger group, the older group showed generally stronger beta perturbations. No condition differences in the alpha band were found. Overall, the findings suggest age-related differences in audiovisual speech integration in a multi-talker environment. While the behavioural benefit of informative visual speech was unaffected by age, older adults had a stronger need for cognitive control when processing conflicting audiovisual speech input. Furthermore, mechanisms of audiovisual integration are differently activated depending on the informational content of the visual information.

Individual Differences in Perception of Emotion Induced by Speech Stimuli among People with Visual Disabilities

Individual Differences in Perception of Emotion Induced by Speech Stimuli among People with Visual Disabilities

Adding noise is a confounded nuisance.

A wide variety of research and clinical assessments involve presenting speech stimuli in the presence of some kind of noise. Here, I selectively review two theoretical perspectives and discuss ways in which these perspectives may help researchers understand the consequences for listeners of adding noise to a speech signal. I argue that adding noise changes more about the listening task than merely making the signal more difficult to perceive. To fully understand the effects of an added noise on speech perception, we must consider not just how much the noise affects task difficulty, but also how it affects all of the systems involved in understanding speech: increasing message uncertainty, modifying attentional demand, altering affective response, and changing motivation to perform the task.

General Auditory and Speech-Specific Contributions to Cortical Envelope Tracking Revealed Using Auditory Chimeras.

In recent years research on natural speech processing has benefited from recognizing that low-frequency cortical activity tracks the amplitude envelope of natural speech. However, it remains unclear to what extent this tracking reflects speech-specific processing beyond the analysis of the stimulus acoustics. In the present study, we aimed to disentangle contributions to cortical envelope tracking that reflect general acoustic processing from those that are functionally related to processing speech. To do so, we recorded EEG from subjects as they listened to auditory chimeras, stimuli composed of the temporal fine structure of one speech stimulus modulated by the amplitude envelope (ENV) of another speech stimulus. By varying the number of frequency bands used in making the chimeras, we obtained some control over which speech stimulus was recognized by the listener. No matter which stimulus was recognized, envelope tracking was always strongest for the ENV stimulus, indicating a dominant contribution from acoustic processing. However, there was also a positive relationship between intelligibility and the tracking of the perceived speech, indicating a contribution from speech-specific processing. These findings were supported by a follow-up analysis that assessed envelope tracking as a function of the (estimated) output of the cochlea rather than the original stimuli used in creating the chimeras. Finally, we sought to isolate the speech-specific contribution to envelope tracking using forward encoding models and found that indices of phonetic feature processing tracked reliably with intelligibility. Together these results show that cortical speech tracking is dominated by acoustic processing but also reflects speech-specific processing.SIGNIFICANCE STATEMENT Activity in auditory cortex is known to dynamically track the energy fluctuations, or amplitude envelope, of speech. Measures of this tracking are now widely used in research on hearing and language and have had a substantial influence on theories of how auditory cortex parses and processes speech. But how much of this speech tracking is actually driven by speech-specific processing rather than general acoustic processing is unclear, limiting its interpretability and its usefulness. Here, by merging two speech stimuli together to form so-called auditory chimeras, we show that EEG tracking of the speech envelope is dominated by acoustic processing but also reflects linguistic analysis. This has important implications for theories of cortical speech tracking and for using measures of that tracking in applied research.

Predicting Aided Outcome With Aided Word Recognition Scores Measured With Linear Amplification at Above-conversational Levels.

Many hearing aid (HA) users receive limited benefit from amplification, especially when trying to understand speech in noise, and they often report hearing-related residual activity limitations. Current HA fitting strategies are typically based on pure-tone hearing thresholds only, even though suprathreshold factors have been linked to aided outcomes. Furthermore, clinical measures of speech perception such as word recognition scores (WRSs) are performed without frequency-specific amplification, likely resulting in suboptimal speech audibility and thus inaccurate estimates of suprathreshold hearing abilities. Corresponding measures with frequency-specific amplification ("aided") would likely improve such estimates and enable more accurate aided outcome prediction. Here, we investigated potential links between either unaided WRSs or aided WRSs measured at several above-conversational levels and two established HA outcome measures: The Hearing-In-Noise Test (HINT) and the International Outcome Inventory for Hearing Aids (IOI-HA). Thirty-seven older individuals with bilateral hearing impairments participated. Two conditions were tested: unaided and aided, with all stimuli presented over headphones. In the unaided condition, the most comfortable level (MCL) for the presented speech stimuli, WRS at MCL+10 dB as well as uncomfortable levels (UCLs) for narrowband noise stimuli were measured. In the aided condition, all stimuli were individually amplified according to the "National Acoustic Laboratories-Revised, Profound" fitting rule. Aided WRSs were then measured using an Interacoustics Affinity system at three above-conversational levels, allowing for the maximum aided WRS as well as the presence of "rollover" in the performance-intensity function to be estimated. Multivariate data analyses were performed to examine the relations between the HINT (measured using a simulated HA with the NAL-RP amplification) or IOI-HA scores (for the participants' own HAs) and various potential predictors (age, pure-tone average hearing loss, unaided WRS, aided WRS, rollover presence [ROp], and UCL). Aided WRSs predicted the HINT scores better than any other predictor and were also the only significant predictor of the IOI-HA scores. In addition, UCL and ROp in the aided WRSs were significant predictors of the HINT scores and competed for variance in the statistical models. Neither age nor pure-tone average hearing loss could predict the two aided outcomes. Aided WRSs can predict HA outcome more effectively than unaided WRSs, age or pure-tone audiometry and could be relatively easily implemented in clinical settings. More research is necessary to better understand the relations between ROp, UCL and speech recognition at above-conversational levels.

Increased nasality of connected speech in patients underwent modified Lothrop procedure for refractory chronic rhinosinusitis: Cross-sectional and prospective study.

The endoscopic modified Lothrop procedure (EMLP) is performed to create a large frontal neostium in patients who had failed previous frontal sinus surgeries. EMLP causes obvious changes in the volume and anatomy of the sinuses, which may cause a more significant change in nasality. This study aimed to evaluate the changes in nasalance in patients who underwent EMLP by comparing their preoperative nasalance to the established normative values and postoperative nasalance. This was a prospective study. Twenty-one patients diagnosed with refractory frontal sinusitis who were indicated to undergo EMLP were enrolled. One hundred one healthy participants were enrolled as norm references. The Nasometer II Model 6400 (KayPENTAX) was used to analyze the nasalance scores. Nasalance scores were tested before surgery and 1 and 3 months after the surgery. The normative references of nasalance were (mean ± SD) 14.6% ± 6.7%, 39.4% ± 8.4%, and 55.8% ± 8.3% for the oral passage, oral-nasal passage, and nasal sentences, respectively. The mean nasalance scores pre-EMLP and 1 and 3 months post-EMLP were 23.2% ± 9.6%, 29.0% ± 9.3%, and 29.9% ± 0.4% for the oral passage; 48.7% ± 10.7%, 54.7% ± 7.7%, and 56.4% ± 7.2% for the oral-nasal passage; and 62.7% ± 10.9%, 69.8% ± 6.7%, and 70.7% ± 6.4% for the nasal sentences, respectively. Compared with the normative references, pre-EMLP nasalance was higher for all the three speech stimuli (t-test, p < 0.05). Post-EMLP nasalance also significantly increased for all the three stimuli at the 1- and 3-month follow-up visits (Paired t-test, p < 0.05). EMLP has a short-term impact on resonance; however, long-term follow-up is required for further study.

Differences of people with visual disabilities in the perceived intensity of emotion inferred from speech of sighted people in online communication settings

Purpose As humans convey information about emotions by speech signals, emotion recognition via auditory information is often employed to assess one’s affective states. There are numerous ways of applying the knowledge of emotional vocal expressions to system designs that accommodate users’ needs adequately. Yet, little is known about how people with visual disabilities infer emotions from speech stimuli, especially via online platforms (e.g., Zoom). This study focussed on examining the degree to which they perceive emotions strongly or weakly, i.e., perceived intensity but also investigating the degree to which their sociodemographic backgrounds affect them perceiving different intensity levels of emotions when exposed to a set of emotional speech stimuli via Zoom. Materials and methods A convenience sample of 30 individuals with visual disabilities participated in zoom interviews. Participants were given a set of emotional speech stimuli and reported the intensity level of the perceived emotions on a rating scale from 1 (weak) to 8 (strong). Results When the participants were exposed to the emotional speech stimuli, calm, happy, fearful, sad, and neutral, they reported that neutral was the dominant emotion they perceived with the greatest intensity. Individual differences were also observed in the perceived intensity of emotions, associated with sociodemographic backgrounds, such as health, vision, job, and age. Conclusions The results of this study are anticipated to contribute to the fundamental knowledge that will be helpful for many stakeholders such as voice technology engineers, user experience designers, health professionals, and social workers providing support to people with visual disabilities. IMPLICATIONS FOR REHABILITATION Technologies equipped with alternative user interfaces (e.g., Siri, Alexa, and Google Voice Assistant) meeting the needs of people with visual disabilities can promote independent living and quality of life. Such technologies can also be equipped with systems that can recognize emotions via users’ voice, such that users can obtain services customized to fit their emotional needs or adequately address their emotional challenges (e.g., early detection of onset, provision of advice, and so on). The results of this study can be beneficial to health professionals (e.g., social workers) who work closely with clients who have visual disabilities (e.g., virtual telehealth sessions) as they could gain insights or learn how to recognize and understand the clients’ emotional struggle by hearing their voice, which is contributing to enhancement of emotional intelligence. Thus, they can provide better services to their clients, leading to building a strong bond and trust between health professionals and clients with visual disabilities even they meet virtually (e.g., Zoom).

Cross-modal functional connectivity supports speech understanding in cochlear implant users.

Sensory deprivation can lead to cross-modal cortical changes, whereby sensory brain regions deprived of input may be recruited to perform atypical function. Enhanced cross-modal responses to visual stimuli observed in auditory cortex of postlingually deaf cochlear implant (CI) users are hypothesized to reflect increased activation of cortical language regions, but it is unclear if this cross-modal activity is "adaptive" or "mal-adaptive" for speech understanding. To determine if increased activation of language regions is correlated with better speech understanding in CI users, we assessed task-related activation and functional connectivity of auditory and visual cortices to auditory and visual speech and non-speech stimuli in CI users (n = 14) and normal-hearing listeners (n = 17) and used functional near-infrared spectroscopy to measure hemodynamic responses. We used visually presented speech and non-speech to investigate neural processes related to linguistic content and observed that CI users show beneficial cross-modal effects. Specifically, an increase in connectivity between the left auditory and visual cortices-presumed primary sites of cortical language processing-was positively correlated with CI users' abilities to understand speech in background noise. Cross-modal activity in auditory cortex of postlingually deaf CI users may reflect adaptive activity of a distributed, multimodal speech network, recruited to enhance speech understanding.

Evaluation of performance of hearing aid users using Acoustic Change Complex (ACC) and behavioral measures

ObjectiveTo validate the clinical use of Acoustic Change Complex (ACC) as an objective tool in children who use hearing Aids and explore how far ACC response parameters could be correlated to behavioral tests. MethodSixty Arabic speaking children, using binaural Hearing Aids (H.As), participated in the present study. Their age ranged from 6 to 12 years. Evaluation of Hearing aid device performance was performed through questionnaire, speech in noise tests and aided/unaided ACC recordings. Two stimuli '' vowel/o/with SNR (0 &+8) ''were developed and standardized to elicit ACC response. The replicated ACC waveforms were collected and analyzed and the aided ACC responses were compared with unaided ACC and correlated to behavioral tests. ResultsThe ACC percent identification in children using H.As using vowel/o/with SNR (+8 & 0) stimuli was 75% and 45% respectively in aided condition, the percent decreased to 40% and 15% in unaided condition. Speech in noise stimuli showed significant difference of ACC-N1 latency between the aided and unaided groups being longer in the unaided condition. The correlations between behavioral tests and ACC response (latency and amplitude) were irrelevant. ConclusionsHearing-impaired subjects can process speech stimulus at the level of the auditory cortex, and in a more effective manner when they used their H.As. ACC can be used as an objective measure for selective auditory attention ability using vowel-in-noise. However, it couldn't replace Behavioral measures.

Rate Discrimination Training May Partially Restore Temporal Processing Abilities from Age-Related Deficits.

The ability to understand speech in complex environments depends on the brain’s ability to preserve the precise timing characteristics of the speech signal. Age-related declines in temporal processing may contribute to the older adult’s experience of communication difficulty in challenging listening conditions. This study’s purpose was to evaluate the effects of rate discrimination training on auditory temporal processing. A double-blind, randomized control design assigned 77 young normal-hearing, older normal-hearing, and older hearing-impaired listeners to one of two treatment groups: experimental (rate discrimination for 100- and 300-Hz pulse trains) and active control (tone detection in noise). All listeners were evaluated during pre- and post-training sessions using perceptual rate discrimination of 100-, 200-, 300-, and 400-Hz band-limited pulse trains and auditory steady-state responses (ASSRs) to the same stimuli. Training generalization was evaluated using several temporal processing measures and sentence recognition tests that included time-compressed and reverberant speech stimuli. Results demonstrated a session × training group interaction for perceptual and ASSR testing to the trained frequencies (100 and 300 Hz), driven by greater improvements in the training group than in the active control group. Further, post-test rate discrimination of the older listeners reached levels that were equivalent to those of the younger listeners at pre-test. Generalization was observed in significant improvement in rate discrimination of untrained frequencies (200 and 400 Hz) and in correlations between performance changes in rate discrimination and sentence recognition of reverberant speech. Further, non-auditory inhibition/attention performance predicted training-related improvement in rate discrimination. Overall, the results demonstrate the potential for auditory training to partially restore temporal processing in older listeners and highlight the role of cognitive function in these gains.

Frequency-Following Response to Steady-State Vowel in Quiet and Background Noise Among Marching Band Participants With Normal Hearing

Human studies enrolling individuals at high risk for cochlear synaptopathy (CS) have reported difficulties in speech perception in adverse listening conditions. The aim of this study is to determine if these individuals show a degradation in the neural encoding of speech in quiet and in the presence of background noise as reflected in neural phase-locking to both envelope periodicity and temporal fine structure (TFS). To our knowledge, there are no published reports that have specifically examined the neural encoding of both envelope periodicity and TFS of speech stimuli (in quiet and in adverse listening conditions) among a sample with loud-sound exposure history who are at risk for CS. Using scalp-recorded frequency-following response (FFR), the authors evaluated the neural encoding of envelope periodicity (FFRENV) and TFS (FFRTFS) for a steady-state vowel (English back vowel /u/) in quiet and in the presence of speech-shaped noise presented at +5- and 0 dB SNR. Participants were young individuals with normal hearing who participated in the marching band for at least 5 years (high-risk group) and non-marching band group with low-noise exposure history (low-risk group). The results showed no group differences in the neural encoding of either the FFRENV or the first formant (F1) in the FFRTFS in quiet and in noise. Paradoxically, the high-risk group demonstrated enhanced representation of F2 harmonics across all stimulus conditions. These results appear to be in line with a music experience-dependent enhancement of F2 harmonics. However, due to sound overexposure in the high-risk group, the role of homeostatic central compensation cannot be ruled out. A larger scale data set with different noise exposure background, longitudinal measurements with an array of behavioral and electrophysiological tests is needed to disentangle the nature of the complex interaction between the effects of central compensatory gain and experience-dependent enhancement.

Evaluation of phase-locking to parameterized speech envelopes.

Humans rely on the temporal processing ability of the auditory system to perceive speech during everyday communication. The temporal envelope of speech is essential for speech perception, particularly envelope modulations below 20 Hz. In the literature, the neural representation of this speech envelope is usually investigated by recording neural phase-locked responses to speech stimuli. However, these phase-locked responses are not only associated with envelope modulation processing, but also with processing of linguistic information at a higher-order level when speech is comprehended. It is thus difficult to disentangle the responses into components from the acoustic envelope itself and the linguistic structures in speech (such as words, phrases and sentences). Another way to investigate neural modulation processing is to use sinusoidal amplitude-modulated stimuli at different modulation frequencies to obtain the temporal modulation transfer function. However, these transfer functions are considerably variable across modulation frequencies and individual listeners. To tackle the issues of both speech and sinusoidal amplitude-modulated stimuli, the recently introduced Temporal Speech Envelope Tracking (TEMPEST) framework proposed the use of stimuli with a distribution of envelope modulations. The framework aims to assess the brain's capability to process temporal envelopes in different frequency bands using stimuli with speech-like envelope modulations. In this study, we provide a proof-of-concept of the framework using stimuli with modulation frequency bands around the syllable and phoneme rate in natural speech. We evaluated whether the evoked phase-locked neural activity correlates with the speech-weighted modulation transfer function measured using sinusoidal amplitude-modulated stimuli in normal-hearing listeners. Since many studies on modulation processing employ different metrics and comparing their results is difficult, we included different power- and phase-based metrics and investigate how these metrics relate to each other. Results reveal a strong correspondence across listeners between the neural activity evoked by the speech-like stimuli and the activity evoked by the sinusoidal amplitude-modulated stimuli. Furthermore, strong correspondence was also apparent between each metric, facilitating comparisons between studies using different metrics. These findings indicate the potential of the TEMPEST framework to efficiently assess the neural capability to process temporal envelope modulations within a frequency band that is important for speech perception.

Decoding selective auditory attention with EEG using a transformer model.

The human auditory system extracts valid information in noisy environments while ignoring other distractions, relying primarily on auditory attention. Studies have shown that the cerebral cortex responds differently to the sound source locations and that auditory attention is time-varying. In this work, we proposed a data-driven encoder-decoder architecture model for auditory attention detection (AAD), denoted as AAD-transformer. The model contains temporal self-attention and channel attention modules and could reconstruct the speech envelope by dynamically assigning weights according to the temporal self-attention and channel attention mechanisms of electroencephalogram (EEG). In addition, the model is conducted based on data-driven without additional preprocessing steps. The proposed model was validated using a binaural listening dataset, in which the speech stimulus was Mandarin, and compared with other models. The results showed that the decoding accuracy of the AAD-transformer in the 0.15-second decoding time window was 76.35%, which was much higher than the accuracy of the linear model using temporal response function in the 3-second decoding time window (increased by 16.27%). This work provides a novel auditory attention detection method, and the data-driven characteristic makes it convenient for neural-steered hearing devices, especially those who speak tonal languages.