Intelligibility Of Target Speech Research Articles

Effects of spatial configuration and fundamental frequency on speech intelligibility in multiple-talker conditions in the ipsilateral horizontal plane and median planea).

Spatial separation and fundamental frequency (F0) separation are effective cues for improving the intelligibility of target speech in multi-talker scenarios. Previous studies predominantly focused on spatial configurations within the frontal hemifield, overlooking the ipsilateral side and the entire median plane, where localization confusion often occurs. This study investigated the impact of spatial and F0 separation on intelligibility under the above-mentioned underexplored spatial configurations. The speech reception thresholds were measured through three experiments for scenarios involving two to four talkers, either in the ipsilateral horizontal plane or in the entire median plane, utilizing monotonized speech with varying F0s as stimuli. The results revealed that spatial separation in symmetrical positions (front-back symmetry in the ipsilateral horizontal plane or front-back, up-down symmetry in the median plane) contributes positively to intelligibility. Both target direction and relative target-masker separation influence the masking release attributed to spatial separation. As the number of talkers exceeds two, the masking release from spatial separation diminishes. Nevertheless, F0 separation remains as a remarkably effective cue and could even facilitate spatial separation in improving intelligibility. Further analysis indicated that current intelligibility models encounter difficulties in accurately predicting intelligibility in scenarios explored in this study.

The Role of Average Fundamental Frequency Difference on the Intelligibility of Real-Life Competing Sentences.

The average fundamental frequency separation (∆fo) between two competing voices has been shown to provide an important cue for target-speech intelligibility. However, some of the previous investigations used speech materials with linguistic properties and fo characteristics that may not be typical of realistic acoustic scenarios. This study investigated to what extent the effect of ∆fo generalizes to more real-life speech. Real-life sentences and a well-controlled method for manipulating the acoustic stimuli were employed. Fifteen young normal-hearing native Danish listeners were tested in a two-competing-voices sentence recognition task at several target-to-masker ratios (TMRs) and ∆fos. Compared to previous studies that addressed the same experimental scenario with less realistic speech materials, the present results showed only a moderate effect of ∆fo at negative TMRs and a negligible effect at positive TMRs. An analysis of the employed stimuli showed that a large ∆fo effect on the target speech intelligibility is only observed when the competing sentences have highly synchronous fo trajectories, which is typical of the artificial speech materials employed in previous studies. Overall, the present results suggest a relatively small effect of ∆fo on the intelligibility of real-life speech, as compared to previously employed artificial speech, in two-competing-sentences conditions.

The effect of explicit and implicit voice training on speech-on-speech intelligibility and listening effort

Listening to familiar voices might improve intelligibility of target speech in multiple-talker situations. In addition to personally familiar voices, voice training—through implicit exposure or explicit learning of a previously unheard voice—can improve speech intelligibility. However, there is no consensus on which method is more effective. We investigated the effect of explicit and implicit voice training on speech-on-speech perception and listening effort (pupillometry), among normal hearing listeners. There was no significant difference in speech intelligibility performance between trained and untrained voices, with either voice training method. However, an interaction between voice training method and target-to-masker ratio (TMR) on accuracy scores showed that the benefit of increased TMR on accuracy scores was significantly larger when voice training was implicit compared to explicit. Pupillometry results showed that explicit voice training significantly reduced listening effort for most disadvantageous TMR (−6dB), but not for more advantageous TMRs (0 and + 6 dB), and not for implicitly trained voices. Our results may imply that explicit voice training is more effective than implicit voice training in reducing listening effort. Still, implicit voice training, perhaps via procedural learning, might be more beneficial on the performance level when TMR becomes more advantageous.

The role of reliable interaural time difference cues in ambiguous binaural signals for the intelligibility of multitalker speech.

When listening to speech in the presence of concurrent talkers, listeners can benefit from glimpses that occur as a result of spectro-temporal modulations in the speech signals. These glimpses are characterized by a high local signal-to-noise ratio and allow listeners to collect relatively undistorted and reliable information on target speech features. A series of experiments was designed to measure the spatial advantage for binaurally presented speech when useful interaural time difference (ITD) information was provided only in glimpses of speech signals with otherwise ambiguous ITDs. For interaurally coherent signals, ITD information provided by target glimpses contributed substantially to the spatial advantage, but consistent target ITDs overall appeared to be of minor importance to speech intelligibility. For interaurally incoherent signals, a similarly large contribution of coherent ITD information in glimpses to the spatial advantage was not observed. Rather, target speech intelligibility depended on the interaural coherence of the interfering speech signals. While the previous observation conforms with models of auditory object formation, and the latter is consistent with equalization-cancellation theory modeling the spatial advantage, the two seem to be at odds for the presented set of experiments. A conceptual framework employing different strategies to process the perceptual foreground and background may solve this issue.

The benefit of acoustic beamforming in solving the “cocktail party problem” for persons with acquired aphasia

Acoustic beamforming has been shown to improve identification of target speech in noisy listening environments for individuals with sensorineural hearing loss. This study examined whether beamforming would provide a similar benefit for individuals with aphasia (an acquired neurological language deficit). Persons with aphasia (PWA) are known to exhibit impaired language comprehension abilities; however, most work on language comprehension in PWA has been conducted in quiet settings, and little is known about the impact of competing auditory information on performance. In this study, we measured the intelligibility of target speech masked by other speech or noise for two presentation/microphone conditions: one condition, designated “KEMAR,” provided natural spatial cues via KEMAR impulse responses; the second condition, designated “BEAM,” enhanced the target speech level via a single-channel beamformer. In each condition, subjects heard a target sentence at 0 deg azimuth concurrent with two independent maskers—speech or speech-shaped speech-envelope-modulated noise—from ±60 deg. Threshold target-to-masker ratios were measured adaptively for each subject, with individually-determined modifications in the procedures made for the subjects. Results indicated substantially lower (better) thresholds for the beamformer condition than for KEMAR, providing preliminary evidence that PWA may benefit from the use of acoustic beamforming in complex, multiple-source listening situations.

On the Interaction of Head and Gaze Control With Acoustic Beam Width of a Simulated Beamformer in a Two-Talker Scenario.

Superdirectional acoustic beamforming technology provides a high signal-to-noise ratio, but potential speech intelligibility benefits to hearing aid users are limited by the way the users move their heads. Steering the beamformer using eye gaze instead of head orientation could mitigate this problem. This study investigated the intelligibility of target speech with a dynamically changing direction when heard through gaze-controlled (GAZE) or head-controlled (HEAD) superdirectional simulated beamformers. The beamformer provided frequency-independent noise attenuation of either 8 dB (WIDE [moderately directional]) or 12 dB (NARROW [highly directional]) relative to no beamformer referred as the OMNI (omni-directional) condition. Before the main experiment, signal-to-noise ratios were normalized for each participant and each beam width condition to yield equal percentage of correct performance in a reference condition. Hence, results are presented as normalized speech intelligibility (NSI). In an ongoing presentation, the participants (n = 17), of varying degree of hearing loss, heard single-word targets every 1.5 s coming from either left (−30°) or right (+30°) presented in continuous, spatially distributed, speech-shaped noise. When the target was static, NSI was better in the GAZE than in the HEAD condition, but only when the beam was NARROW. When the target switched location without warning, NSI performance dropped. In this case, the WIDE HEAD condition provided the best average NSI performance, because some participants tended to orient their head in between the targets, allowing them to hear out the target regardless of location. The difference in NSI between GAZE and HEAD conditions for individual participants was related to the observed head-orientation strategy, which varied widely across participants.

Use of non-individualized head-related transfer functions to measure spatial release from masking in children with normal hearing

Spatial hearing studies with children have typically been conducted using loudspeakers in laboratories. However, loudspeaker arrays are rare in clinics due to high cost and technical set-up requirements. The use of virtual auditory space (VAS) with non-individualized head-related transfer functions (HRTFs) can increase the feasibility of assessing spatial hearing abilities in clinical settings. A novel paradigm for measuring spatial release from masking (SRM) was developed using non-individualized HRTFs. This paradigm measures the minimum angular separation needed between target and masker to achieve a 20% increase in target speech intelligibility. First, the 50% speech reception threshold (SRT) was measured with target and masker co-located to one side. Then, the masker position was adaptively changed to achieve 70.7% intelligibility while maintaining the signal-to-noise ratio at the level of the co-located SRT. To verify the use of non-individualized HRTFs, normal-hearing children were tested (1) using a loudspeaker array and (2) in headphone-based VAS created using KEMAR HRTFs measured in the same setup as (1). Preliminary results showed that co-located SRTs and target-masker angle separation to achieve a 20% SRM were similar in loudspeaker array and in headphone-based VAS. This suggests that non-individualized HRTFs might be used in an SRM task for clinical testing.

Listeners Experience Linguistic Masking Release in Noise-Vocoded Speech-in-Speech Recognition.

The purpose of this study was to evaluate whether listeners with normal hearing perceiving noise-vocoded speech-in-speech demonstrate better intelligibility of target speech when the background speech was mismatched in language (linguistic release from masking [LRM]) and/or location (spatial release from masking [SRM]) relative to the target. We also assessed whether the spectral resolution of the noise-vocoded stimuli affected the presence of LRM and SRM under these conditions. In Experiment 1, a mixed factorial design was used to simultaneously manipulate the masker language (within-subject, English vs. Dutch), the simulated masker location (within-subject, right, center, left), and the spectral resolution (between-subjects, 6 vs. 12 channels) of noise-vocoded target-masker combinations presented at +25 dB signal-to-noise ratio (SNR). In Experiment 2, the study was repeated using a spectral resolution of 12 channels at +15 dB SNR. In both experiments, listeners' intelligibility of noise-vocoded targets was better when the background masker was Dutch, demonstrating reliable LRM in all conditions. The pattern of results in Experiment 1 was not reliably different across the 6- and 12-channel noise-vocoded speech. Finally, a reliable spatial benefit (SRM) was detected only in the more challenging SNR condition (Experiment 2). The current study is the first to report a clear LRM benefit in noise-vocoded speech-in-speech recognition. Our results indicate that this benefit is available even under spectrally degraded conditions and that it may augment the benefit due to spatial separation of target speech and competing backgrounds.

An account for the spatial advantage in multitalker situations based on glimpses

Spectro-temporal regions with a high local signal-to-noise ratio (SNR), so-called glimpses, play a vital role in the intelligibility of target speech against fluctuating interferers (e.g., concurrent speech signals). These glimpses provide access to reliable information on local signal properties. In a situation with spatially separated speech sources, a spatial advantage relative to a situation with collocated sources can be observed. This advantage is generally conceived to be composed of a monaural contribution due to better-ear listening, and a binaural contribution due to either binaural unmasking or segregation supported by spatial cues. A previous study [Schoenmaker and van de Par (2016), Adv. Exp. Med. Biol. 894, 73-81] provided evidence against the use of binaural unmasking and in favor of spatial segregation based on spatial cues extracted from glimpses. New data suggest that the better-ear contribution relies on the amount of target speech in glimpses, rather than the global SNR of the masked t...

Linguistic masking release in older adults

Past studies of speech-on-speech masking in children and young adults (YA) indicate that intelligibility of target speech can improve when target and masker speech are in different languages. We investigated whether such linguistic release from masking is obtained in older adults (OA) with age-typical hearing abilities. All participants were asked to recognize English sentences in the presence of two-talker maskers spoken in either English or Spanish presented at four different Signal-to-Noise ratios (SNR) to each group. Differences in energetic masking between the masker languages were minimized. Overall sentence recognition accuracy was greater for YA participants. However, both groups equally benefited from a linguistic mismatch in the target-masker language with a significant masking release for the Spanish language masker. The magnitude of masking release increased as SNR decreased, ranging across SNR conditions from 1 to 27 percentage points. Age, hearing-in-noise ability and hearing sensitivity of OA listeners did not correlate with masking release. Results confirm previous findings of masking release associated with a linguistic mismatch between target and masker speech, and indicate that in speech-on-speech masking older listeners can improve speech intelligibility by utilizing nonenergetic linguistic differences between the target and masker speech.

A study on the effect by adding reverberation to speech-like masking sound

Problems arising from the acoustical privacy point of view in public spaces have been known to be an issue. The lack of acoustical privacy has been known to affect the human's health both physically and psychologically, thus keeping the acoustical privacy in public spaces will significantly reduce social loss. Masking is the most commonly practically used technique to make a target speech unintelligible to the unintended listeners without needing to install any physical structures. Time-reversed speech has been known to effectively mask information for speech privacy applications; however, the annoyance and distraction caused by the time-reversed speech is known to be higher than other masking sound. This study explores a solution to compromise the suggested problem by adding a reverberant effect to a time-reversed speech. Subjective listening tests have been conducted to measure the intelligibility of target speech, annoyance and distraction caused by the masking sound. The experimental results suggest that adding artificial reverberation to a time-reversed speech has a significant effect to reduce the annoyance level while maintaining the masking effectiveness of the original time-reversed speech. A trend was also observed that the addition of artificial reverberation could reduce the level of distraction caused by the masking sound.

Effect of adding artificial reverberation to speech-like masking sound

Time-reversed speech has been known to effectively mask information for speech privacy applications. However, the annoyance and distraction caused by the time-reversed speech-like masking sound is higher than other masking sound. This study investigates the effects of adding artificial reverberation to the time-reversed speech. Subjective listening tests have been conducted to measure the intelligibility of target speech, annoyance and distraction caused by the masking sound. The experimental results suggest that adding artificial reverberation to a speech-like masking sound has a significant effect to reduce the annoyance level while maintaining the masking effectiveness of the original masking sound. A trend was also observed that the addition of artificial reverberation could reduce the level of distraction caused by the masking sound.

Effects of a consistent target or masker voice on target speech intelligibility in two- and three-talker mixtures.

When the spatial location or identity of a sound is held constant, it is not masked as effectively by competing sounds. This suggests that experience with a particular voice over time might facilitate perceptual organization in multitalker environments. The current study examines whether listeners benefit from experience with a voice only when it is the target, or also when it is a masker, using diotic presentation and a closed-set task (coordinate response measure). A reliable interaction was observed such that, in two-talker mixtures, consistency of masker or target voice over 3-7 trials significantly benefited target recognition performance, whereas in three-talker mixtures, target, but not masker, consistency was beneficial. Overall, this work suggests that voice consistency improves intelligibility, although somewhat differently when two talkers, compared to three talkers, are present, suggesting that consistent-voice information facilitates intelligibility in at least two different ways. Listeners can use a template-matching strategy to extract a known voice from a mixture when it is the target. However, consistent-voice information facilitates segregation only when two, but not three, talkers are present.

The effect of informational and energetic masking on foreign-accent adaptation

Speech understanding in noisy environments can be compromised through energetic (EM) and informational (IM) masking. EM reduces intelligibility of target speech through spectro-temporal overlap with the masker at the auditory periphery level. IM refers to the higher-level interference, such as competing attention, linguistic interference, and increased cognitive load. The present study examined the effects of EM and IM on foreign-accent adaptation. Native English listeners heard blocks of sentences produced by native-accented (NA) or foreign-accented (FA) talkers (Korean, Spanish) mixed with speech-shaped noise (SSN) or two-talker, native-accented babble and responded to a visual probe. Preliminary results show that listeners were more accurate and faster in babble compared to SSN. The more successful FA adaptation in the babble condition may be related to the presence of dips in the masker energy or the ability of listeners to successfully separate FA target from the NA background speech. Additionally, the acoustic properties of the target sentences, such as speaking rate and pausing, are examined to understand their effect on FA adaptation in each of the two noise conditions. These findings suggest that IM was less disruptive than EM for FA adaptation even though listeners were processing FA speech with an increased cognitive load.

Quantifying the better ear advantage in the presence of interfering speech

In cocktail party listening with spatially separated speech sources, better ear listening is known to make a major contribution to speech intelligibility. The better ear is generally defined as the ear that receives the highest signal-to-noise ratio (SNR). Usually, this SNR is calculated based on the total length of the signal. However, this seems inappropriate when speech signals are involved since these are highly modulated both in the time and frequency domain. On a perceptual level, modulated maskers give rise to a higher target speech intelligibility than their unmodulated counterparts through the presence of glimpses. A simple measure to quantify the better ear advantage while taking these spectrotemporal fluctuations into account is introduced. In a headphone experiment, three simultaneous sequences of vowel-consonant-vowel utterances were presented at a fixed target-to-masker ratio. The stimuli were rendered with head-related transfer functions and contrasted against stimuli that did not contain any interaural level differences (ILDs) and, as a consequence, allowed no better ear listening. Using the proposed metric, we are able to explain differences in intelligibility for these speech-in-speech mixtures that would remain unexplained by the conventional SNR both for stimuli with and without ILDs.

Improvement of intelligibility of ideal binary-masked noisy speech by adding background noise

When a target-speech/masker mixture is processed with the signal-separation technique, ideal binary mask (IBM), intelligibility of target speech is remarkably improved in both normal-hearing listeners and hearing-impaired listeners. Intelligibility of speech can also be improved by filling in speech gaps with un-modulated broadband noise. This study investigated whether intelligibility of target speech in the IBM-treated target-speech/masker mixture can be further improved by adding a broadband-noise background. The results of this study show that following the IBM manipulation, which remarkably released target speech from speech-spectrum noise, foreign-speech, or native-speech masking (experiment 1), adding a broadband-noise background with the signal-to-noise ratio no less than 4 dB significantly improved intelligibility of target speech when the masker was either noise (experiment 2) or speech (experiment 3). The results suggest that since adding the noise background shallows the areas of silence in the time-frequency domain of the IBM-treated target-speech/masker mixture, the abruption of transient changes in the mixture is smoothed and the perceived continuity of target-speech components becomes enhanced, leading to improved target-speech intelligibility. The findings are useful for advancing computational auditory scene analysis, hearing-aid/cochlear-implant designs, and understanding of speech perception under "cocktail-party" conditions.

Intelligibility comparison of Japanese speech with competing noise spatialized in real and virtual acoustic environments

We investigated the effect of a competing noise source on the intelligibility of target speech in various acoustic environments. To spatialize these sources in virtual acoustic space, we used head-related transfer functions (HRTFs) measured using the KEMAR dummy head, as well as HRTFs measured individually for each subject. We also compared the intelligibility when these sound sources were generated from loudspeakers located at the target and competing source positions in real acoustic space. The speech intelligibility of the target speech was evaluated using the Japanese diagnostic rhyme test (DRT). The target speech was placed directly in front of the listener, and a single competing source was placed on the same horizontal plane at various azimuths and distances. Individual HRTFs showed slightly better intelligibility than those with KEMAR HRTFs, but the difference was small in most cases. Intelligibility in real acoustic space still outperformed both types of HRTFs, especially when the competing sound source was closer to the listener than the target speech and the competing source was located at 180°. However, this difference was small in most of the other competing source locations tested.

How visual cues help us understand speech in a complex environment.

At a cocktail party, visual cues may help a listener by showing them where or when to direct attention, what acoustic modulations a target utterance contains, and/or what articulatory gestures produce the target. Here, we investigated target speech intelligibility while varying the visual cues available in a complex, confusing auditory scene. In all cases, subjects listened for a target utterance in the presence of multiple masker utterances with similar grammatical structure spoken by the same talker. The timing and direction of the target (and maskers) varied randomly, increasing the uncertainty about where and when to focus auditory attention. The number of correctly reported target key words measured performance. Performance tended to improve as the amount of visual information increased, particularly when masker phrases came from the direction of the target. Performance was generally similar whether listeners saw full videos of the target talker from the correct direction or only a static image of the talker at the right time in the correct direction. However, temporal information about where and when the target occurred improved performance over knowing only target location. Results suggest that in these scenes, visual cues aid target understanding by indicating where and roughly when to direct attention.

Transient auditory storage of acoustic details is associated with release of speech from informational masking in reverberant conditions.

Perceptual integration of the sound directly emanating from the source with reflections needs both temporal storage and correlation computation of acoustic details. We examined whether the temporal storage is frequency dependent and associated with speech unmasking. In Experiment 1, a break in correlation (BIC) between interaurally correlated wideband or narrowband noises was detectable even when an interaural interval (IAI) was introduced. The longest IAI, which varied markedly across participants, could be up to about 20 ms for wideband noise and decreased as the center frequency was increased for narrowband noises. In Experiment 2, when the interval between target speech and its single-reflection simulation (intertarget interval [ITI]) was reduced from 64 to 0 ms, intelligibility of target speech was markedly improved under speech-masking but not noise-masking conditions. The longest effective ITI correlated with the longest IAI for detecting the BIC only in the low-frequency (<or=400 Hz) narrowband noise. Thus the ability to temporally store fine details contributes to perceptual integration of correlated leading and lagging sounds, which in turn, contributes to releasing speech from informational masking in noisy, reverberant environments.

The ability of temporally integrating acoustic waveforms is associated with release of speech from informational masking under reverberant conditions

Perceptual integration of the sound wave directly emanating from the source with reflections of the source needs both bridging temporal gaps and calculating correlations between sound waves. In this study, we examined whether the temporal integration of source/reflection signals is frequency dependent and associated with speech unmasking under simulated reverberant conditions. In Experiment 1, a break in correlation (BIC) between correlated wideband or narrowband noises at the two ears was detectable even when an interaural interval (IAI) was introduced. The longest IAI varied markedly across listeners. In wideband noise, it could be up to 21 ms; in narrowband noise, it decreased as the center frequency was increased. In Experiment 2, when the interval between target speech and its single‐reflection simulation (inter‐target interval, ITI) was reduced from 64 to 0 ms, intelligibility of target speech was markedly improved under the speech‐masking condition but not the noise‐masking condition. The longest effective ITI under the speech‐masking condition significantly correlated with the longest IAI for detecting the BIC in low‐frequency (< or = 400 Hz) narrowband noises. Thus the frequency‐dependent temporal integration of sound‐waveform signals is critical for releasing speech from informational masking in reverberant environments. Supported by the National Natural Science Foundation of China.