Speech-modulated Noise Research Articles

Can dual-carrier processing restore masking release in vocoded speech?

Speech processed to replace the original temporal fine structure (TFS) with tones or noise carriers (vocoder processing) are generally less intelligible than natural or unprocessed speech, especially if a background noise is present. Moreover, the poorer intelligibility associated with vocoder processing is typically larger if the background fluctuates over time. This deleterious effect of vocoder processing has led to the postulate that TFS cues play a critical role when listening into the dips in the background. Recently, we have proposed a technique to reintroduce synthetic TFS cues in vocoded speech using one carrier for the target and one carrier for the background. This “dual-carrier” approach allows sentence intelligibility with a speech masker to reach a level almost comparable to that of natural speech. The goal of the present study was to investigate the extent to which dual-carrier processing generally improves speech recognition in various noises or if it truly compensates for the loss of TFS cues, therefore engendering masking release, as does natural speech. Results comparing masking release for three processing conditions (single-carrier, dual-carrier, and natural speech) in five backgrounds (speech-shaped noise, speech-modulated noise, and 1, 2, or 8 talkers) will be discussed.

Contribution of near- and suprathreshold hearing deficits to speech recognition

This study addresses the contribution of audibility and suprathreshold deficits to speech recognition in quiet, stationary, and speech-modulated noise. The relative importance of these deficits for different frequency ranges of hearing was determined by measuring speech recognition using broadband speech and noise, but also high- and low-pass filtered versions of these stimuli. Audibility was assessed using pure tone thresholds and speech reception thresholds (SRT) in quiet. Three listener groups were considered to parse out hearing deficits: young (yNH) and older normal-hearing (oNH) listeners and hearing-impaired (HI) listeners. oNH and HI listeners performed equally poor in stationary noise where yNH listeners showed significantly better SRTs in both noise conditions. Since audibility did not correlate with SRTs in stationary noise but with modulated noise, and SRTs in stationary and modulated noise correlated with each other, a common mechanism (not audibility) must affect speech recognition in noise. Furthermore, our results suggest that performance in the broadband condition cannot be explained by the low frequency mechanisms since all groups performed similar in the low-pas conditions whereas the oNH and oHI group performed equally bad in high-pas condition. Instead, we found evidence for a contribution of high frequency envelope coding mechanisms to broadband performance

The Contribution of Auditory and Cognitive Factors to Intelligibility of Words and Sentences in Noise.

Understanding the causes for speech-in-noise (SiN) perception difficulties is complex, and is made even more difficult by the fact that listening situations can vary widely in target and background sounds. While there is general agreement that both auditory and cognitive factors are important, their exact relationship to SiN perception across various listening situations remains unclear. This study manipulated the characteristics of the listening situation in two ways: first, target stimuli were either isolated words, or words heard in the context of low- (LP) and high-predictability (HP) sentences; second, the background sound, speech-modulated noise, was presented at two signal-to-noise ratios. Speech intelligibility was measured for 30 older listeners (aged 62-84) with age-normal hearing and related to individual differences in cognition (working memory, inhibition and linguistic skills) and hearing (PTA(0.25-8 kHz) and temporal processing). The results showed that while the effect of hearing thresholds on intelligibility was rather uniform, the influence of cognitive abilities was more specific to a certain listening situation. By revealing a complex picture of relationships between intelligibility and cognition, these results may help us understand some of the inconsistencies in the literature as regards cognitive contributions to speech perception.

Application of a short-time version of the Equalization-Cancellation model to speech intelligibility experiments with speech maskers.

A short-time-processing version of the Equalization-Cancellation (EC) model of binaural processing is described and applied to speech intelligibility tasks in the presence of multiple maskers, including multiple speech maskers. This short-time EC model, called the STEC model, extends the model described by Wan et al. [J. Acoust. Soc. Am. 128, 3678-3690 (2010)] to allow the EC model's equalization parameters τ and α to be adjusted as a function of time, resulting in improved masker cancellation when the dominant masker location varies in time. Using the Speech Intelligibility Index, the STEC model is applied to speech intelligibility with maskers that vary in number, type, and spatial arrangements. Most notably, when maskers are located on opposite sides of the target, this STEC model predicts improved thresholds when the maskers are modulated independently with speech-envelope modulators; this includes the most relevant case of independent speech maskers. The STEC model describes the spatial dependence of the speech reception threshold with speech maskers better than the steady-state model. Predictions are also improved for independently speech-modulated noise maskers but are poorer for reversed-speech maskers. In general, short-term processing is useful, but much remains to be done in the complex task of understanding speech in speech maskers.

Sentence intelligibility during segmental interruption and masking by speech-modulated noise

Amplitude modulation from a competing talker can interfere with a listener's ability to process informative speech cues. The current study investigated how noise modulated by the wideband temporal envelope of a single competing talker impacts the contribution of consonants and vowels to the intelligibility of the target sentence. High variability speech materials were used, including talkers from different dialect regions. Young normal-hearing, older normal-hearing, and older hearing-impaired listeners completed speech recognition tests. All listeners received spectrally shaped speech matched to their individual audiometric thresholds to ensure sufficient audibility. Preliminary results demonstrated similar performance among the listener groups. When the modulated masker was interrupted, performance in consonant and vowel conditions was similar. However, sentence intelligibility in continuous single-talker-modulated noise was higher when vowels in the target sentence were preserved, as compared to consonants. Thus, masker continuity may facilitate source segregation for vowels more than for consonants. A second experiment that varied masker modulation rate for younger adults with normal hearing showed that modulation rate had more effect on processing cues conveyed by consonants than by vowels. Implications for hearing-impaired and older adults will be discussed. [Work supported by grants from NIH/NIDCD and ASHA.]

Recognition of accented and unaccented speech in different maskers by younger and older listeners

This investigation examined the effect of accent of target talkers and background speech maskers on listeners' ability to use cues to separate speech from noise. Differences in accent may create a disparity in the relative timing between signal and background, and such timing cues may be used to separate the target talker from the background speech masker. However, the use of this cue could be reduced for older listeners with temporal processing deficits, especially those with hearing loss. Participants were younger and older listeners with normal hearing and older listeners with hearing loss. Stimuli were IEEE sentences recorded in English by male native speakers of English and Spanish. These sentences were presented in different maskers that included speech-modulated noise and background babbles varying in talker gender and accent. Signal-to-noise ratios corresponding to 50% correct performance were measured. Results indicate that a pronounced Spanish accent limits a listener's ability to take advantage of cues to speech segregation and that a difference in accentedness between the target talker and background masker may be a useful cue for speech segregation. Older hearing-impaired listeners performed poorly in all conditions with the accented talkers.

The fluctuating-masker benefit for normal-hearing and hearing-impaired listeners with equal audibility at a fixed signal-to-noise ratio

While normal-hearing (NH) listeners demonstrate better speech intelligibility for fluctuating-masker than for stationary-noise conditions, hearing-impaired (HI) listeners generally show little or no fluctuating-masker benefit (FMB). This result has been interpreted in terms of suprathreshold deficits (e.g., reduced spectral or temporal resolution or distorted stream-segregation cues) that limit “dip-listening.” However, reduced FMB for HI listeners might instead be attributable to audibility limitations or to differences between the signal-to-noise ratios (SNRs) at which NH and HI listeners are tested. This study examined this issue by equalizing stationary-noise performance to allow measurements at a common SNR, equalizing audibility, and presenting identical signals to pairs of NH and HI listeners. Audibility was equalized using linear gain, low-pass filtering (4 kHz) and intensity filtering to remove speech-signal elements below the HI audiometric threshold. Nonsense-syllable identification performance in stationary noise was equalized by adjusting the response set size. Stationary-noise trials (adapting set size) were interleaved with fluctuating-masker trials (adapting SNR), ensuring stable stationary-noise performance throughout the test. Fluctuating maskers included low- and high-rate modulated noise, speech-modulated noise, and an interfering-talker condition. Results were assessed to determine whether and under which conditions the HI listeners demonstrated reduced FMB not attributable to SNR or audibility effects.

Effects of spectral smearing and temporal fine-structure distortion on the fluctuating-masker benefit for speech at a fixed signal-to-noise ratio

Normal-hearing listeners receive less benefit from momentary dips in the level of a fluctuating masker for speech processed to degrade spectral detail or temporal fine structure (TFS) than for unprocessed speech. This has been interpreted as evidence that the magnitude of the fluctuating-masker benefit (FMB) reflects the ability to resolve spectral detail and TFS. However, the FMB for degraded speech is typically measured at a higher signal-to-noise ratio (SNR) to yield performance similar to normal speech for the baseline (stationary-noise) condition. Because the FMB decreases with increasing SNR, this SNR difference might account for the reduction in FMB for degraded speech. In this study, the FMB for unprocessed and processed (TFS-removed or spectrally smeared) speech was measured in a paradigm that adjusts word-set size, rather than SNR, to equate stationary-noise performance across processing conditions. Compared at the same SNR and percent-correct level (but with different set sizes), processed and unprocessed stimuli yielded a similar FMB for four different fluctuating maskers (speech-modulated noise, one opposite-gender interfering talker, two same-gender interfering talkers, and 16-Hz interrupted noise). These results suggest that, for these maskers, spectral or TFS distortions do not directly impair the ability to benefit from momentary dips in masker level.

Spectral and temporal changes to speech produced in the presence of energetic and informational maskers

Talkers change the way they speak in noisy conditions. For energetic maskers, speech production changes are relatively well-understood, but less is known about how informational maskers such as competing speech affect speech production. The current study examines the effect of energetic and informational maskers on speech production by talkers speaking alone or in pairs. Talkers produced speech in quiet and in backgrounds of speech-shaped noise, speech-modulated noise, and competing speech. Relative to quiet, speech output level and fundamental frequency increased and spectral tilt flattened in proportion to the energetic masking capacity of the background. In response to modulated backgrounds, talkers were able to reduce substantially the degree of temporal overlap with the noise, with greater reduction for the competing speech background. Reduction in foreground-background overlap can be expected to lead to a release from both energetic and informational masking for listeners. Passive changes in speech rate, mean pause length or pause distribution cannot explain the overlap reduction, which appears instead to result from a purposeful process of listening while speaking. Talkers appear to monitor the background and exploit upcoming pauses, a strategy which is particularly effective for backgrounds containing intelligible speech.

Language-independent processing in speech perception: Identification of English intervocalic consonants by speakers of eight European languages

Processing speech in a non-native language requires listeners to cope with influences from their first language and to overcome the effects of limited exposure and experience. These factors may be particularly important when listening in adverse conditions. However, native listeners also suffer in noise, and the intelligibility of speech in noise clearly depends on factors which are independent of a listener’s first language. The current study explored the issue of language-independence by comparing the responses of eight listener groups differing in native language when confronted with the task of identifying English intervocalic consonants in three masker backgrounds, viz. stationary speech-shaped noise, temporally-modulated speech-shaped noise and competing English speech. The study analysed the effects of (i) noise type, (ii) speaker, (iii) vowel context, (iv) consonant, (v) phonetic feature classes, (vi) stress position, (vii) gender and (viii) stimulus onset relative to noise onset. A significant degree of similarity in the response to many of these factors was evident across all eight language groups, suggesting that acoustic and auditory considerations play a large rôle in determining intelligibility. Language-specific influences were observed in the rankings of individual consonants and in the masking effect of competing speech relative to speech-modulated noise.

The fluctuating-masker benefit for word identification with spectral or fine-structure distortions.

Normal-hearing (NH) listeners presented with speech processed to reduce spectral detail or remove temporal fine-structure (TFS) information show a reduced benefit from momentary dips in the level of a fluctuating masker. This has been interpreted as evidence that related deficits may underlie the reduced fluctuating-masker benefit (FMB) observed for the hearing impaired. However, the reduced FMB for processed stimuli could be attributable to the greater signal-to-noise ratio (SNR) required to yield performance levels equivalent to the unprocessed case for unmodulated noise (the reference condition). NH listeners were tested in the identification of isolated words presented in stationary-noise, interfering-talker, and speech-modulated noise maskers. Different response-set sizes were used to offset performance differences between processed and unprocessed conditions. A validation experiment showed no effect of set size on the FMB when compared at a common stationary-noise SNR. A second experiment compared the FMB for unprocessed, spectrally smeared, and noise vocoded speech. When compared at the same SNR and percent-correct level (but different set sizes), processed and unprocessed stimuli yielded similar FMB. This suggests that for the conditions tested here, spectral or TFS distortions do not directly impair the ability to listen in the gaps of a fluctuating masker. [Sponsored by the Oticon Foundation.]

Auditory and auditory-visual intelligibility of speech in fluctuating maskers for normal-hearing and hearing-impaired listeners

Speech intelligibility for audio-alone and audiovisual (AV) sentences was estimated as a function of signal-to-noise ratio (SNR) for a female target talker presented in a stationary noise, an interfering male talker, or a speech-modulated noise background, for eight hearing-impaired (HI) and five normal-hearing (NH) listeners. At the 50% keywords-correct performance level, HI listeners showed 7-12 dB less fluctuating-masker benefit (FMB) than NH listeners, consistent with previous results. Both groups showed significantly more FMB under AV than audio-alone conditions. When compared at the same stationary-noise SNR, FMB differences between listener groups and modalities were substantially smaller, suggesting that most of the FMB differences at the 50% performance level may reflect a SNR dependence of the FMB. Still, 1-5 dB of the FMB difference between listener groups remained, indicating a possible role for reduced audibility, limited spectral or temporal resolution, or an inability to use auditory source-segregation cues, in directly limiting the ability to listen in the dips of a fluctuating masker. A modified version of the extended speech-intelligibility index that predicts a larger FMB at less favorable SNRs accounted for most of the FMB differences between listener groups and modalities. Overall, these data suggest that HI listeners retain more of an ability to listen in the dips of a fluctuating masker than previously thought. Instead, the fluctuating-masker difficulties exhibited by HI listeners may derive from the reduced FMB associated with the more favorable SNRs they require to identify a reasonable proportion of the target speech.

The neural processing of masked speech: Evidence for different mechanisms in the left and right temporal lobes

It has been previously demonstrated that extensive activation in the dorsolateral temporal lobes associated with masking a speech target with a speech masker, consistent with the hypothesis that competition for central auditory processes is an important factor in informational masking. Here, masking from speech and two additional maskers derived from the original speech were investigated. One of these is spectrally rotated speech, which is unintelligible and has a similar (inverted) spectrotemporal profile to speech. The authors also controlled for the possibility of "glimpsing" of the target signal during modulated masking sounds by using speech-modulated noise as a masker in a baseline condition. Functional imaging results reveal that masking speech with speech leads to bilateral superior temporal gyrus (STG) activation relative to a speech-in-noise baseline, while masking speech with spectrally rotated speech leads solely to right STG activation relative to the baseline. This result is discussed in terms of hemispheric asymmetries for speech perception, and interpreted as showing that masking effects can arise through two parallel neural systems, in the left and right temporal lobes. This has implications for the competition for resources caused by speech and rotated speech maskers, and may illuminate some of the mechanisms involved in informational masking.

Masking release for low- and high-pass-filtered speech in the presence of noise and single-talker interference

Speech intelligibility was measured for sentences presented in spectrally matched steady noise, single-talker interference, or speech-modulated noise. The stimuli were unfiltered or were low-pass (LP) (1200 Hz cutoff) or high-pass (HP) (1500 Hz cutoff) filtered. The cutoff frequencies were selected to produce equal performance in both LP and HP conditions in steady noise and to limit access to the temporal fine structure of resolved harmonics in the HP conditions. Masking release, or the improvement in performance between the steady noise and single-talker interference, was substantial with no filtering. Under LP and HP filtering, masking release was roughly equal but was much less than in unfiltered conditions. When the average F0 of the interferer was shifted lower than that of the target, similar increases in masking release were observed under LP and HP filtering. Similar LP and HP results were also obtained for the speech-modulated-noise masker. The findings are not consistent with the idea that pitch conveyed by the temporal fine structure of low-order harmonics plays a crucial role in masking release. Instead, any reduction in speech redundancy, or manipulation that increases the target-to-masker ratio necessary for intelligibility to beyond around 0 dB, may result in reduced masking release.

Voice recognition and the posterior cingulate: An fMRI study of prosopagnosia

Voices, in addition to faces, enable person identification. Voice recognition has been shown to evoke a distributed network of brain regions that includes, in addition to the superior temporal sulcus (STS), the anterior temporal pole, fusiform face area (FFA), and posterior cingulate gyrus (pCG). Here we report an individual (MS) with acquired prosopagnosia who, despite bilateral damage to much of this network, demonstrates the ability to distinguish voices of several well-known acquaintances from voices of people that he has never heard before. Functional magnetic resonance imaging (fMRI) revealed that, relative to speech-modulated noise, voices rated as familiar and unfamiliar by MS elicited enhanced haemodynamic activity in the left angular gyrus, left posterior STS, and posterior midline brain regions, including the retrosplenial cortex and the dorsal pCG. More interestingly, relative to noise and unfamiliar voices, the familiar voices elicited greater haemodynamic activity in the left angular gyrus and medial parietal regions including the dorsal pCG and precuneus. The findings are consistent with theories implicating the pCG in recognizing people who are personally familiar, and furthermore suggest that the pCG region of the voice identification network is able to make functional contributions to voice recognition even though other areas of the network, namely the anterior temporal poles, FFA, and the right parietal lobe, may be compromised.

Influences of auditory object formation on phonemic restoration

In phonemic restoration, intelligibility of interrupted speech is enhanced when noise fills the speech gaps. When the broadband envelope of missing speech amplitude modulates the intervening noise, intelligibility is even better. However, this phenomenon represents a perceptual failure: The amplitude modulation, a noise feature, is misattributed to the speech. Experiments explored whether object formation influences how information in the speech gaps is perceptually allocated. Experiment 1 replicates the finding that intelligibility is enhanced when speech-modulated noise rather than unmodulated noise is presented in the gaps. In Experiment 2, interrupted speech was presented diotically, but intervening noises were presented either diotically or with an interaural time difference leading in the right ear, causing the noises to be perceived to the side of the listener. When speech-modulated noise and speech are perceived from different directions, intelligibility is no longer enhanced by the modulation. However, perceived location has no effect for unmodulated noise, which contains no speech-derived information. Results suggest that enhancing object formation reduces misallocation of acoustic features across objects, and demonstrate that our ability to understand noisy speech depends on a cascade of interacting processes, including glimpsing sensory inputs, grouping sensory inputs into objects, and resolving ambiguity through top-down knowledge.

Directional Benefit in the Presence of Speech and Speechlike Maskers

Recent research suggests that omnidirectional hearing aids are relatively ineffective at improving speech understanding in everyday conversational speech settings when the background noise contains both energetic and informational masking components. Energetic masking refers to situations where the peripheral (or neural) activity of the target is less than that of the masker, thus making the target inaudible. In contrast, informational masking effects, in this paper, refer to additional masking effects that are not energetic in nature. The current study evaluated the benefits of directional technology in the presence of background noises that contained both energetic and informational masking components. Aided speech recognition (in both omnidirectional and directional modes) was assessed in the presence of three types of maskers (forward and reversed speech and speech-modulated noise) that varied in the amount of informational masking they were expected to produce. Study results showed significant directional benefit in all conditions. This finding suggests that in everyday conversational speech environments, directional technology is equally efficacious regardless of the magnitude of informational masking present in the background noise. In addition, study findings suggest that the semantic information present in the masking speech may play only a limited role in contributing to informational masking in everyday environments.

The benefit of binaural hearing in a cocktail party: effect of location and type of interferer.

The "cocktail party problem" was studied using virtual stimuli whose spatial locations were generated using anechoic head-related impulse responses from the AUDIS database [Blauert et al., J. Acoust. Soc. Am. 103, 3082 (1998)]. Speech reception thresholds (SRTs) were measured for Harvard IEEE sentences presented from the front in the presence of one, two, or three interfering sources. Four types of interferer were used: (1) other sentences spoken by the same talker, (2) time-reversed sentences of the same talker, (3) speech-spectrum shaped noise, and (4) speech-spectrum shaped noise, modulated by the temporal envelope of the sentences. Each interferer was matched to the spectrum of the target talker. Interferers were placed in several spatial configurations, either coincident with or separated from the target. Binaural advantage was derived by subtracting SRTs from listening with the "better monaural ear" from those for binaural listening. For a single interferer, there was a binaural advantage of 2-4 dB for all interferer types. For two or three interferers, the advantage was 2-4 dB for noise and speech-modulated noise, and 6-7 dB for speech and time-reversed speech. These data suggest that the benefit of binaural hearing for speech intelligibility is especially pronounced when there are multiple voiced interferers at different locations from the target, regardless of spatial configuration; measurements with fewer or with other types of interferers can underestimate this benefit.

Use of speech-modulated noise adds strong "bottom-up" cues for phonemic restoration.

When deleted segments of speech are replaced by extraneous sounds rather than silence, the missing speech fragments may be perceptually restored and intelligibility improved. This phonemic restoration (PhR) effect has been used to measure various aspects of speech processing, with deleted portions of speech typically being replaced by stochastic noise. However, several recent studies of PhR have used speech-modulated noise, which may provide amplitude-envelope cues concerning the replaced speech. The present study compared the effects upon intelligibility of replacing regularly spaced portions of speech with stochastic (white) noise versus speech-modulated noise. In Experiment 1, filling periodic gaps in sentences with noise modulated by the amplitude envelope of the deleted speech fragments produced twice the intelligibility increase obtained with interpolated stochastic noise. Moreover, when lists of isolated monosyllables were interrupted in Experiment 2, interpolation of speech-modulated noise increased intelligibility whereas stochastic noise reduced intelligibility. The augmentation of PhR produced by modulated noise appeared without practice, suggesting that speech processing normally involves not only a narrowband analysis of spectral information but also a wideband integration of amplitude levels across critical bands. This is of considerable theoretical interest, but it also suggests that since PhRs produced by speech-modulated noise utilize potent bottom-up cues provided by the noise, they differ from the PhRs produced by extraneous sounds, such as coughs and stochastic noise.

Researchers beware: Use of speech-modulated noise adds strong ‘‘bottom-up’’ cues to phonemic restoration

When deleted segments of speech are replaced by extraneous sound, the missing speech fragments may be perceptually restored and intelligibility improved. This phonemic restoration (PhR) effect has been used to measure various aspects of speech processing, with deleted portions of speech typically replaced by Gaussian noise. However, several recent studies [e.g., A. G. Samuel, J. Mem. Lang. 26, 36–56 (1987)] used noise that followed the amplitude envelope of the speech fragment, providing precise moment-by-moment control over the signal/noise ratio, but also possibly providing envelope cues concerning the replaced speech. The present study examined this possibility by comparing the effects of Gaussian noise and speech-modulated noise interrupters upon the restoration of intelligibility. Filling periodic interruptions in sentences with modulated noise produced twice the intelligibility increase obtained with Gaussian noise. Further, when isolated monosyllables were interrupted, interpolation of modulated noise increased intelligibility whereas Gaussian noise reduced intelligibility. It is suggested that, although the enhancement of intelligibility of speech-modulated noise is an interesting topic, the results obtained with the stimulus are confounded by the addition of bottom-up information not present when PhRs are produced by independent extraneous sounds. [Work supported by NIH.]