Noise Vocoder Research Articles

Expectancy Constraints in Degraded Speech Modulate the Language Comprehension Network

In speech comprehension, the processing of auditory information and linguistic context are mutually dependent. This functional magnetic resonance imaging study examines how semantic expectancy ("cloze probability") in variably intelligible sentences ("noise vocoding") modulates the brain bases of comprehension. First, intelligibility-modulated activation along the superior temporal sulci (STS) was extended anteriorly and posteriorly in low-cloze sentences (e.g., "she weighs the flour") but restricted to a mid-superior temporal gyrus/STS area in more predictable high-cloze sentences (e.g., "she sifts the flour"). Second, the degree of left inferior frontal gyrus (IFG) (Brodmann's area 44) involvement in processing low-cloze constructions was proportional to increasing intelligibility. Left inferior parietal cortex (IPC; angular gyrus) activation accompanied successful speech comprehension that derived either from increased signal quality or from semantic facilitation. The results show that successful decoding of speech in auditory cortex areas regulates language-specific computation (left IFG and IPC). In return, semantic expectancy can constrain these speech-decoding processes, with fewer neural resources being allocated to highly predictable sentences. These findings offer an important contribution toward the understanding of the functional neuroanatomy in speech comprehension.

Contribution of very low amplitude-modulation rates to intelligibility in a competing-speech task

It is generally agreed that the slow fluctuations in the envelope of speech in different spectral channels carry critical information for intelligibility. Previous studies in which amplitude modulation (AM) was selectively removed from the speech signal showed that modulation rates between 4 and 16 Hz are most important, and that rates falling outside this range contribute little or not at all to speech intelligibility. The present study investigated the role of very low (<4 Hz) AM rates in the ability to identify sentences in an interfering background talker. The mixture was processed through a noise vocoder. The depth of AM with rates below 4, 1.3, or 0.4 Hz was reduced using a multi-channel envelope compressor with a high compression ratio. Data obtained using nine normal-hearing listeners demonstrate that low-rate AM, in the range 0.4-4 Hz, contributes to the intelligibility of relatively long speech utterances, at least for adverse listening conditions in which background noise is present and listeners are forced to rely on envelope cues in a few spectral channels.

Benefit of high-rate envelope cues in vocoder processing: Effect of number of channels and spectral region

In cochlear implants, or vocoder simulations of cochlear implants, the transmission of envelope cues at high rates (related to voice fundamental frequency, f0) may be limited by the widths of the filters used to form the channels and/or by the cutoff frequency, f(lp), of the low-pass filters used for envelope extraction. The effect of varying f(lp) in tone and noise vocoders was investigated for channel numbers, N, from 6 to 18. As N increased, the widths of the channels decreased. The value of f(lp) was 45 Hz (envelope or "E" filter), or 180 Hz (pitch or "P" filter). The following combinations of cutoff frequencies were used for channels below and above 1500 Hz, respectively: EE, PE, EP, and PP. Results from a competing-talker task showed that the tone vocoder led to better intelligibility than the noise vocoder. The PP condition led to the best intelligibility and the EE condition to the worst. For N=6, intelligibility was better for condition PE than for condition EP. For N=18, the reverse was true. The results indicate that the channel bandwidths can compromise the transmission of f0-related envelope information, and suggest that vocoder simulations of cochlear-implant processing have limitations.

Simulating the effect of spread of excitation in cochlear implants

A model was developed to simulate acoustically the effects of excitation spread in cochlear implants (CI). Based on neurophysiologic data, the proposed model simulates the electrical-current decay rate associated with broad and narrow types of excitation, such as those produced by monopolar and bipolar electrode configurations. The effect of excitation spread on speech intelligibility was simulated in normal-hearing subjects by varying the slopes of the synthesis bands in the noise vocoder. Sentences and monosyllabic words processed via 4–16 channels of stimulation with varying degrees of excitation spread were presented to normal-hearing listeners for identification. Results showed significant interaction between spectral resolution (number of channels) and spread of excitation. The effect of narrowing the excitation spread was minimal when the spectral resolution was sufficiently good (>8 channels) but it was significant when the spectral resolution was poor (4 channels). A significant decrement in performance was observed for extremely narrow excitation spread. This outcome is partly consistent with behavioral data obtained with cochlear implant studies in that CI users tend to do as well or better with monopolar stimulation than with bipolar stimulation.

Contribution of very low modulation frequencies to speech intelligibility in a competing speech task

Vocoder studies have demonstrated that low-rate temporal-envelope cues (below ∼ 50 Hz) in a few frequency bands are sufficient for good speech identification in quiet. From the effects of low pass and high pass filtering of the temporal envelopes extracted from band pass filtered sentences, Drullman et al. [J. Acoust. Soc. Am. 95, 2670-2680 (1994); J. Acoust. Soc. Am. 95, 1053-1064 (1994)] argued that modulation frequencies between 4 and 16 Hz are the most important for speech identification in a steady background noise. However, Stone and Moore (JASA, in press) found that modulation frequencies below 2 Hz were also beneficial when the speech signal was processed through an 8-channel (but not a 12-channel) noise vocoder. The present study investigated speech identification for nine normal-hearing listeners using IEEE sentences presented with a competing talker; the mixed signal was processed using a 6-, 9-, or 15-channel noise vocoder. Low-rate temporal-envelope cues were manipulated using a compressor for each channel. Compression speed varied from slow to fast, so as to progressively degrade modulation frequencies below 4 Hz. Independent of the spectral resolution of the vocoder, mean intelligibility decreased as the compression speed increased; even modulation frequencies below 1.3 Hz contributed significantly to speech intelligibility.

Influence of multi‐channel dynamic range compression on intelligibility: effect of envelope modulation bandwidth

Stone and Moore [J. Acoust. Soc. Am. (in press)] showed that, as the speed and number of channels in a multi‐channel compressor increased, intelligibility of noise‐vocoded signals in a competing speech task decreased. The noise vocoder is often used to simulate the information conveyed by cochlear implants. However, the vocoder of Stone and Moore preserved only low‐rate (<45 Hz) envelope modulations whereas some implantees show sensitivity to envelope modulation rates up to about 300 Hz. Furthermore, intrinsic fluctuations in the noise carriers affect the reception of low‐rate modulations of the signal (Whitmal et al. [J. Acoust. Soc. Am. 122:2376‐2388, (2007)]). Here, a tone vocoder with N=8 or N=16 channels was used. Vocoding was preceded by N/8, N/4, or N/2 channels of compression, each using one of three speeds, affecting modulation rates up to about 2, 6, or 18 Hz, respectively. The lowpass filters extracting the channel envelopes had corner frequencies of 45 or 180 Hz. Intelligibility was measured using IEEE sentences with a competing speaker. The deleterious effect of compression with increasing channel number and speed was greater for the lower corner frequency. Compression of rates below 6 Hz affected intelligibility, independent of the presence of higher modulation rates.

The role of temporal fine structure in speech source segregation

Normal‐hearing (NH) listeners show better speech recognition when a stationary noise masker is replaced by an opposite‐gender competing talker at the same signal‐to‐noise ratio (SNR). Hearing‐impaired (HI) listeners often do not show this interfering‐talker benefit (ITB). This may be due to a reduced ability to use temporal fine structure (TFS). Consistent with this idea, NH listeners also show little ITB when TFS is removed. We hypothesized that TFS underlies the ITB by providing source‐segregation cues. To test this hypothesis, non‐auditory segregation cues were introduced in the form of a video of the talker's face. Speech intelligibility was estimated in NH listeners as a function of SNR for sentences spoken by a female talker and masked by speech‐spectrum shaped stationary noise or a single male talker. Target and masker were summed before processing by a 15‐channel noise vocoder to remove TFS, and presented with or without accompanying video. Without video, listeners received little ITB, consistent with previous results. Auditory‐visual conditions yielded as much as 9 dB of ITB, supporting the hypothesis that a diminished ability to perceptually segregate sources contributes to the lack of ITB in the absence of TFS. Similar results were obtained for unprocessed speech in HI listeners.

Effects of spectro-temporal modulation changes produced by multi-channel compression on intelligibility in a competing-speech task

These experiments are concerned with the intelligibility of target speech in the presence of a background talker. Using a noise vocoder, Stone and Moore [J. Acoust. Soc. Am. 114, 1023-1034 (2003)] showed that single-channel fast-acting compression degraded intelligibility, but slow compression did not. Stone and Moore [J. Acoust. Soc. Am. 116, 2311-2323 (2004)] showed that intelligibility was lower when fast single-channel compression was applied to the target and background after mixing rather than before, and suggested that this was partly due to compression after mixing introducing "comodulation" between the target and background talkers. Experiment 1 here showed a similar effect for multi-channel compression. In experiment 2, intelligibility was measured as a function of the speed of multi-channel compression applied after mixing. For both eight- and 12-channel vocoders with one compressor per channel, intelligibility decreased as compression speed increased. For the eight-channel vocoder, a compressor that only affected modulation depth for rates below 2 Hz still reduced intelligibility. Experiment 3 used 12- or 18-channel vocoders. There were between 1 and 12 compression channels, and four speeds of compression. Intelligibility decreased as the number and speed of compression channels increased. The results are interpreted using several measures of the effects of compression, especially "across-source modulation correlation."

Effect of age, presentation method, and learning on identification of noise-vocoded words

Noise vocoding was used to investigate the ability of younger and older adults with normal audiometric thresholds in the speech range to use amplitude envelope cues to identify words. In Experiment 1, four 50-word lists were tested, with each word presented initially with one frequency band and the number of bands being incremented until it was correctly identified by the listener. Both age groups required an average of 5.25 bands for 50% correct word identification and performance improved across the four lists. In Experiment 2, the same participants who completed Experiment 1 identified words in four blocked noise-vocoded conditions (16, 8, 4, 2 bands). Compared to Experiment 1, both age groups required more bands to reach the 50% correct word identification threshold in Experiment 2, 6.13, and 8.55 bands, respectively, with younger adults outperforming older adults. Experiment 3 was identical to Experiment 2 except the participants had no prior experience with noise-vocoded speech. Again, younger adults outperformed older adults, with thresholds of 6.67 and 8.97 bands, respectively. The finding of age effects in Experiments 2 and 3, but not in Experiment 1, seems more likely to be related to differences in the presentation methods than to experience with noise vocoding.

Priming and sentence context support listening to noise-vocoded speech by younger and older adults

Older adults are known to benefit from supportive context in order to compensate for age-related reductions in perceptual and cognitive processing, including when comprehending spoken language in adverse listening conditions. In the present study, we examine how younger and older adults benefit from two types of contextual support, predictability from sentence context and priming, when identifying target words in noise-vocoded sentences. In the first part of the experiment, benefit from context based on primarily semantic knowledge was evaluated by comparing the accuracy of identification of sentence-final target words that were either highly predictable or not predictable from the sentence context. In the second part of the experiment, benefit from priming was evaluated by comparing the accuracy of identification of target words when noise-vocoded sentences were either primed or not by the presentation of the sentence context without noise vocoding and with the target word replaced with white noise. Younger and older adults benefited from each type of supportive context, with the most benefit realized when both types were combined. Supportive context reduced the number of noise-vocoded bands needed for 50% word identification more for older adults than their younger counterparts.

Perceptual learning of noise vocoded words: Effects of feedback and lexicality.

Speech comprehension is resistant to acoustic distortion in the input, reflecting listeners' ability to adjust perceptual processes to match the speech input. This adjustment is reflected in improved comprehension of distorted speech with experience. For noise vocoding, a manipulation that removes spectral detail from speech, listeners' word report showed a significantly greater improvement over trials for listeners that heard clear speech presentations before rather than after hearing distorted speech (clear-then-distorted compared with distorted-then-clear feedback, in Experiment 1). This perceptual learning generalized to untrained words suggesting a sublexical locus for learning and was equivalent for word and nonword training stimuli (Experiment 2). These findings point to the crucial involvement of phonological short-term memory and top-down processes in the perceptual learning of noise-vocoded speech. Similar processes may facilitate comprehension of speech in an unfamiliar accent or following cochlear implantation.

Speech intelligibility in cochlear implant simulations: Effects of carrier type, interfering noise, and subject experience

Channel vocoders using either tone or band-limited noise carriers have been used in experiments to simulate cochlear implant processing in normal-hearing listeners. Previous results from these experiments have suggested that the two vocoder types produce speech of nearly equal intelligibility in quiet conditions. The purpose of this study was to further compare the performance of tone and noise-band vocoders in both quiet and noisy listening conditions. In each of four experiments, normal-hearing subjects were better able to identify tone-vocoded sentences and vowel-consonant-vowel syllables than noise-vocoded sentences and syllables, both in quiet and in the presence of either speech-spectrum noise or two-talker babble. An analysis of consonant confusions for listening in both quiet and speech-spectrum noise revealed significantly different error patterns that were related to each vocoder's ability to produce tone or noise output that accurately reflected the consonant's manner of articulation. Subject experience was also shown to influence intelligibility. Simulations using a computational model of modulation detection suggest that the noise vocoder's disadvantage is in part due to the intrinsic temporal fluctuations of its carriers, which can interfere with temporal fluctuations that convey speech recognition cues.

Vowel recognition via cochlear implants and noise vocoders: Effects of formant movement and duration

Previous work has demonstrated that normal-hearing individuals use fine-grained phonetic variation, such as formant movement and duration, when recognizing English vowels. The present study investigated whether these cues are used by adult postlingually deafened cochlear implant users, and normal-hearing individuals listening to noise-vocoder simulations of cochlear implant processing. In Experiment 1, subjects gave forced-choice identification judgments for recordings of vowels that were signal processed to remove formant movement and/or equate vowel duration. In Experiment 2, a goodness-optimization procedure was used to create perceptual vowel space maps (i.e., best exemplars within a vowel quadrilateral) that included F1, F2, formant movement, and duration. The results demonstrated that both cochlear implant users and normal-hearing individuals use formant movement and duration cues when recognizing English vowels. Moreover, both listener groups used these cues to the same extent, suggesting that postlingually deafened cochlear implant users have category representations for vowels that are similar to those of normal-hearing individuals.

Utilization of acoustic cues in identifying speech intonation contrasts in cochlear implant recipients and listeners with normal hearing

This study investigated the utilization of acoustic cues in identifying speech intonation contrasts in adult cochlear implant (CI) recipients and normal‐hearing (NH) individuals listening to spectrally and temporally degraded stimuli. Fundamental frequency, intensity, and duration patterns of a disyllabic word, popcorn, were manipulated orthogonally, resulting in 360 resynthesized stimuli. In a two‐alternative forced‐choice task, CI and NH participants identified whether each stimulus was question‐like or statement‐like. Each NH listener also identified stimuli that were noise vocoded. Preliminary results from seven CI and four NH listeners indicated that (a) weighting of fundamental frequency in the identification of CI listeners was less pronounced than that of NH listeners, (b) unlike NH listeners who showed little reliance upon intensity or duration patterns in identifying unprocessed stimuli, CI users demonstrated systematic dependence upon intensity and duration cues in identifying the same set of stimuli, and (c) with noise‐vocoded (spectrally and temporally degraded) stimuli, NH listeners’ identification exhibited weighting patterns for intensity and duration cues in a way similar to that of CI listeners. Implications for the processing of multidimensional acoustic cues for speech intonation in normal and electrical hearing will be discussed. [Work supported by NIDCD‐R01DC04786.]

Segregation of vowel sequences having spectral cues reduced using a noise-band vocoder

Previous studies have shown that a reduction in spectral cues can be associated with a deficit in pitch-based sequential segregation. The difficulties experienced by hearing-impaired listeners in cocktail party situations may then be explained in part by a segregation deficit due to the broadening of auditory filters. The reduced speech-recognition abilities of cochlear implant users in noisy environments could also potentially be related to a loss of spectral cues leading to a segregation deficit. The current study was designed to test the ability of listener’s hearing cochlear implant simulations to segregate sequences of vowels having alternating F0 values simulating concurrent voices. The vowels were either unprocessed or processed using noise vocoders having limited numbers of channels. Performance on the segregation task was evaluated using a paradigm in which listeners judged the temporal order of items within the sequence. By relating systematic reductions in the number of channels to performance on the segregations task, the current results have the potential to clarify potential limitations placed upon auditory scene analysis by cochlear implant processing.

Vowel formant movement and duration perceived through noise vocoders and cochlear implants

Formant movement and duration have been increasingly shown to be important cues for vowel recognition by normal-hearing adults; individuals enhance formant movement and duration contrasts when speaking clearly, and vowel recognition accuracy declines when these differences are reduced in signal-processed or synthesized speech. This study investigated how these cues contribute to vowel recognition by cochlear implant users and normal-hearing individuals listening to noise-vocoded speech. Individuals were tested on vowel recognition using stimuli with and without formant movement and duration cues, and performed a goodness-rating task that found best exemplars for vowels that had formant movement and duration variation. The results suggested that duration was an important cue to recognition for all listeners; removing duration reduced recognition accuracy by 6–19 percentage points in all conditions. Removing formant movement cues reduced the recognition of diphthongs to chance levels, but had little effect on the recognition of monophthongs. Goodness-rating data demonstrated that all listeners preferred systematic patterns of formant movement and duration variation, as occurs in natural speech. The results suggest that cochlear implant users and normal-hearing individuals give similar cue weightings to vowel formant movement and duration, but that formant movement may have less importance under these conditions than in previous studies.

Speech perception abilities of children with cochlear implants, tactile aids, or hearing aids : Osberger M.J.; Robbins A.M.; Miyamoto R.T.; Berry S.W.; Myres WA.; Kessler K.S.; Pope M.L.AM. J. OTOL. (1991) 12/SUPPL. (105–115)

Verbal communication in social environments often requires dividing attention between two or more simultaneous talkers. The ability to do this, however, may be diminished when the listener has limited access to acoustic cues or those cues are degraded, as is the case for hearing-impaired listeners or users of cochlear implants or hearing aids. The aim of the present study was to investigate the ability of normal-hearing (NH) listeners to divide their attention and recognize speech from two simultaneous talkers in simulated free-field listening conditions, with and without reduced acoustic cues. Participants (N = 11 or 12 depending on the experiment) were asked to recognize and repeat as many words as possible from two simultaneous, time-centered sentences uttered by a male and a female talker. In Experiment 1, the female and male talkers were located at 15° and +15°, 45° and +45°, or 90° and +90° azimuth, respectively. Speech was natural or processed through a noise vocoder and was presented at a comfortable loudness level (∼65 dB SPL). In Experiment 2, the female and male talkers were located at 45° and +45° azimuth, respectively. Speech was natural but was presented at a lower level (35 dB SPL) to reduce audibility. In Experiment 3, speech was vocoded and presented at a comfortable loudness level (∼65 dB SPL), but the location of the talkers was switched relative to Experiment 1 (i.e., the male and female talkers were at 45° and +45°, respectively) to reveal possible interactions of talker sex and location. Listeners recognized overall more natural words at a comfortable loudness level (76%) than vocoded words at a similar level (39%) or natural words at a lower level (43%). This indicates that recognition was more difficult for the two latter stimuli. On the other hand, listeners recognized roughly the same proportion of words (76%) from the two talkers when speech was natural and comfortable in loudness, but a greater proportion of words from the male than from the female talker when speech was vocoded (50% vs 27%, respectively) or was natural but lower in level (55% vs 32%, respectively). This asymmetry occurred and was similar for the three spatial configurations. These results suggest that divided listening becomes asymmetric when speech cues are reduced. They also suggest that listeners preferentially recognized the male talker, located on the right side of the head. Switching the talker's location produced similar recognition for the two talkers for vocoded speech, suggesting an interaction between talkers’ location and their speech characteristics. For natural speech at comfortable loudness level, listeners can divide their attention almost equally between two simultaneous talkers. When speech cues are limited (as is the case for vocoded speech or for speech at low sensation level), by contrast, the ability to divide attention equally between talkers is diminished and listeners favor one of the talkers based on their location, sex, and/or speech characteristics. Findings are discussed in the context of limited cognitive capacity affecting dividing listening in difficult listening situations.