Consonant Identification Performance Research Articles

The ability of cochlear implant users to use temporal envelope cues recovered from speech frequency modulation

Previous studies have demonstrated that normal-hearing listeners can understand speech using the recovered "temporal envelopes," i.e., amplitude modulation (AM) cues from frequency modulation (FM). This study evaluated this mechanism in cochlear implant (CI) users for consonant identification. Stimuli containing only FM cues were created using 1, 2, 4, and 8-band FM-vocoders to determine if consonant identification performance would improve as the recovered AM cues become more available. A consistent improvement was observed as the band number decreased from 8 to 1, supporting the hypothesis that (1) the CI sound processor generates recovered AM cues from broadband FM, and (2) CI users can use the recovered AM cues to recognize speech. The correlation between the intact and the recovered AM components at the output of the sound processor was also generally higher when the band number was low, supporting the consonant identification results. Moreover, CI subjects who were better at using recovered AM cues from broadband FM cues showed better identification performance with intact (unprocessed) speech stimuli. This suggests that speech perception performance variability in CI users may be partly caused by differences in their ability to use AM cues recovered from FM speech cues.

Spectral masking release with HiRes120 sound processing in cochlear implant users

Current sound processors in cochlear implants show poor frequency resolution compared to normal auditory processes. They smooth spectral gaps within background maskers, thereby preventing implant users from experiencing release from masking (i.e., improved speech identification in maskers showing spectral gaps compared to maskers with continuous spectra). The Advanced Bionics' HiRes120 coding scheme aims to increase the transmission of fine-grain spectral information using a strategy based upon current steering. If HiRes120 improves the precision of spectral coding compared to the standard HiRes strategy, listeners should show improved release from masking for the slowest spectral modulations in a noise masker. This hypothesis is tested in newly implanted adults who alternately use HiRes and HiRes120 sound processing for a period of 10 months. Every two or three months, consonant identification is measured in each implantee in the presence of a stationary speech-shaped noise (SSN) masker at a fixed signal-to-noise ratio yielding 30-40% correct identification with unmodulated noise. The SSN masker is either unmodulated or spectrally modulated to produce periodic, three ERB-wide “spectral gaps” in the noise masker. Masking release scores defined as the change in consonant identification performance in modulated versus unmodulated noise will be presented and discussed.

Consonant identification in noise: Effects of temporal asynchrony, noise fluctuations, and temporal envelope expansion

The effects of two temporal envelope expansion schemes on consonant identification performance were investigated with normal-hearing listeners. These effects were examined in the presence or absence of short-term energy fluctuations in the masker. The effects of adding a 500-ms forward fringe to the background noise was also investigated. Speech stimuli were 16 consonant–/a/ syllables. The expanded stimuli were obtained by raising their original envelope to the power k. In the first scheme, k was fixed at 2. In the second scheme, k varied from 4 to 0.3 as a function of the envelope power; the low-level parts of the envelope were therefore expanded whereas the high-level parts were compressed. As reported in many studies, identification was substantially improved by the presence of energy fluctuations in the masker. Adding the forward fringe generally degraded identification in fluctuating noise, but could improve identification in the stationary noise. Overall, expansion led to (i) poorer consonant identification when k was a constant and (ii) better consonant identification when k was varied. These effects were larger in stationary noise. This study suggests that consonant identification can be improved by a dual expansion/compression scheme. Some results obtained with hearing-impaired listeners will be presented.

Consonant recognition in noise with temporal cues: I. Effects of temporal envelope enhancement on identification performance

This study sought to determine whether consonant identification performance in noise can be improved by applying an expansive nonlinearity to the temporal envelope of the speech stimuli. Consonants in /a-consonant-a/ context were processed through four broad bandpass filters, half-wave rectified and lowpass filtered at 500 Hz to extract the temporal envelope from each band. The envelope from each band was used to modulate a band-limited noise. In each band, the temporal envelope was either left intact or expanded by raising it to the power 1.414 or 2. Three signal-to-noise ratios (S/N) were used: −6, 0, and +6 dB. Twelve young normal-hearing listeners were tested twice in each condition, one week apart, using a forced-choice paradigm. Overall, identification performance increased with S/N and training. For each S/N and testing session, small but consistent improvements in identification performance of 2%–6% were observed when raising the envelopes to the power 1.414 or 2. Information transmission analysis performed on the aggregate confusion matrix showed that improved identification with expanded envelopes was mainly due to an increase in the reception of voicing.

Childrens’ Phoneme Identification in Reverberation and Noise

This study assessed the effects of reverberation, noise, and their combination on listeners' identification of consonants and vowels in naturally produced nonsense syllables presented at different sensation levels (re: speech recognition threshold). A secondary purpose of this study was to assess listeners' identification of voicing, manner, and place of articulation for consonants at 50 dB SL in the reverberation, noise, and combined conditions. Listeners, aged 6-30 years, identified consonant-vowel-consonant-vowel (CVCV) stimuli presented at four different sensation levels (re: speech recognition threshold) of 30, 40, 50, and 60 dB SL in 4 listening conditions: (a) an optimal listening situation (no reverberation, no noise), (b) reverberation only (1.3 seconds), (c) noise only (+13 dB S/N against a multitalker babble), and (d) reverberation plus noise. Results showed that all listener groups achieved maximum consonant identification performance at 50 dB SL. Vowel identification scores were unaffected by SL. Statistical analyses revealed that children's ability to identify consonants varied according to listening condition. For example, children's consonant identification abilities reached adult-like levels of performance at about age 14 years in the reverberation-only and noise-only listening conditions. However, in the reverberation-plus-noise listening condition, children's consonant identification abilities do not mature until the late teenage years. The ability to identify vowels, on the other hand, develops much earlier. A feature analysis of the consonant data showed that for all 3 features (voicing, manner, and place), identification scores were highest in the control condition, similar for the reverberation-only and noise-only conditions, and lowest in the reverberation-plus-noise condition. Voicing was easier for listeners to identify than manner or place of articulation features in reverberation and noise. Taken together, these results suggest that the ability to identify speech in reverberation and noise reaches adult-like level of performance at different ages for different components of the speech signal.

Importance of tonal envelope cues in Chinese speech recognition

Temporal waveform envelope cues provide significant information for English speech recognition, and, when combined with lip reading, could produce near-perfect consonant identification performance [Van Tasell et al., 1152–1161 (1987)]. Tonal patterns are important for Chinese speech recognition and can be effectively conveyed by temporal envelope cues [D. H. Whalen and Y. Xu, Phonetics 49, 25–47 (1992)]. This study investigates whether tones can help Chinese-speaking listeners use envelope cues more effectively than English listeners. The speech envelope was extracted from broad frequency bands and used to modulate a noise of the same bandwidth. Mandarin vowels, consonants, tones, and sentences were identified by ten native Chinese-speaking listeners with 1, 2, 3, and 4 noise bands (or channels). The results showed that recognition of vowels, consonants and sentences increases dramatically with the number of channels, a pattern similar to that observed in English speech recognition. However, tones were consistently recognized at about 80% correct level independent of the number of channels. This high level of tone recognition produced a significant difference in open-set sentence recognition between Chinese (11%) and English (1%, p<0.01) for the one channel condition where no spectral information is available.

Perception of stop consonants in speech signals reconstructed from phase or amplitude

It is well known that under certain conditions, a signal is completely specified either by its Fourier transform phase or amplitude. An iterative technique similar to that used by Hayes et al. [IEEE Trans. Acoust., Speech, Signal Process. ASSP‐28, 672–680 (1980)] was applied to the global Fourier transform phases or amplitudes of natural VCV (vowel–stop consonant–vowel) utterances and the perception of stop consonants was experimentally studied. Under a variety of conditions, the consonant identification performance in the phase‐only stimuli improved from about 66% to more than 90% after the iterations. The influence of the initial amplitude guess on the quality of the reconstructed signals was also investigated. The retrieval of stop consonant information from phase for various window lengths and various phase noise levels were analyzed with regard to vowel contexts, stop manner, and place of articulation. In contrast to the case of signal reconstruction from its phase, the stimulus reconstructed from th...

Relations among different measures of speech reception in subjects using a cochlear implant.

A comprehensive set of speech reception measures were obtained in a group of about 20 postlingually deafened adult users of the Ineraid multichannel cochlear implant. The measures included audio, visual, and audiovisual recognition of words embedded in two types of sentences (with differing degrees of difficulty) and audio-only recognition of isolated monosyllabic words, consonant identification (12 alternatives, /Ca/), and vowel identification (8 alternatives, /bVt/). For most implantees, the audiovisual gains in the sentence tests were very high. Quantitative relations among audio-only scores were assessed using power-law transformations suggested by Boothroyd and Nittrouer [J. Acoust. Soc. Am. 84, 101-114 (1988)] that can account for the benefit of sentence context (via a factor k) and the relation between word and phoneme recognition (via a factor j). Across the broad range of performance that existed among the subjects, substantial order was observed among measures of speech reception along the continuum from recognition of words in sentences, words in isolation, speech segments, and the retrieval of underlying phonetic features. Correlations exceeded 0.85 among direct and sentence-derived measures of isolated word recognition as well as among direct and word-derived measures of segmental recognition. Results from a variety of other studies involving presentation of limited auditory signals, single-channel and multichannel implants, and tactual systems revealed a similar pattern among word recognition, overall consonant identification performance, and consonantal feature recruitment. Finally, improving the reception of consonantal place cues was identified as key to producing the greatest potential gains in speech reception.

The importance of phase in the perception of intervocalic stop consonants

Identification experiments were performed to assess the relative importance of phase information versus amplitude information of vowel–stop–consonant–vowel (VCV) utterances for intervocalic consonant perception. Three types of stimuli were constructed from the VCVs spoken by six German speakers: (1) swapped stimuli: synthesized from the phase spectrum of one VCV utterance and the amplitude spectrum of another; (2) amplitude-only stimuli: the original amplitude spectrum of a VCV utterance combined with a zero or random phase spectrum; (3) phase-only stimuli: original phase spectrum with zero or random amplitude spectrum. About 66% of intervocalic stop consonants in the phase-only stimuli could be correctly identified as compared to less than 25% for amplitude-only stimuli. Over 55% of type 1 stimuli were identified as the VCV utterances that had provided the phase information, and less than 22% as those that had provided the amplitude information. Consonant identification performance was seen to vary with voicing and place of articulation of the stops, as well as with vowel context. The results indicated that it was the phase spectrum, rather than the amplitude spectrum of a VCV utterance that conveys more important information for stop consonant perception.

Frequency discrimination and consonant recognition in the hearing impaired

The frequency difference limen (DL) for short‐duration second‐formant transitions (F2) presented alone and in speechlike environments was examined in normal hearing and hearing‐impaired subjects. Four stimulus conditions were included: F2 transition alone; F2 transition in the presence of all formants (full formant); full‐formant stimuli preceded by a burst and followed by a vowel; and full‐formant stimuli preceded and followed by a vowel. For the simplest stimulus condition (F2 alone), all hearing‐impaired subjects had DLs within the 95% confidence interval around the mean of the normals. For all other conditions, however, the hearing impaired demonstrated extreme intersubject variation. Some hearing‐impaired subjects could not detect a formant transition of even 800 Hz, whereas others continued to perform like normal hearers. Consonant identification performance was also examined in CV and VCV environments. Consonants whose identification is believed to involve formant transitions were selected for study. In spite of modifications to the frequency discrimination task and use of relevant syllables, relatively weak correlations continue to be observed between F2 DL and consonant recognition ability, as well as between F2 DL and degree of hearing loss. [Work supported by NIH.]

Speech waveform envelope cues for consonant recognition.

This study investigated the cues for consonant recognition that are available in the time-intensity envelope of speech. Twelve normal-hearing subjects listened to three sets of spectrally identical noise stimuli created by multiplying noise with the speech envelopes of 19(aCa) natural-speech nonsense syllables. The speech envelope for each of the three noise conditions was derived using a different low-pass filter cutoff (20, 200, and 2000 Hz). Average consonant identification performance was above chance for the three noise conditions and improved significantly with the increase in envelope bandwidth from 20-200 Hz. SINDSCAL multidimensional scaling analysis of the consonant confusions data identified three speech envelope features that divided the 19 consonants into four envelope feature groups ("envemes"). The enveme groups in combination with visually distinctive speech feature groupings ("visemes") can distinguish most of the 19 consonants. These results suggest that near-perfect consonant identification performance could be attained by subjects who receive only enveme and viseme information and no spectral information.

Effects of reducing low-frequency amplification on consonant perception in quiet and noise.

The aim of this study was to assess the effect of low-frequency amplification on speech recognition performance by hearing-impaired listeners. Consonant identification performance by subjects with flat hearing losses and high-frequency hearing losses was assessed in three different hearing aid conditions, in quiet and noise. The experimental hearing aids all provided extra high-frequency amplification but differed in the amount of low-frequency amplification. The results showed that listeners with flat hearing losses benefited by low-frequency amplification, whereas subjects with high-frequency hearing losses exhibited deteriorating scores in conditions with greatest low-frequency amplification. Analyses of phonetic feature perception and individual consonant recognition scores revealed subtle interactions between hearing loss configuration and amplification contour.