Consonant Recognition Research Articles

Effect of Computer-Assisted Speech Training on Speech Recognition and Subjective Benefits for Hearing Aid Users with Severe to Profound Prelingual Hearing Loss

Objective: Computer-assisted speech training is a speech recognition training system developed for cochlear implant users. With minimal facilities and skills, cochlear implant users can conduct this training at home. The purpose of this study was to apply this system to adolescent and young adult hearing aid users with prelingual severe to profound hearing loss. Study sample: Fifteen Mandarin-speaking hearing aid users with prelingual severe to profound hearing loss were included into this study. Another 6 hearing aid users with a similar background were recruited as the control group. Results: The training group had significant improvements in monosyllabic word recognition after 8 weeks of training, however the improvement diminished after 12 weeks of training. The training group also had significant improvements in consonant recognition after training for 12 weeks. There were no differences in improvement and final scores on the client-oriented scale of improvement between the training and control groups. Conclusion: With moderate training, the computer-assisted speech training system showed some benefits in training hearing aid users with prelingual severe to profound hearing loss, especially in the recognition of monosyllabic words and consonants.

Comparison between frequency transposition and frequency compression hearing aids

Abstract High-frequency hearing loss is one of the challenges for accurate hearing. One method of amplification toward improved detection and discrimination of high frequencies is through lowering of high frequencies to a lower-frequency hearing region. Frequency compression (FC) or frequency transposition (FT) can be used for individuals with different configurations of hearing impairment, such as steeply sloping and high-frequency hearing loss (containing high-frequency dead regions). The purpose of this review was to compare the effects of FC and FT on the speech recognition abilities of people with hearing impairment. This study is a systematic review of previous investigations carried out to test the efficacy of frequency-lowering algorithms (i.e. FC or FT) in improving the detection of high frequencies. The concept of FC and FT depends on enhancement of speech recognition in patients suffering from high-frequency hearing loss. In this review, compression shifting and frequency transposition were compared on the basis of improvements in the audibility of high frequency sound to improve the outstanding and acceptance level of speech. It can be concluded from this review that both FC and FT are useful in people with high-frequency hearing loss. FC has a potential role in recognition of monosyllabic words, consonants, and sentences in the presence of background noise. In contrast, FT facilitates detection of fricatives that ultimately leads to improvement in the discrimination of consonants. Throughout the literature, only one study has compared both FT and FC using various speech tests to evaluate the performance of different hearing instruments. Therefore, further studies addressing the benefits of both modalities of hearing aids with a more standardized outcome measure for both adult and pediatric patients are required.

Speech-cue transmission by an algorithm to increase consonant recognition in noise for hearing-impaired listeners.

Consonant recognition was assessed following extraction of speech from noise using a more efficient version of the speech-segregation algorithm described in Healy, Yoho, Wang, and Wang [(2013) J. Acoust. Soc. Am. 134, 3029-3038]. Substantial increases in recognition were observed following algorithm processing, which were significantly larger for hearing-impaired (HI) than for normal-hearing (NH) listeners in both speech-shaped noise and babble backgrounds. As observed previously for sentence recognition, older HI listeners having access to the algorithm performed as well or better than young NH listeners in conditions of identical noise. It was also found that the binary masks estimated by the algorithm transmitted speech features to listeners in a fashion highly similar to that of the ideal binary mask (IBM), suggesting that the algorithm is estimating the IBM with substantial accuracy. Further, the speech features associated with voicing, manner of articulation, and place of articulation were all transmitted with relative uniformity and at relatively high levels, indicating that the algorithm and the IBM transmit speech cues without obvious deficiency. Because the current implementation of the algorithm is much more efficient, it should be more amenable to real-time implementation in devices such as hearing aids and cochlear implants.

Mandarin Speech Perception in Combined Electric and Acoustic Stimulation

For deaf individuals with residual low-frequency acoustic hearing, combined use of a cochlear implant (CI) and hearing aid (HA) typically provides better speech understanding than with either device alone. Because of coarse spectral resolution, CIs do not provide fundamental frequency (F0) information that contributes to understanding of tonal languages such as Mandarin Chinese. The HA can provide good representation of F0 and, depending on the range of aided acoustic hearing, first and second formant (F1 and F2) information. In this study, Mandarin tone, vowel, and consonant recognition in quiet and noise was measured in 12 adult Mandarin-speaking bimodal listeners with the CI-only and with the CI+HA. Tone recognition was significantly better with the CI+HA in noise, but not in quiet. Vowel recognition was significantly better with the CI+HA in quiet, but not in noise. There was no significant difference in consonant recognition between the CI-only and the CI+HA in quiet or in noise. There was a wide range in bimodal benefit, with improvements often greater than 20 percentage points in some tests and conditions. The bimodal benefit was compared to CI subjects’ HA-aided pure-tone average (PTA) thresholds between 250 and 2000 Hz; subjects were divided into two groups: “better” PTA (<50 dB HL) or “poorer” PTA (>50 dB HL). The bimodal benefit differed significantly between groups only for consonant recognition. The bimodal benefit for tone recognition in quiet was significantly correlated with CI experience, suggesting that bimodal CI users learn to better combine low-frequency spectro-temporal information from acoustic hearing with temporal envelope information from electric hearing. Given the small number of subjects in this study (n = 12), further research with Chinese bimodal listeners may provide more information regarding the contribution of acoustic and electric hearing to tonal language perception.

Automated modification of consonant–vowel ratio of stops for improving speech intelligibility

Increasing the level of the consonant segments relative to the nearby vowel segments, known as consonant---vowel ratio (CVR) modification, is reported to be effective in improving speech intelligibility for listeners in noisy backgrounds and for hearing-impaired listeners. A technique for real-time CVR modification of stops using the rate of change of spectral centroid for detection of spectral transitions is presented. Its effectiveness in improving the recognition of consonants in the presence of speech-spectrum shaped noise is evaluated by conducting listening tests on normal-hearing subjects. At lower values of SNR, there was an increase of 7---21 % in recognition scores and an equivalent SNR advantage of 3 dB. The technique is implemented on a DSP board based on a 16-bit fixed point processor with on-chip FFT hardware and tested for satisfactory real-time operation.

Consonant recognition in noise for bilingual children with simulated hearing loss

The negative impacts of noisy listening environments and hearing loss on speech communication are known to be greater for children and non-native speakers than adult native speakers. Naturally, the synergistic influence of listening environment and hearing loss is expected to be greater for bilingual children than their monolingual or normal-hearing peers, but limited studies have explored this issue. The present study compared the consonant recognition performance of highly fluent school-age Spanish-English bilingual children to that of monolingual English-speaking peers. Stimulus materials were 13 English consonants embedded in three symmetrical vowel-consonant-vowel (VCV) syllables. To control for variability in hearing loss profiles, mild-to-moderate sloping sensorineural hearing loss modeled after Pittman & Stelmachowicz [Ear Hear 24, 198–205 (2003)] was simulated following the method used by Desloge et al. [Trends Amplification 16(1), 19–39 (2012)]. Listeners heard VCVs in quiet and in the background of speech-shaped noise with and without simulated hearing loss. Overall performance and the recognition of individual consonants will be discussed in terms of the influence of language background (bilingual vs. monolingual), listening condition, simulated hearing loss, and vowel context. [Work supported by NIH.].

Relationship between multipulse integration and speech recognition with cochlear implants.

Comparisons of performance with cochlear implants and postmortem conditions in the cochlea in humans have shown mixed results. The limitations in those studies favor the use of within-subject designs and non-invasive measures to estimate cochlear conditions. One non-invasive correlate of cochlear health is multipulse integration, established in an animal model. The present study used this measure to relate neural health in human cochlear implant users to their speech recognition performance. The multipulse-integration slopes were derived based on psychophysical detection thresholds measured for two pulse rates (80 and 640 pulses per second). A within-subject design was used in eight subjects with bilateral implants where the direction and magnitude of ear differences in the multipulse-integration slopes were compared with those of the speech-recognition results. The speech measures included speech reception threshold for sentences and phoneme recognition in noise. The magnitude of ear difference in the integration slopes was significantly correlated with the magnitude of ear difference in speech reception thresholds, consonant recognition in noise, and transmission of place of articulation of consonants. These results suggest that multipulse integration predicts speech recognition in noise and perception of features that use dynamic spectral cues.

Phoneme recognition in vocoded maskers by normal-hearing and aided hearing-impaired listeners.

This study compared modulation benefit for phoneme recognition obtained by normal-hearing (NH) and aided hearing-impaired (HI) listeners. Consonant and vowel recognition scores were measured using nonsense syllables in the presence of a steady-state noise and four vocoded speech maskers. Vocoded maskers were generated by modulating the steady-state noise, in either one or six frequency channels, with the speech envelope extracted from the speech of either a single talker or a four-talker babble. Aided HI listeners obtained lower consonant recognition scores than NH listeners in all masker conditions. Vowel recognition scores for aided HI listeners were comparable to NH scores, except in the six-channel vocoded masker conditions where they were relatively lower. Analysis using the extended speech intelligibility index developed by Rhebergen, Versfeld, and Dreschler [(2006). J. Acoust. Soc. Am. 120(6), 3988-3997] suggested that the signal-to-noise ratio deficit observed in aided HI listeners was largely due to uncompensated audibility loss. There was no significant difference between modulation masking release obtained by NH and aided HI listeners for both consonant and vowel recognition.

A NEW VECTOR QUANTIZATION APPROACH FOR DISCRETE HMM SPEECH RECOGNITION SYSTEM

In order to address accuracy issues of discrete Hidden Markov Models (HMMs), in this paper, a new vector quantization (VQ) approach is presented. This new VQ approach performs an optimal distribution of VQ codebook components on HMM states. This technique that we named the distributed vector quantization (DVQ) of hidden Markov models, succeeds in unifying acoustic micro-structure and phonetic macro-structure, when the estimation of HMM parameters is performed. The DVQ technique is implemented through two variants. The first variant uses the K-means algorithm (K-means-DVQ) to optimize the VQ, while the second variant exploits the benefits of the classification behavior of neural networks (NN-DVQ) for the same purpose. The proposed variants are compared with the HMMbased baseline system by experiments of specific Arabic consonants recognition. The results show that the distributed vector quantization technique increase the performance of the discrete HMM system.

Adaptation to nonlinear frequency compression in normal-hearing adults: A comparison of training approaches

Objective: To identify which training approach, if any, is most effective for improving perception of frequency-compressed speech. Design: A between-subject design using repeated measures. Study sample: Forty young adults with normal hearing were randomly allocated to one of four groups: a training group (sentence or consonant) or a control group (passive exposure or test-only). Test and training material differed in terms of material and speaker. Results: On average, sentence training and passive exposure led to significantly improved sentence recognition (11.0% and 11.7%, respectively) compared with the consonant training group (2.5%) and test-only group (0.4%), whilst, consonant training led to significantly improved consonant recognition (8.8%) compared with the sentence training group (1.9%), passive exposure group (2.8%), and test-only group (0.8%). Conclusions: Sentence training led to improved sentence recognition, whilst consonant training led to improved consonant recognition. This suggests learning transferred between speakers and material but not stimuli. Passive exposure to sentence material led to an improvement in sentence recognition that was equivalent to gains from active training. This suggests that it may be possible to adapt passively to frequency-compressed speech.

An algorithm to improve speech recognition in noise for hearing-impaired listeners: Consonant identification and articulatory feature transmission

Previous work has shown that a supervised-learning algorithm estimating the ideal binary mask (IBM) can improve sentence intelligibility in noise for hearing-impaired (HI) listeners from scores below 30% to above 80% [Healy et al., J. Acoust. Soc. Am. 134 (2013)]. The algorithm generates a binary mask by using a deep neural network to classify speech-dominant and noise-dominant time-frequency units. In the current study, these results are extended to consonant recognition, in order to examine the specific speech cues responsible for the observed performance improvements. Consonant recognition in speech-shaped noise or babble was examined in normal-hearing and HI listeners in three conditions: unprocessed, noise removed via the IBM, and noise removed via the classification-based algorithm. The IBM demonstrated substantial performance improvements, averaging up to 45% points. The algorithm also produced sizeable gains, averaging up to 34% points. An information-transmission analysis of cues associated with manner of articulation, place of articulation, and voicing indicated general similarity in the cues transmitted by the IBM versus the algorithm. However, important differences were observed, which may guide the future refinement of the algorithm. [Work supported by NIH.]

English

English

Benefit from non-linear frequency compression hearing aids in a clinical setting: The effects of duration of experience and severity of high-frequency hearing loss

Objectives: To determine whether non-linear frequency compression (NLFC) is effective for hearing-impaired adults in a clinical setting. To determine whether benefit from NLFC is related to duration of NLFC experience or severity of high-frequency hearing loss. Design: Participants were fitted with Phonak frequency compression hearing aids as part of their standard clinical care, using the manufacturer's default fitting settings. Participants had been using NLFC for between 1 and 121 weeks at the time of testing. Speech recognition thresholds in noise and consonant recognition in quiet were measured with and without NLFC enabled. Study sample: Forty-six experienced adult hearing-aid users. Results: Consonant recognition in quiet, but not speech recognition in noise was significantly better with NLFC enabled. There was no significant correlation between duration of frequency compression experience and benefit. Benefit for consonant recognition was negatively correlated with mean audiometric thresholds from 2–6 kHz. Conclusions: NLFC was beneficial for consonant recognition but not speech recognition in noise. There was no evidence to support the idea that a long period of acclimatization is necessary to gain full benefit. The relation between benefit and high-frequency thresholds might be explained by the poor audibility of compressed information for some listeners with severe loss.

Prediction of consonant recognition in quiet for listeners with normal and impaired hearing using an auditory model

Consonant recognition was assessed in normal-hearing (NH) and hearing-impaired (HI) listeners in quiet as a function of speech level using a nonsense logatome test. Average recognition scores were analyzed and compared to recognition scores of a speech recognition model. In contrast to commonly used spectral speech recognition models operating on long-term spectra, a "microscopic" model operating in the time domain was used. Variations of the model (accounting for hearing impairment) and different model parameters (reflecting cochlear compression) were tested. Using these model variations this study examined whether speech recognition performance in quiet is affected by changes in cochlear compression, namely, a linearization, which is often observed in HI listeners. Consonant recognition scores for HI listeners were poorer than for NH listeners. The model accurately predicted the speech reception thresholds of the NH and most HI listeners. A partial linearization of the cochlear compression in the auditory model, while keeping audibility constant, produced higher recognition scores and improved the prediction accuracy. However, including listener-specific information about the exact form of the cochlear compression did not improve the prediction further.

Admissible wavelet packet features based on human inner ear frequency response for Hindi consonant recognition

It was observed that for non-stationary and quasi-stationary signals, wavelet transform has been found to be an effective tool for the time–frequency analysis. In the recent years wavelet transform being used for feature extraction in speech recognition applications. Here a new filter structure using admissible wavelet packet analysis is proposed for Hindi phoneme recognition. These filters have the benefit of having frequency bands spacing similar to the auditory Equivalent Rectangular Bandwidth (ERB) scale whose central frequencies are equally distributed along the frequency response of human cochlea. The phoneme recognition performance of proposed feature is compared with the standard baseline features and 24-band admissible wavelet packet-based features using a Hidden Markov Model (HMM) based classifier. Proposed feature shows better performance compared to conventional features for Hindi consonant recognition. To evaluate the robustness of proposed feature in the noisy environment NOISEX-92 database has been used.

Spatial Separation Benefit for Unaided and Aided Listening

The benefit attributable to spatial separation of speech and noise was measured as a function of low-pass cutoff frequency with and without bilateral hearing aids. Fourteen younger and 10 older adults with normal hearing and 12 older adults with mild-to-moderate sloping high frequency hearing loss were included to assess the effects of age and hearing loss. Subjects with hearing loss were provided commercially available bilateral hearing aids. Consonant recognition was measured at 70 dB SPL in a background of speech-shaped noise at 66 dB SPL. Speech and noise were low-pass filtered at 1.7, 3.4, and 7.1 kHz. Nonsense syllables were always at 0° and noise was at either 0° or 90°. Speech and noise spectra for all conditions were digitally recorded using a probe microphone placed in each ear canal of each subject. Spectra and levels of speech, and quiet thresholds for narrowband noises, were used to calculate the Articulation Index and provide predictions of unaided and aided (hearing-impaired only) consonant recognition, spatial benefit, and hearing aid benefit for each condition. Subjective ratings of workload (NASA Task Load Index) were obtained for all unaided and aided measures of speech recognition. Consonant recognition in noise improved for all groups with speech and noise spatially separated and with the addition of high-frequency speech information. Scores were poorer overall for the older adults with hearing loss than for the other groups. For normal-hearing subjects, observed scores and spatial benefit were better than predicted. For hearing-impaired subjects, scores did not significantly improve with hearing aids, even with higher frequencies and spatial separation, and were poorer than predicted especially for aided listening. Similar to subjects with normal hearing, spatial benefit for hearing-impaired subjects was larger than predicted. Younger and older adults with normal hearing benefited from spatial separation of speech and noise sources to a greater extent than predicted based on simple audibility. Thus, no age-related deficits in the use of interaural difference cues were observed. Although hearing aid benefit was negligible, perceived listening effort was lower aided than unaided, especially with spatial separation. Articulation Index predictions revealed that speech audibility was generally restored with hearing aids across a wide bandwidth of speech, especially in the far ear. Thus, reduced audibility was not a primary factor in limited hearing aid benefit, suggesting that peripheral, central-auditory and/or cognitive changes may have played a role. In contrast, unaided and aided spatial benefit was better than predicted, and spatial benefit was slightly larger with hearing aids than without. Thus, these older adults with hearing loss using bilateral hearing aids were able to take advantage of binaural cues to improve consonant recognition in noise.

Analysis and Recognition of Arabic Consonants Using Locus Equations

 Abstract—The acoustic measurement of articulation place for consonants and the acoustic measurement of the amount of coarticulation are two long-standing problems in phonetics. Previous work on consonant place of articulation, using articulatory-acoustic models, within the acoustic theory of speech production (Stevens, 1998), has found certain acoustic correlates of place, but the variability found in empirically-based studies has not provided an easy match to the theoretical predictions. This study is to analyze the precision with which locus equations determine the place of articulation for both Arabic plosives / q / (/ق /) and /? / (/ء/) and the two fricatives / ʕ / (/ع/) and / h / (/ه/), where these are preceded by the front vowel /u / and posterior vowel /i/ with respect the middle vowel /a/. The results helped to make the distinction (or classification) of these four phonemes by locus equation.

Speech Recognition in Natural Background Noise

In the real world, human speech recognition nearly always involves listening in background noise. The impact of such noise on speech signals and on intelligibility performance increases with the separation of the listener from the speaker. The present behavioral experiment provides an overview of the effects of such acoustic disturbances on speech perception in conditions approaching ecologically valid contexts. We analysed the intelligibility loss in spoken word lists with increasing listener-to-speaker distance in a typical low-level natural background noise. The noise was combined with the simple spherical amplitude attenuation due to distance, basically changing the signal-to-noise ratio (SNR). Therefore, our study draws attention to some of the most basic environmental constraints that have pervaded spoken communication throughout human history. We evaluated the ability of native French participants to recognize French monosyllabic words (spoken at 65.3 dB(A), reference at 1 meter) at distances between 11 to 33 meters, which corresponded to the SNRs most revealing of the progressive effect of the selected natural noise (−8.8 dB to −18.4 dB). Our results showed that in such conditions, identity of vowels is mostly preserved, with the striking peculiarity of the absence of confusion in vowels. The results also confirmed the functional role of consonants during lexical identification. The extensive analysis of recognition scores, confusion patterns and associated acoustic cues revealed that sonorant, sibilant and burst properties were the most important parameters influencing phoneme recognition. . Altogether these analyses allowed us to extract a resistance scale from consonant recognition scores. We also identified specific perceptual consonant confusion groups depending of the place in the words (onset vs. coda). Finally our data suggested that listeners may access some acoustic cues of the CV transition, opening interesting perspectives for future studies.

Degraded word recognition in isolation vs a carrier phrase

Recognizing a spoken word presented in isolation is a markedly different task from recognizing a word in a carrier phrase. The presence of a carrier phrase provides additional challenges such as lexical segmentation but also provides additional information relevant to word recognition such as speaking rate and talker-specific spectral characteristics. The current set of studies is part of an attempt to determine how target word recognition differs in isolation versus in a carrier phrase. In an initial experiment, a set of 220 spoken CVC words were noise-vocoded (6 channel) and presented to listeners either in isolation or following a noise-vocoded carrier phrase—“The next word on the list is…” The target words were transcribed in each condition and scored for initial consonant accuracy and overall word accuracy. Despite the lack of semantic or syntactic information provided by the carrier phrase accuracy for both word and consonant recognition were much higher in the carrier phrase context. A second experiment, used different talkers for the carrier phrase and target word. A smaller but significant benefit over isolation was present for these mixed talker stimuli suggesting that the benefits of the carrier phrase include, but are not limited to, talker-specific information.

Semantic and Phonemic Listener Confusions in a Case of Isolated Congenital Aglossia

This article is the second in a series that examines the intelligibility of a person with congenital aglossia (PWCA). Specific factors examined in this study included (a) intelligibility for meaningful words versus nonsense words, (b) intelligibility for consonant-vowel-consonant words (CVCs) as a function of phonemic segment types, and (c) whether there is a correlation between intelligibility for these factors and the acoustic properties of the speech samples. Results revealed greater intelligibility for meaningful versus nonsense CVCs, greater intelligibility for back, low, and high-back versus front vowels embedded in CVCs, and greater intelligibility for productions as a function of phonemic variables, which demonstrated the following hierarchy: initial consonant &gt; consonant vowel &gt; vowel consonant &gt; final consonant. Further results suggest that consonant recognition was consistently affected by “vowel context.” This suggests that movement sequencing appeared to be of importance for speech perception in productions of a PWCA.