Consonant Confusions Research Articles

Self-administered, internet-enabled, modified rhyme test (MRT) for evaluating consonant confusion in remote subjects

A self-administered MRT has been developed for persons unable to attend on-site for testing. Subjects remotely log on to a website and are instructed to use their own computer and headphones or earphones/earbuds and seek a quiet location without distractions for the test. After inputting personal identification, the software produces the maximum and minimum intensity sounds that will be heard to allow the subject to adjust the audio gain to a comfortable listening level. The subject is then instructed not to re-adjust the audio gain during the MRT. Each trial is initiated by the subject and consists of the forced choice of one of six rhyming words, which is embedded in a carrier sentence. The pre-recorded speech in a single test, consisting of 25 trials, is replayed at a pre-selected, fixed speech signal-to-noise ratio. The internet-enabled MRT has been validated for subjects with believed normal hearing (N = 18) by comparison with a conventional MRT in our audiology clinic in which subjects' hearing thresholds were determined, and stimuli were presented at a prescribed sensation level (N = 7). Results were also compatible with word scores obtained on-site without audiological controls when subjects wore high-fidelity headphones (N = 6). [Work supported by NIOSH and the Alpha Foundation.]

Using consonant confusion to predict unexplained hearing loss

Some listeners have difficulty understanding speech in noise that is not explained by traditional audiometry. Speech perception errors can manifest as consonant confusions such as /s/ for /sh/. Consonant confusions contain information that may be used to detect or quantify unexplained hearing loss. We developed an efficient consonant confusion test, Quick-VC, based on ten vowel-consonant-vowel (VCV) phonemes. Speech spectrum noise was added to each VCV to make consonant confusions sensitive to hearing impairment. Puretone threshold averages and consonant confusion matrices were collected from 15 listeners with normal hearing and 45 listeners with hearing loss. A logistic regression with regularized coefficients was performed on principal components of the consonant confusions. After removing the effect of audibility, the model predicted unexplained hearing loss greater than 5 dB for seven individuals with a cross-validated area under the curve (AUC) of 0.757, which is clinically acceptable. Six of these seven listeners reported bilateral tinnitus and four had a history of noise exposure, which implicates the peripheral auditory system. The Quick-VC test may be a valuable tool for identifying and quantifying unexplained hearing difficulties, perhaps due to partial loss of auditory nerve fibers, for which there is currently no clinical test.

Web-based psychoacoustics: Hearing screening, infrastructure, and validation

Anonymous web-based experiments are increasingly used in many domains of behavioral research. However, online studies of auditory perception, especially of psychoacoustic phenomena pertaining to low-level sensory processing, are challenging because of limited available control of the acoustics, and the inability to perform audiometry to confirm normal-hearing status of participants. Here, we outline our approach to mitigate these challenges and validate our procedures by comparing web-based measurements to lab-based data on a range of classic psychoacoustic tasks. Individual tasks were created using jsPsych, an open-source JavaScript front-end library. Dynamic sequences of psychoacoustic tasks were implemented using Django, an open-source library for web applications, and combined with consent pages, questionnaires, and debriefing pages. Subjects were recruited via Prolific, a subject recruitment platform for web-based studies. Guided by a meta-analysis of lab-based data, we developed and validated a screening procedure to select participants for (putative) normal-hearing status based on their responses in a suprathreshold task and a survey. Headphone use was standardized by supplementing procedures from prior literature with a binaural hearing task. Individuals meeting all criteria were re-invited to complete a range of classic psychoacoustic tasks. For the re-invited participants, absolute thresholds were in excellent agreement with lab-based data for fundamental frequency discrimination, gap detection, and sensitivity to interaural time delay and level difference. Furthermore, word identification scores, consonant confusion patterns, and co-modulation masking release effect also matched lab-based studies. Our results suggest that web-based psychoacoustics is a viable complement to lab-based research. Source code for our infrastructure is provided.

PHONEME RECOGNITION AND CONFUSIONS IN PATIENTS WITH SENSORINEURAL HEARING LOSS

Hearing impaired listeners show different phoneme confusions during speech recognition testing. The aim of the study was to analyze phoneme recognition in patients with sensorineural hearing loss during word recognition testing with monosyllable words, as well as, to compare consonant confusions in different vowel context. Recognition of 18 initial and final consonants was analyzed in a total of 698 presentations of the words. There were 1154 (82.7%) correct recognitions and 100 consonant confusions (7.2%). The patients did not response at a total of 71 presentations of the words which means that consonants in 142 cases (10.2%) were not recognized, nor confused. There are no consonant confusion patterns during suprathreshold testing with real words. In cases of phoneme confusions, listeners replace the stimulus word with another word from the lexical neighborhood. In terms of the vowel context, the consonants are the most easily identified in the context of the vowel /a/.

Visual phonemic knowledge and audiovisual speech-in-noise perception in school-age children

Our mental representations of speech sounds include information about the visible articulatory gestures that accompany different speech sounds. We call this visual phonemic knowledge. This study examined development of school-age children’s visual phonemic knowledge and their ability to use visual phonemic knowledge to supplement audiovisual speech processing. Sixty-two children (5–16 years) and 18 adults (19–35 years) completed auditory-only, visual-only, and audiovisual tests of consonant-vowel syllable repetition. Auditory-only and audiovisual conditions were presented in steady-state, speech-spectrum noise at individually set SNRs. Consonant confusions were analyzed to define visemes (clusters of phonemes that are visually confusable with one another but visually distinct from other phonemes) evident in adults’ responses to visual-only consonants and to compute the proportion of errors in each participant and modality that were within adults' visemes. Children were less accurate than adults at visual-only consonant identification. However, children as young as 5 years of age demonstrated some visual phonemic knowledge. Comparison of error patterns across conditions indicated that children used visual phonemic knowledge during audiovisual speech-in-noise recognition. Details regarding the order of acquisition of viseme will be discussed.

Effect of the Target and Conflicting Frequency and Time Ranges on Consonant Enhancement in Normal-Hearing Listeners.

In this paper, the effects of intensifying useful frequency and time regions (target frequency and time ranges) and the removal of detrimental frequency and time regions (conflicting frequency and time ranges) for consonant enhancement were determined. Thirteen normal-hearing (NH) listeners participated in two experiments. In the first experiment, the target and conflicting frequency and time ranges for each consonant were identified under a quiet, dichotic listening condition by analyzing consonant confusion matrices. The target frequency range was defined as the frequency range that provided the highest performance and was decreased 40% from the peak performance from both high-pass filtering (HPF) and low-pass filtering (LPF) schemes. The conflicting frequency range was defined as the frequency range that yielded the peak errors of the most confused consonants and was 20% less than the peak error from both filtering schemes. The target time range was defined as a consonant segment that provided the highest performance and was decreased 40% from that peak performance when the duration of the consonant was systematically truncated from the onset. The conflicting time ranges were defined on the coincided target time range because, if they temporarily coincide, the conflicting frequency ranges would be the most detrimental factor affecting the target frequency ranges. In the second experiment, consonant recognition was binaurally measured in noise under three signal processing conditions: unprocessed, intensified target ranges by a 6-dB gain (target), and combined intensified target and removed conflicting ranges (target-conflicting). The results showed that consonant recognition improved significantly with the target condition but greatly deteriorated with a target-conflicting condition. The target condition helped transmit voicing and manner cues while the target-conflicting condition limited the transmission of these cues. Confusion analyses showed that the effect of the signal processing on consonant improvement was consonant-specific: the unprocessed condition was the best for /da, pa, ma, sa/; the target condition was the best for /ga, fa, va, za, ʒa/; and the target-conflicting condition was the best for /na, ʃa/. Perception of /ba, ta, ka/ was independent of the signal processing. The results suggest that enhancing the target ranges is an efficient way to improve consonant recognition while the removal of conflicting ranges negatively impacts consonant recognition.

Speech Categorization Reveals the Role of Early-Stage Temporal-Coherence Processing in Auditory Scene Analysis.

Temporal coherence of sound fluctuations across spectral channels is thought to aid auditory grouping and scene segregation. Although prior studies on the neural bases of temporal-coherence processing focused mostly on cortical contributions, neurophysiological evidence suggests that temporal-coherence-based scene analysis may start as early as the cochlear nucleus (i.e., the first auditory region supporting cross-channel processing over a wide frequency range). Accordingly, we hypothesized that aspects of temporal-coherence processing that could be realized in early auditory areas may shape speech understanding in noise. We then explored whether physiologically plausible computational models could account for results from a behavioral experiment that measured consonant categorization in different masking conditions. We tested whether within-channel masking of target-speech modulations predicted consonant confusions across the different conditions and whether predictions were improved by adding across-channel temporal-coherence processing mirroring the computations known to exist in the cochlear nucleus. Consonant confusions provide a rich characterization of error patterns in speech categorization, and are thus crucial for rigorously testing models of speech perception; however, to the best of our knowledge, they have not been used in prior studies of scene analysis. We find that within-channel modulation masking can reasonably account for category confusions, but that it fails when temporal fine structure cues are unavailable. However, the addition of across-channel temporal-coherence processing significantly improves confusion predictions across all tested conditions. Our results suggest that temporal-coherence processing strongly shapes speech understanding in noise and that physiological computations that exist early along the auditory pathway may contribute to this process.SIGNIFICANCE STATEMENT Temporal coherence of sound fluctuations across distinct frequency channels is thought to be important for auditory scene analysis. Prior studies on the neural bases of temporal-coherence processing focused mostly on cortical contributions, and it was unknown whether speech understanding in noise may be shaped by across-channel processing that exists in earlier auditory areas. Using physiologically plausible computational modeling to predict consonant confusions across different listening conditions, we find that across-channel temporal coherence contributes significantly to scene analysis and speech perception and that such processing may arise in the auditory pathway as early as the brainstem. By virtue of providing a richer characterization of error patterns not obtainable with just intelligibility scores, consonant confusions yield unique insight into scene analysis mechanisms.

Temporal fine structure influences voicing confusions for consonant identification in multi-talker babble.

To understand the mechanisms of speech perception in everyday listening environments, it is important to elucidate the relative contributions of different acoustic cues in transmitting phonetic content. Previous studies suggest that the envelope of speech in different frequency bands conveys most speech content, while the temporal fine structure (TFS) can aid in segregating target speech from background noise. However, the role of TFS in conveying phonetic content beyond what envelopes convey for intact speech in complex acoustic scenes is poorly understood. The present study addressed this question using online psychophysical experiments to measure the identification of consonants in multi-talker babble for intelligibility-matched intact and 64-channel envelope-vocoded stimuli. Consonant confusion patterns revealed that listeners had a greater tendency in the vocoded (versus intact) condition to be biased toward reporting that they heard an unvoiced consonant, despite envelope and place cues being largely preserved. This result was replicated when babble instances were varied across independent experiments, suggesting that TFS conveys voicing information beyond what is conveyed by envelopes for intact speech in babble. Given that multi-talker babble is a masker that is ubiquitous in everyday environments, this finding has implications for the design of assistive listening devices such as cochlear implants.

Aoustical and perceptual characteristics of mandarin consonants produced with an electrolarynx

The electrolarynx (EL) is an electromechanical device that enables patients to produce voice following the surgical removal of their larynx. The purpose of this study is to understand the acoustic and perceptual characteristics of Mandarin consonants produced by EL speakers. First, the acoustic characteristics (including speech intensity, consonant duration, spectral peak, and F2 onset) of Mandarin EL consonants are investigated by comparing the EL and normal consonants. Then, a perceptual evaluation of EL consonants is conducted to identify the relationship between acoustical characteristics and perceptual intelligibility. The results suggest three consonant confusion types are mainly responsible for the poor intelligibility of Mandarin EL consonants: (1) the “unaspirated-for-aspirated” confusion caused by the significantly shortened voice onset time of aspirated consonants; (2) the “voiced-for-voiceless” confusion caused by the continuous pulsing of the EL device and low consonant intensity; and (3) the “perceptual omission” caused by low intensity of consonants and / or consonant omission. The results obtained are promising and potential for further improvements in Mandarin EL speech intelligibility.

Effects of speech-shaped noise on consonant identification by native and non-native Korean and English listeners

The global proliferation of machine-human interface via voice and audio suggests a growing need for including language-specificity in the design and implementation of acoustic signal processing devices for all individuals, especially those in educational settings. Second language learners are often more susceptible to acoustic distortions in their second language (i.e., L2). Speech understanding challenges that L2 learners face in noisy environments are often compounded by other normal cross-language influences that are expected to occur during L2 speech perception. This study evaluated consonant confusion patterns using consonant-vowel (CV) English tokens spoken by native Korean and English speakers. The speech stimuli were embedded in speech-shaped noise with varying signal-to-noise ratios. Speech scores were measured using an alternative forced-choice paradigm. Overall scores were evaluated for each listener group to first assess whether L2 learners were most robust to acoustic-variances in English when spoken by Korean compared to English native-speakers. Second, their speech scores and acoustic recordings were evaluated for constructive as well as destructive effects of these acoustic variance on their masking and consonant confusion patterns. Results will be presented on the similarity and differences in acoustics and perception between English and Korean native-speakers. The clinical implications of our findings will be discussed.

Feasibility of Revision Cochlear Implant Surgery for Better Speech Comprehension.

Background and ObjectivesThe purpose of this study was to evaluate the efficacy of revision cochlear implant (CI) surgery for better speech comprehension targeting patients with low satisfaction after first CI surgery.Subjects and MethodsEight patients who could not upgrade speech processors because of an too early CI model and who wanted to change the whole system were included. After revision CI surgery, we compared speech comprehension before and after revision CI surgery. Categoies of Auditory Performance (CAP) score, vowel and consonant confusion test, Ling 6 sounds, word and sentence identification test were done.ResultsThe interval between surgeries ranged from eight years to 19 years. Same manufacturer’s latest product was used for revision surgery in six cases of eight cases. Full insertion of electrode was possible in most of cases (seven of eight). CAP score (p-value=0.01), vowel confusion test (p-value=0.041), one syllable word identification test (p-value=0.026), two syllable identification test (p-value=0.028), sentence identification test (pvalue=0.028) had significant improvement. Consonant confusion test (p-value=0.063), Ling 6 sound test (p-value=0.066) had improvement but it is not significant.ConclusionsAlthough there are some limitations of our study design, we could identify the effect of revision (upgrade) CI surgery indirectly. So we concluded that if patient complain low functional gain or low satisfaction after first CI surgery, revision (device upgrade) CI surgery is meaningful even if there is no device failure.

A Matrixed Speech-in-Noise Test to Discriminate Favorable Listening Conditions by Means of Intelligibility and Response Time Results.

The primary aim of this study was to develop and examine the potentials of a new speech-in-noise test in discriminating the favorable listening conditions targeted in the acoustical design of communication spaces. The test is based on the recognition and recall of disyllabic word sequences. A secondary aim was to compare the test with current speech-in-noise tests, assessing its benefits and limitations. Young adults (19-40 years old), self-reporting normal hearing, were presented with the newly developed Words Sequence Test (WST; 16 participants, Experiment 1) and with a consonant confusion test and a sentence recognition test (Experiment 2, 36 participants randomly assigned to the 2 tests). Participants performing the WST were presented with word sequences of different lengths (from 2 up to 6 words). Two listening conditions were selected: (a) no noise and no reverberation, and (b) reverberant, steady-state noise (Speech Transmission Index: 0.47). The tests were presented in a closed-set format; data on the number of words correctly recognized (speech intelligibility, IS) and the response times (RTs) were collected (onset RT, single words' RT). It was found that a sequence composed of 4 disyllabic words ensured both the full recognition score in quiet conditions and a significant decrease in IS results when noise and reverberation degraded the speech signal. RTs increased with the worsening of the listening conditions and the number of words of the sequence. The greatest onset RT variation was found when using a sequence of 4 words. In the comparison with current speech-in-noise tests, it was found that the WST maximized the IS difference between the selected listening conditions as well as the RT increase. Overall, the results suggest that the new speech-in-noise test has good potentials in discriminating conditions with near-ceiling accuracy. As compared with current speech-in-noise tests, it appears that the WST with a 4-word sequence allows for a finer mapping of the acoustical design target conditions of public spaces through accuracy and onset RT data.

Modeling talker- and listener-based sources of variability in babble-induced consonant confusions.

Speech communication often occurs in the presence of noise. Patterns of perceptual errors induced by background noise are influenced by properties of the listener and of the noise and target speech. The present study introduces a modification of multilevel general recognition theory in which talker- and listener-based variability in confusion patterns are modeled as global or dimension-specific scaling of shared, group-level perceptual distributions. Listener-specific perceptual correlations and response bias are also modeled as random variables. This model is applied to identification-confusion data from 11 listeners' identifications of ten tokens of each of four consonant categories-[t], [d], [s], [z]-produced by 20 talkers in CV syllables and masked by 10-talker babble. The results indicate that dimension-specific scaling for both listeners and talkers provides a good account of confusion patterns. These findings are discussed in relation to other recent research showing substantial listener-, talker-, and token-based sources of variability in noise-masked speech perception.

Consonant and Vowel Identification in Cochlear Implant Users Measured by Nonsense Words: A Systematic Review and Meta-Analysis.

The purpose of this systematic review and meta-analysis was to establish a baseline of the vowel and consonant identification scores in prelingually and postlingually deaf users of multichannel cochlear implants (CIs) tested with consonant-vowel-consonant and vowel-consonant-vowel nonsense syllables. Six electronic databases were searched for peer-reviewed articles reporting consonant and vowel identification scores in CI users measured by nonsense words. Relevant studies were independently assessed and screened by 2 reviewers. Consonant and vowel identification scores were presented in forest plots and compared between studies in a meta-analysis. Forty-seven articles with 50 studies, including 647 participants, thereof 581 postlingually deaf and 66 prelingually deaf, met the inclusion criteria of this study. The mean performance on vowel identification tasks for the postlingually deaf CI users was 76.8% (N = 5), which was higher than the mean performance for the prelingually deaf CI users (67.7%; N = 1). The mean performance on consonant identification tasks for the postlingually deaf CI users was higher (58.4%; N = 44) than for the prelingually deaf CI users (46.7%; N = 6). The most common consonant confusions were found between those with same manner of articulation (/k/ as /t/, /m/ as /n/, and /p/ as /t/). The mean performance on consonant identification tasks for the prelingually and postlingually deaf CI users was found. There were no statistically significant differences between the scores for prelingually and postlingually deaf CI users. The consonants that were incorrectly identified were typically confused with other consonants with the same acoustic properties, namely, voicing, duration, nasality, and silent gaps. A univariate metaregression model, although not statistically significant, indicated that duration of implant use in postlingually deaf adults predict a substantial portion of their consonant identification ability. As there is no ceiling effect, a nonsense syllable identification test may be a useful addition to the standard test battery in audiology clinics when assessing the speech perception of CI users.

Effects of Expanding Envelope Fluctuations on Consonant Perception in Hearing-Impaired Listeners.

This study examined the perceptual consequences of three speech enhancement schemes based on multiband nonlinear expansion of temporal envelope fluctuations between 10 and 20 Hz: (a) “idealized” envelope expansion of the speech before the addition of stationary background noise, (b) envelope expansion of the noisy speech, and (c) envelope expansion of only those time-frequency segments of the noisy speech that exhibited signal-to-noise ratios (SNRs) above −10 dB. Linear processing was considered as a reference condition. The performance was evaluated by measuring consonant recognition and consonant confusions in normal-hearing and hearing-impaired listeners using consonant-vowel nonsense syllables presented in background noise. Envelope expansion of the noisy speech showed no significant effect on the overall consonant recognition performance relative to linear processing. In contrast, SNR-based envelope expansion of the noisy speech improved the overall consonant recognition performance equivalent to a 1- to 2-dB improvement in SNR, mainly by improving the recognition of some of the stop consonants. The effect of the SNR-based envelope expansion was similar to the effect of envelope-expanding the clean speech before the addition of noise.

Evaluation of a Frequency-Lowering Algorithm for Adults With High-Frequency Hearing Loss.

The objective was to determine the effects of a frequency-lowering algorithm (frequency composition, Fcomp) on consonant identification, word-final /s, z/ detection, the intelligibility of sentences in noise, and subjective benefit, for people with high-frequency hearing loss, including people with dead regions (DRs) in the cochlea. A single-blind randomized crossover design was used. Performance with Bernafon Acriva 9 hearing aids was compared with Fcomp off and Fcomp on. Participants wore the hearing aids in each condition in a counterbalanced order. Data were collected after at least 8 weeks of experience with a condition. Outcome measures were audibility, scores from the speech perception tests, and scores from a questionnaire comparing self-perceived hearing ability with Fcomp off and Fcomp on. Ten adults with mild to severe high-frequency hearing loss (seven with extensive DRs, one with patchy or restricted DRs, and two with no DR) were tested. Fcomp improved the audibility of high-frequency sounds for 6 out of 10 participants. There was no overall effect of Fcomp on consonant identification, but the pattern of consonant confusions varied across conditions and participants. For word-final /s, z/ detection, performance was significantly better with Fcomp on than with Fcomp off. Questionnaire scores showed no differences between conditions. In summary, Fcomp improved word-final /s, z/ detection. No benefit was found for the other measures.

Computational models of speech perception by cochlear implant users

Cochlear implant (CI) users have access to fewer acoustic cues than normal hearing listeners, resulting in less than perfect identification of phonemes (vowels and consonants), even in quiet. This makes it possible to develop models of phoneme identification based on CI users’ ability to discriminate along a small set of linguistically-relevant continua. Vowel and consonant confusions made by CI users provide a very rich platform to test such models. The preliminary implementation of these models used a single perceptual dimension and was closely related to the model of intensity resolution developed jointly by Nat Durlach and Lou Braida. Extensions of this model to multiple dimensions, incorporating aspects of Lou’s novel work on “crossmodal integration,” have successfully explained patterns of vowel and consonant confusions; perception of “conflicting-cue” vowels; changes in vowel identification as a function of different intensity mapping curves and frequency-to-electrode maps; adaptation (or lack thereof) to changes in frequency-place functions; and some aspects of speech perception in noise. Our latest studies predict that enhanced phoneme identification by cochlear implant users may result from deactivation of a subset of electrodes in a patient’s map. All these results build upon, and were made possible by concepts from Lou’s work.

Predicting consonant recognition and confusions using a microscopic speech perception model

The perception of consonants has been investigated in various studies and shown to critically depend on fine details in the stimuli. The present study proposes a microscopic speech perception model that combines an auditory processing front end with a correlation-based template-matching back end to predict consonant recognition and confusions. The model represents an extension of the auditory signal processing model by Dau et al. [(1997), J. Acoust. Soc. Am. 102, 2892-2905] toward predicting microscopic speech perception data. Model predictions were computed for the extensive consonant perception data set provided by Zaar and Dau [(2015), J. Acoust. Soc. Am. 138, 1253-1267], obtained with consonant-vowels (CVs) in white noise. The predictions were in good agreement with the perceptual data both in terms of consonant recognition and confusions. The model was further evaluated with respect to perceptual artifacts induced by (i) different hearing-aid signal processing strategies and (ii) simulated cochlear-implant processing, based on data from DiNino et al. [(2016), J. Acoust. Soc. Am., 140, 4404-4418]. The model successfully predicted the strong consonant confusions measured in these conditions. Overall, the results suggest that the proposed model may provide a valuable framework for assessing acoustic transmission channels and hearing-instrument signal processing.

Comparisons in Consonant Confusions with and without gain for the Hearing-Impaired Listeners

Comparisons in Consonant Confusions with and without gain for the Hearing-Impaired Listeners

Assessing the efficacy of hearing-aid amplification using a phoneme test.

Consonant-vowel (CV) perception experiments provide valuable insights into how humans process speech. Here, two CV identification experiments were conducted in a group of hearing-impaired (HI) listeners, using 14 consonants followed by the vowel /ɑ/. The CVs were presented in quiet and with added speech-shaped noise at signal-to-noise ratios of 0, 6, and 12 dB. The HI listeners were provided with two different amplification schemes for the CVs. In the first experiment, a frequency-independent amplification (flat-gain) was provided and the CVs were presented at the most-comfortable loudness level. In the second experiment, a frequency-dependent prescriptive gain was provided. The CV identification results showed that, while the average recognition error score obtained with the frequency-dependent amplification was lower than that obtained with the flat-gain, the main confusions made by the listeners on a token basis remained the same in a majority of the cases. An entropy measure and an angular distance measure were proposed to assess the highly individual effects of the frequency-dependent gain on the consonant confusions in the HI listeners. The results suggest that the proposed measures, in combination with a well-controlled phoneme speech test, may be used to assess the impact of hearing-aid signal processing on speech intelligibility.