Listeners Benefit Research Articles

Hearing impairment and audiovisual speech integration ability: a case study report.

Research in audiovisual speech perception has demonstrated that sensory factors such as auditory and visual acuity are associated with a listener's ability to extract and combine auditory and visual speech cues. This case study report examined audiovisual integration using a newly developed measure of capacity in a sample of hearing-impaired listeners. Capacity assessments are unique because they examine the contribution of reaction-time (RT) as well as accuracy to determine the extent to which a listener efficiently combines auditory and visual speech cues relative to independent race model predictions. Multisensory speech integration ability was examined in two experiments: an open-set sentence recognition and a closed set speeded-word recognition study that measured capacity. Most germane to our approach, capacity illustrated speed-accuracy tradeoffs that may be predicted by audiometric configuration. Results revealed that some listeners benefit from increased accuracy, but fail to benefit in terms of speed on audiovisual relative to unisensory trials. Conversely, other listeners may not benefit in the accuracy domain but instead show an audiovisual processing time benefit.

Cochlear dead regions constrain the benefit of combining acoustic stimulation with electric stimulation.

The aims of this study were to (1) detect the presence and edge frequency (fe) of a cochlear dead region in the ear with residual acoustic hearing for bimodal cochlear implant users, and (2) determine whether amplification based on the presence or absence of a dead region would improve speech understanding and sound quality. Twenty-two listeners with a cochlear implant in one ear and residual acoustic hearing in the nonimplanted ear were tested. Eleven listeners had a cochlear dead region in the acoustic-hearing ear and 11 did not. Dead regions were assessed with the threshold-equalizing noise (TEN) and the sweeping noise, psychophysical tuning curve tests. Speech understanding was assessed with monosyllabic words and the AzBio sentences at +10 dB signal-to-noise ratio. Speech- and music-quality judgments were obtained with the Judgment of Sound Quality questionnaire. Using shifted tips of the psychophysical tuning curve as a basis for diagnosis, the TEN had high sensitivity (0.91) and poor specificity (0.55) for this population. The value of fe was lower when estimated with the sweeping noise, psychophysical tuning curve test than with the TEN test. For the listeners with cochlear dead regions, speech understanding, speech quality and music quality were best when no amplification was applied for frequencies within the dead region. For listeners without dead regions, speech understanding was best with full-bandwidth amplification and was reduced when amplification was not applied when the audiometric threshold exceeded 80 dB HL. The data from this study suggest that, to improve bimodal benefit for listeners who combine electric and acoustic stimulation, audiologists should routinely test for the presence of cochlear dead regions and determine amplification bandwidth accordingly.

Evaluation of the sparse coding shrinkage noise reduction algorithm in normal hearing and hearing impaired listeners

Although there are numerous single-channel noise reduction strategies to improve speech perception in noise, most of them improve speech quality but do not improve speech intelligibility, in circumstances where the noise and speech have similar frequency spectra. Current exceptions that may improve speech intelligibility are those that require a priori knowledge of the speech or noise statistics, which limits practical application. Hearing impaired (HI) listeners suffer more in speech intelligibility than normal hearing listeners (NH) in the same noisy environment, so developing better single-channel noise reduction algorithms for HI listeners is justified. Our model-based “sparse coding shrinkage” (SCS) algorithm extracts key speech information in noisy speech. We evaluate it by comparison with a state-of-the-art Wiener filtering approach using speech intelligibility tests with NH and HI listeners. The model-based SCS algorithm relies only on statistical signal information without prior information. Results show that the SCS algorithm improves speech intelligibility in stationary noise and is comparable to the Wiener filtering algorithm. Both algorithms improve intelligibility for HI listeners but not for NH listeners. Improvement is less in fluctuating (babble) noise than in stationary noise. Both noise reduction algorithms perform better at higher input signal-to-noise ratios (SNR) where HI listeners can benefit but where NH listeners have already reached ceiling performance. The difference between NH and HI subjects in intelligibility gain depends fundamentally on the input SNR rather than the hearing loss level. We conclude that HI listeners need different signal processing algorithms from NH subjects and that the SCS algorithm offers a promising alternative to Wiener filtering. Performance of all noise reduction algorithms is likely to vary according to extent of hearing loss and algorithms that show little benefit for listeners with moderate hearing loss may be more beneficial for listeners with more severe hearing loss.

Listening with a foreign-accent: The interlanguage speech intelligibility benefit in Mandarin speakers of English

This study examined the intelligibility of native and Mandarin-accented English speech for native English and native Mandarin listeners. In the latter group, it also examined the role of the language environment and English proficiency. Three groups of listeners were tested: native English listeners (NE), Mandarin-speaking Chinese listeners in the US (M-US) and Mandarin listeners in Beijing, China (M-BJ). As a group, M-US and M-BJ listeners were matched on English proficiency and age of acquisition. A nonword transcription task was used. Identification accuracy for word-final stops in the nonwords established two independent interlanguage intelligibility effects. An interlanguage speech intelligibility benefit for listeners (ISIB-L) was manifest by both groups of Mandarin listeners outperforming native English listeners in identification of Mandarin-accented speech. In the benefit for talkers (ISIB-T), only M-BJ listeners were more accurate identifying Mandarin-accented speech than native English speech. Thus, both Mandarin groups demonstrated an ISIB-L while only the M-BJ group overall demonstrated an ISIB-T. The English proficiency of listeners was found to modulate the magnitude of the ISIB-T in both groups. Regression analyses also suggested that the listener groups differ in their use of acoustic information to identify voicing in stop consonants.

Working Memory, Age, and Hearing Loss

Hearing aids use complex processing intended to improve speech recognition. Although many listeners benefit from such processing, it can also introduce distortion that offsets or cancels intended benefits for some individuals. The purpose of the present study was to determine the effects of cognitive ability (working memory) on individual listeners' responses to distortion caused by frequency compression applied to noisy speech. The present study analyzed a large data set of intelligibility scores for frequency-compressed speech presented in quiet and at a range of signal-to-babble ratios. The intelligibility data set was based on scores from 26 adults with hearing loss with ages ranging from 62 to 92 years. The listeners were grouped based on working memory ability. The amount of signal modification (distortion) caused by frequency compression and noise was measured using a sound quality metric. Analysis of variance and hierarchical linear modeling were used to identify meaningful differences between subject groups as a function of signal distortion caused by frequency compression and noise. Working memory was a significant factor in listeners' intelligibility of sentences presented in babble noise and processed with frequency compression based on sinusoidal modeling. At maximum signal modification (caused by both frequency compression and babble noise), the factor of working memory (when controlling for age and hearing loss) accounted for 29.3% of the variance in intelligibility scores. Combining working memory, age, and hearing loss accounted for a total of 47.5% of the variability in intelligibility scores. Furthermore, as the total amount of signal distortion increased, listeners with higher working memory performed better on the intelligibility task than listeners with lower working memory did. Working memory is a significant factor in listeners' responses to total signal distortion caused by cumulative effects of babble noise and frequency compression implemented with sinusoidal modeling. These results, together with other studies focused on wide-dynamic range compression, suggest that older listeners with hearing loss and poor working memory are more susceptible to distortions caused by at least some types of hearing aid signal-processing algorithms and by noise, and that this increased susceptibility should be considered in the hearing aid fitting process.

The fluctuating-masker benefit for normal-hearing and hearing-impaired listeners with equal audibility at a fixed signal-to-noise ratio

While normal-hearing (NH) listeners demonstrate better speech intelligibility for fluctuating-masker than for stationary-noise conditions, hearing-impaired (HI) listeners generally show little or no fluctuating-masker benefit (FMB). This result has been interpreted in terms of suprathreshold deficits (e.g., reduced spectral or temporal resolution or distorted stream-segregation cues) that limit “dip-listening.” However, reduced FMB for HI listeners might instead be attributable to audibility limitations or to differences between the signal-to-noise ratios (SNRs) at which NH and HI listeners are tested. This study examined this issue by equalizing stationary-noise performance to allow measurements at a common SNR, equalizing audibility, and presenting identical signals to pairs of NH and HI listeners. Audibility was equalized using linear gain, low-pass filtering (4 kHz) and intensity filtering to remove speech-signal elements below the HI audiometric threshold. Nonsense-syllable identification performance in stationary noise was equalized by adjusting the response set size. Stationary-noise trials (adapting set size) were interleaved with fluctuating-masker trials (adapting SNR), ensuring stable stationary-noise performance throughout the test. Fluctuating maskers included low- and high-rate modulated noise, speech-modulated noise, and an interfering-talker condition. Results were assessed to determine whether and under which conditions the HI listeners demonstrated reduced FMB not attributable to SNR or audibility effects.

The matched and mismatched interlanguage speech intelligibility benefit for listeners (ISIB-L): Chinese L2 talkers’ production of English liquids

The matched and mismatched interlanguage speech intelligibility benefit for listeners (ISIB-L): Chinese L2 talkers’ production of English liquids

If I Had Three Wishes: The Educational and Social/Emotional Benefits of Oral Storytelling

Abstract: Librarians, educators, and professional storytellers have long claimed that storytelling can result in a range of educational and social/emotional benefits for listeners, but there is limited research supporting or negating these claims. To begin to redress this lack of research, this study investigated elementary school children's impressions of a live oral storytelling performance as revealed through their response drawings and written and oral responses and analyzed the resulting educational and social/emotional benefits to the listeners. The results showed that storytelling supports the improved development of critical thinking skills, creativity, active participation/engagement in learning, literacy skills, narrative thinking abilities, self-exploration, and interpersonal skills. These results provide compelling support for the use of live storytelling in classrooms and other formal and informal educational settings and for the further investigation of the possible benefits of storytelling for educational purposes.

The role of first-language production accuracy and talker-listener alignment in second-language speech Intelligibility

This study investigated variability in L2 speech intelligibility as a function of L1 speech intelligibility and of talker-listener L1 match. Non-native Korean talkers varying in their L2 proficiency were recorded reading simple English (L2) and Korean (L1) sentences. The intelligibility of these sentences was then assessed by Korean listeners (both Korean and English productions) and English listeners (English productions only) in a sentence recognition task. The results revealed that for these Korean-English bilingual talkers, L1 intelligibility was significantly correlated with L2 intelligibility for both Korean and English listeners, suggesting that variability in L1 speech intelligibility can serve as a predictor of variability in L2 production accuracy. We also examined the interlanguage speech intelligibility benefit for non-native listeners (ISIB-L) (e.g., Bent & Bradlow, 2003; Hayes-Harb et al., 2008). Korean listeners performed better at identifying the sentences produced by Korean talkers with relatively low L2 intelligibility, implying that the benefit of a shared native language between talkers and listeners may be larger when non-native listeners process speech from a talker with low L2 intelligibility. Overall, these findings indicate that variability in L2 speech intelligibility is related to language-general talker characteristics as well as to the talker-listener language alignment.

Effects of spatial localization on sentence recognition in noise

Speech recognition in noise improves when speech and noise are separated in space. Interaural level and time difference cues are used in spatial localization. Most people with hearing loss have better hearing in the low frequencies. Interaural cues for sound localization are also more important in the low frequencies. An objective of the current experiment was to assess the effects of spatial separation of speech and noise on the recognition of low-pass filtered speech in persons with normal hearing and hearing loss. Speech levels corresponding to 50% correct recognition of sentences were measured in a 65dB SPL multi talker noise. Spatial benefits for listeners with and without hearing loss are discussed.

Talker Differences in Clear and Conversational Speech: Vowel Intelligibility for Older Adults With Hearing Loss

To establish the range of talker variability for vowel intelligibility in clear versus conversational speech for older adults with hearing loss and to determine whether talkers who produced a clear speech benefit for young listeners with normal hearing also did so for older adults with hearing loss. Clear and conversational vowels in /bVd/ context produced by 41 talkers were presented in noise for identification by 40 older (ages 65-87 years) adults with sloping sensorineural hearing loss. Vowel intelligibility within each speaking style and the size of the clear speech benefit varied widely among talkers. The clear speech benefit was equivalent to that enjoyed by young listeners with normal hearing in an earlier study. Most talkers who had produced a clear speech benefit for young listeners with normal hearing also did so for the older listeners with hearing loss in the present study. However, effects of talker gender differed between listeners with normal hearing and listeners with hearing loss. The clear speech vowel intelligibility benefit generated for listeners with hearing loss varied considerably among talkers. Most talkers who produced a clear speech benefit for normal-hearing listeners also produced a benefit for listeners with hearing loss.

Informational Masking and Spatial Hearing in Listeners With and Without Unilateral Hearing Loss

This study assessed selective listening for speech in individuals with and without unilateral hearing loss (UHL) and the potential relationship between spatial release from informational masking and localization ability in listeners with UHL. Twelve adults with UHL and 12 normal-hearing controls completed a series of monaural and binaural speech tasks that were designed to measure informational masking. They also completed a horizontal localization task. Monaural performance by participants with UHL was comparable to that of normal-hearing participants. Unlike the normal-hearing participants, the participants with UHL did not exhibit a true spatial release from informational masking. Rather, their performance could be predicted by head shadow effects. Performance among participants with UHL in the localization task was quite variable, with some showing near-normal abilities and others demonstrating no localization ability. Individuals with UHL did not show deficits in all listening situations but were at a significant disadvantage when listening to speech in environments where normal-hearing listeners benefit from spatial separation between target and masker. This inability to capitalize on spatial cues for selective listening does not appear to be related to localization ability.

Bidirectional clear speech perception benefit for native and high-proficiency non-native talkers and listeners: Intelligibility and accentedness

This study investigated how native language background interacts with speaking style adaptations in determining levels of speech intelligibility. The aim was to explore whether native and high proficiency non-native listeners benefit similarly from native and non-native clear speech adjustments. The sentence-in-noise perception results revealed that fluent non-native listeners gained a large clear speech benefit from native clear speech modifications. Furthermore, proficient non-native talkers in this study implemented conversational-to-clear speaking style modifications in their second language (L2) that resulted in significant intelligibility gain for both native and non-native listeners. The results of the accentedness ratings obtained for native and non-native conversational and clear speech sentences showed that while intelligibility was improved, the presence of foreign accent remained constant in both speaking styles. This suggests that objective intelligibility and subjective accentedness are two independent dimensions of non-native speech. Overall, these results provide strong evidence that greater experience in L2 processing leads to improved intelligibility in both production and perception domains. These results also demonstrated that speaking style adaptations along with less signal distortion can contribute significantly towards successful native and non-native interactions.

The Effect of Different Cochlear Implant Microphones on Acoustic Hearing Individuals' Binaural Benefits for Speech Perception in Noise

Cochlear implant microphones differ in placement, frequency response, and other characteristics such as whether they are directional. Although normal-hearing (NH) individuals are often used as controls in studies examining cochlear implant users' binaural benefits, the considerable differences across cochlear implant microphones make such comparisons potentially misleading. The goal of this study was to examine binaural benefits for speech perception in noise for NH individuals using stimuli processed by head-related transfer functions (HRTFs) based on the different cochlear implant microphones. HRTFs were created for different cochlear implant microphones and used to test participants on the Hearing in Noise Test. Experiment 1 tested cochlear implant users and NH individuals with HRTF-processed stimuli and with sound field (SF) testing to determine whether the HRTFs adequately simulated SF testing. Experiment 2 determined the measurement error and performance-intensity function for the Hearing in Noise Test with NH individuals listening to stimuli processed with the various HRTFs. Experiment 3 compared NH listeners' performance across HRTFs to determine how the HRTFs affected performance. Experiment 4 evaluated binaural benefits for NH listeners using the various HRTFs, including ones that were modified to investigate the contributions of interaural time and level cues. The results indicated that the HRTFs adequately simulated SF testing for the Hearing in Noise Test. They also demonstrated that the test-retest reliability and performance-intensity function were consistent across HRTFs, and that the measurement error for the test was 1.3 dB, with a change in signal-to-noise ratio of 1 dB reflecting a 10% change in intelligibility. There were significant differences in performance when using the various HRTFs, with particularly good thresholds for the HRTF based on the directional microphone when the speech and masker were spatially separated, emphasizing the importance of measuring binaural benefits separately for each HRTF. Evaluation of binaural benefits indicated that binaural squelch and spatial release from masking were found for all HRTFs, and binaural summation was found for all but one HRTF, although binaural summation was less robust than the other types of binaural benefits. In addition, the results indicated that neither interaural time nor level cues dominated binaural benefits for the NH participants. This study provides a means to measure the degree to which cochlear implant microphones affect acoustic hearing with respect to speech perception in noise. It also provides measures that can be used to evaluate the independent contributions of interaural time and level cues. These measures provide tools that can aid researchers in understanding and improving binaural benefits in acoustic hearing individuals listening via cochlear implant microphones.

Effect of signal-temporal uncertainty during childhood: Detection of a tonal signal in a random-frequency, two-tone masker.

The purpose of this study was to determine if listeners benefit from a cue indicating when in time a signal occurs. In order to examine age-related differences in performance, three groups of children (5–7, 8–10, and 11–13 year-olds) and adults were tested. Thresholds were measured for a 120-ms, 1000-Hz signal presented in a continuous random-frequency, two-tone masker presented at 50 dB SPL. Each 120-ms masker burst was composed of a pair of tones, one selected from a uniform distribution 300–920 Hz and the other from 1080–3000 Hz. This masker was selected because it is expected to produce primarily informational masking. Listeners completed two temporal conditions: (1) temporally-defined, with the signal embedded in a 600-ms light cue and (2) temporally-uncertain, with no light cue. Two phases of testing were completed for each temporal condition. For phase 1, a single-interval, adaptive track procedure determined the midpoint of the psychometric function. In phase 2, signal levels around the midpoint were tested using a single-interval method of constant stimuli procedure. For both temporal conditions, listeners completed four runs of 40 trials, with an equal signal-nonsignal probability. Results indicate a substantial release from masking with reduced signal-temporal uncertainty for both children and adults.

Localization in noise and the role of forward masker fringe.

Previous research suggests that the localization of a target sound in a masking noise is more accurate when the masker is preceded by a forward masker fringe. It is unclear what information listeners are exploiting to produce this improvement in performance. Does the fringe simply reduce spatial uncertainty (by cuing the location of the masker) or does it provide a “context” against which the onset of the signal is more easily discerned? In order to address these questions, we examined conditions in which the masker is (a) pulsed on and off with a 60 ms target (no fringe), (b) immediately preceded by a 500‐ms masker fringe; or (c) preceded by a 500‐ms fringe + a 500‐ms gap. Masker spatial uncertainty was also manipulated by presenting the noise from a fixed and known location throughout a block, or from a location that varied across trials. The results suggest that listeners benefit from the presence of a forward masker fringe even when the masker location is known. However, localization performance is best when this fringe occurs immediately prior to the masker (i.e., no gap), suggesting that onset‐related cues also contribute to the improvement in localization. [Work supported by the AFOSR]

The interlanguage speech intelligibility benefit-listeners for Chinese accented English liquids.

This study investigated the intelligibility of Chinese-accented English liquids for native English, Chinese, and Korean listeners. The lateral /l/ and the retroflex /r/ in syllable initial and final positions were examined in three different contexts: segment, word, and sentence. Three groups of listeners with different language backgrounds accomplished a forced-choice identification task. For Chinese talkers’ production of the English liquids, both matched and mismatched interlanguage speech intelligibility benefits for listeners (ISIB-L) occurred in all the four positions except for the initial /l/ produced by low proficiency Chinese talkers. The ISIB-L, however, held only for high proficiency non-native listeners. The only exception was the syllable initial /l/ stimuli produced by low proficiency Chinese talkers in such a way that native English listeners were more accurate than Chinese and Korean listeners at identifying the strong Chinese accented syllable initial /l/ sound. Considering the contextual effects on ISIB-L, matched ISIB-L only occurred for high proficiency listeners in the segment context. There was evidence of both matched and mismatched ISIB-L effects only for high proficiency listeners in word context. In the sentence context, English listeners performed better than all non-native listeners; therefore, there was no evidence for ISIB-L.

Spoken-word recognition in foreign-accented speech by L2 listeners

Two cross-modal priming studies investigated the recognition of English words spoken with a foreign accent. Auditory English primes were either typical of a Dutch accent or typical of a Japanese accent in English and were presented to both Dutch and Japanese L2 listeners. Lexical-decision times to subsequent visual target words revealed that foreign-accented words can facilitate word recognition for L2 listeners if at least one of two requirements is met: the foreign-accented production is in accordance with the language background of the L2 listener, or the foreign accent is perceptually confusable with the standard pronunciation for the L2 listener. If neither one of the requirements is met, no facilitatory effect of foreign accents on L2 word recognition is found. Taken together, these findings suggest that linguistic experience with a foreign accent affects the ability to recognize words carrying this accent, and there is furthermore a general benefit for L2 listeners for recognizing foreign-accented words that are perceptually confusable with the standard pronunciation.

Speech-on-speech masking with variable access to the linguistic content of the masker speech

It has been reported that listeners can benefit from a release in masking when the masker speech is spoken in a language that differs from the target speech compared to when the target and masker speech are spoken in the same language [Freyman, R. L. et al. (1999). J. Acoust. Soc. Am. 106, 3578-3588; Van Engen, K., and Bradlow, A. (2007), J. Acoust. Soc. Am. 121, 519-526]. It is unclear whether listeners benefit from this release in masking due to the lack of linguistic interference of the masker speech, from acoustic and phonetic differences between the target and masker languages, or a combination of these differences. In the following series of experiments, listeners' sentence recognition was evaluated using speech and noise maskers that varied in the amount of linguistic content, including native-English, Mandarin-accented English, and Mandarin speech. Results from three experiments indicated that the majority of differences observed between the linguistic maskers could be explained by spectral differences between the masker conditions. However, when the recognition task increased in difficulty, i.e., at a more challenging signal-to-noise ratio, a greater decrease in performance was observed for the maskers with more linguistically relevant information than what could be explained by spectral differences alone.

Do nonnative listeners benefit as much as native listeners from spatial cues that release speech from masking?

Since most everyday communication takes place in less than optimal acoustic settings, it is important to understand how such environments affect nonnative listeners. In this study we compare the speech reception abilities of native and nonnative English speakers when they are asked to repeat semantically anomalous sentences masked by steady-state noise or two other talkers in two conditions: when the target and masker appear to be colocated; and when the target and masker appear to emanate from different loci. We found that the later the age of language acquisition, the higher the threshold for speech reception under all conditions, suggesting that the ability to extract speech information from masking sounds in complex acoustic situations depends on language competency. Interestingly, however, native and nonnative listeners benefited equally from perceived spatial separation (an acoustic cue that releases speech from masking) independent of whether the speech target was masked by speech or noise, suggesting that the acoustic factors that release speech from masking are not affected by linguistic competence. In addition speech reception thresholds were correlated with vocabulary scores in all individuals, both native and nonnative. The implications of these findings for nonnative listeners in acoustically complex environments are discussed.