Talker Intelligibility Research Articles

The effect of talker intelligibility on adaptation to an unfamiliar accent

Listening effort, as indexed by task-evoked pupil response, increases as the intelligibility of second language (L2) speech decreases, and this processing cost is mitigated by experience with talkers of the same accent (Porretta and Tucker, 2019). Prior research has shown that listeners can adapt to unfamiliar L2-accented speech and generalize the adaptation to different L2-accented talkers with the same L1 (Bradlow and Bent, 2008). The current study investigates what type of L2-accented speech exposure (high or low intelligibility) best reduces later listening effort to a novel talker of the same accent. We hypothesize that exposure to highly intelligible L2 speech supports adaptation to the features of an accent because it allows listeners to map accented tokens to the talker’s intended lexical representations. Alternatively, adaptation to a less intelligible talker may require exposure to other low-intelligibility speech, which is characterized by features that deviate more significantly from L1 norms. To test this, participants were exposed to simple sentences spoken in English by either L1 talkers, highly intelligible L2 talkers, or less intelligible L2 talkers. At test, all participants listened to different sentences spoken by a previously unheard L2 talker while listening effort was measured via pupillometry. Data collection is ongoing.

Using a novel in-mask non-invasive ventilator microphone to improve talker intelligibility in healthy and hospitalised adults

Purpose: Non-invasive ventilation (NIV) provides respiratory support without invasive endotracheal intubation but can hinder patients’ ability to communicate effectively. The current study presents preliminary results using a novel in-mask ventilator microphone to enhance talker intelligibility while receiving NIV. Method: A proof-of-concept study assessed sentence intelligibility of five healthy adult talkers using a prototype model of the microphone under continuous positive airway pressure (CPAP; 5/5 cm H2O) and bilevel positive airway pressure (BiPAP; 8/4 cm H2O) ventilator conditions. A pilot study then assessed intelligibility, subjective comprehensibility and naturalness, and patient- and conversation partner-reported communication outcomes for eight patients undergoing therapeutic NIV while being treated in an intensive care unit (ICU). Result: Intelligibility increased significantly with the microphone on in the BiPAP condition for healthy volunteers. For patients undergoing NIV in an ICU, intelligibility, comprehensibility, and patient and conversation partner ratings of conversation satisfaction significantly improved with the microphone on. Patients with lower baselines without the microphone in certain measures (intelligibility, comprehensibility) generally showed a greater microphone benefit than patients with higher baselines. Conclusion: Use of a novel microphone integrated into NIV improved intelligibility during ventilation for both healthy volunteers and patients undergoing therapeutic NIV. Additional clinical studies will define precise benefits and implications of such improved intelligibility.

Validating acoustic predictors of sentence intelligibility using a racially diverse corpus

The goal of the present work is to validate a set of widely used acoustic measures that have been found to predict talker intelligibility using a new, racially diverse corpus of native speakers of American English. Previous work by Bradlow et al (1996, Speech Communication) predicted sentence intelligibility on a set of 20 demographically unspecified talkers from the Midwest, where they found changes in F0 and larger vowel spaces to be associated with higher sentence intelligibility. Here, we examine whether these same acoustic features can be generalized to a more racially diverse population of talkers, of which only 9 are white and non-Latine. The current corpus comprises 28 talkers, where each talker produced 60 sentences. Here, we focus on a core set of sentences, 10 Havard/IEEE and 10 BEL sentences, that all 28 talkers produced. Key acoustic features such as F0, speaking rate, and first and second formants of vowels, were extracted to associate with intelligibility measurements for these individuals. Intelligibility measures are taken from a previous study (Tripp et al., 2022, J. Acoust. Soc. Am.). Results from the present work are important in generalizing which acoustic features impact intelligibility in a racially diverse corpus.

Digging into sentence intelligibility: Interactions with talker

An implicit feature found in studies of intelligibility and perception is the assumption that noise masks all spoken-sentence stimuli equivalently regardless of the stimulus-talker. For example, when using controlled and normed sentences, such as the IEEE Harvard set, the mean intelligibility across talkers should be affected by masker-noise equivalently. In other words, any particular sentence spoken by one talker should be equivalently intelligible to when it is spoken by another talker using the same style and rate. However, previous studies have shown that there is significant intelligibility variation between talkers in controlled stimuli. Therefore, we predict talker-masker interactions in the IEEE stimuli. We investigated the UAW speech intelligibility dataset which contains typed responses from over 900 native listeners of English to over 604 sentences from the UWNU IEEE sentence corpus presented in three signal-to-noise ratios ( + 2, 0, −2 dB). We calculated the Levenshtein Distance for the listeners’ transcriptions of the sentences. We find that different talkers have different intrinsic intelligibility even when reading the same set of sentences. Moreover, we find an interaction with SNR level and individual talker intelligibility. The present results suggest that individual talker effects in speech stimuli may influence the outcomes of perception and intelligibility studies.

Predicting talker intelligibility: Contrasting talkers’ racialized identities and listeners’ racializations

Recent studies have found that speech intelligibility can be manipulated by changing the race of a person whose face is used as a visual prime presented with a spoken sentence (Babel & Russell, 2015; McGowan, 2015; Kutlu et al., 2022). These findings suggest that the differences in racializing faces—the process whereby people ascribe a race to an individual an explanation for observed traits and behaviors and to predict future behavior–may mediate speech intelligibility. However, other data suggests that the contrastive effect arises more from perceived differences in vocal behavior (McLaughlin et al., 2022). In this study, we address this discrepancy in previous research. We examine variability in the process of racialization from faces and voices by contrasting the predictive power of talkers’ self-reported racialized identities on speech intelligibility with that of racializations reported by listeners. We examine this in an experiment assessing the audio-only and audio-visual intelligibility in noise of 28 talkers of various racialized identities by 242 listeners. The listeners both reported the speech they heard, and indicated what they believed the talkers’ racialized identities to be. Data collection is complete, and analysis is ongoing. [Funded by NIH grant R21 DC018070]

The impact of masker language and talker intelligibility on early bilinguals listeners' speech perception

Most real-life speech perception occurs in noisy environments. Previous research shows that listeners are better at understanding target speech that is masked by competing speech (babble masker) in a different (mismatched) language compared to when the target speech and babble are in the same (matched) language, a phenomenon referred to as “release from masking.” Here, we investigated release from masking for monolingual English and early Korean-English bilingual listeners. We also investigated whether talker intelligibility mediates release from masking. Listeners completed a sentence recognition task with both matched and mismatched babble. Critically, the early bilinguals heard English and Korean target speech. Results indicate that early bilinguals did not differ significantly from monolinguals when listening to English. Interestingly, when listening to Korean, early bilinguals also received a release from masking, despite the babble being their dominant language (English). Overall, these results indicate that earlybilinguals may be able to tolerate similar levels of adverse listening conditions as monolinguals in their dominant language. Furthermore, they are able to benefit from a target-babble linguistic mismatch in both their dominant and nondominant languages. Finally, across both languages, low-intelligibility talkers conditioned larger release from masking effects than high-intelligibility talkers, something previous research has not accounted for.

Effects of Spectral Envelope and Fundamental Frequency Shifts on the Perception of Foreign-Accented Speech.

To investigate the role of spectral pattern information in the perception of foreign-accented speech, we measured the effects of spectral shifts on judgments of talker discrimination, perceived naturalness, and intelligibility when listening to Mandarin-accented English and native-accented English sentences. In separate conditions, the spectral envelope and fundamental frequency (F0) contours were shifted up or down in three steps using coordinated scale factors (multiples of 8% and 30%, respectively). Experiment 1 showed that listeners perceive spectrally shifted sentences as coming from a different talker for both native-accented and foreign-accented speech. Experiment 2 demonstrated that downward shifts applied to male talkers and the largest upward shifts applied to all talkers reduced the perceived naturalness, regardless of accent. Overall, listeners rated foreign-accented speech as sounding less natural even for unshifted speech. In Experiment 3, introducing spectral shifts further lowered the intelligibility of foreign-accented speech. When speech from the same foreign-accented talker was shifted to simulate five different talkers, increased exposure failed to produce an improvement in intelligibility scores, similar to the pattern observed when listeners actually heard five foreign-accented talkers. Intelligibility of spectrally shifted native-accented speech was near ceiling performance initially, and no further improvement or decrement was observed. These experiments suggest a mechanism that utilizes spectral envelope and F0 cues in a talker-dependent manner to support the perception of foreign-accented speech.

Accent familiarity and adverse listening conditions

Our ability to understand spoken language depends on both the speaker and listening environment. Speech intelligibility is higher in more favorable listening environments. Accent familiarity also plays a major role, with familiar accents being more intelligible than unfamiliar accents. Previous research [S. Gittleman and K. J. Van Engen, “Effects of noise and talker intelligibility on judgments of accentedness,” J. Acoust. Soc. Am. 143(5), 3138–3145 (2018)] found a significant relationship between the level of noise and perceived “foreignness” of an accent for Mandarin-accented English and locally accented English speakers. Their results show that when participants listened to Mandarin-accented speakers they rated them as less accented in lower signal-to-noise ratios (SNR). However, when participants listened to local English speakers, they rated the speakers as more accented in lower SNRs. The current study replicates these procedures using locally accented English (n = 5) and Hawaiian Pidgin English (n = 5). In a within-listener design, listeners complete an intelligibility task and an accentedness rating task. In the accent rating task, listeners are presented with sentences in four conditions: clear, + 4 db SNR, 0 db SNR, and −4 db SNR. These results will contribute to our understanding of familiarity in speech processing.

Varying intelligibility and linguistic complexity in sentence recognition in noise: Insights from older listeners

Recognition of foreign-accented speech by older listeners with hearing loss has typically been examined with linguistically simple materials (words and short sentences), which may not capture the complexity of realistic speech experiences. In this study, older listeners with varying degrees of hearing acuity recognized simple (i.e., short, mono-clausal, canonical declarative syntax) and complex (i.e., longer, multi-clausal, non-canonical syntax, and/or passive voice) sentences in broadband noise at a signal-to-noise ratio of 5 decibels. Three talkers produced the sentences: one L1-English, one high-intelligibility and one low-intelligibility L2 English talkers (based on prior intelligibility testing with young, normal hearing listeners). As anticipated, sentence recognition accuracy dropped with decreasing talker intelligibility and increasing sentence complexity. However, the effect of linguistic complexity on recognition accuracy was modulated by talker intelligibility, as reflected in a robust interaction between the two variables. Namely, accuracy dropped with increasing linguistic complexity only for the native and high intelligibility talkers. This pattern persisted after accounting for the effects of working memory and degree of hearing acuity. These findings suggest that older listeners exhibit different listening strategies for low versus high intelligibility talkers, wherein recognition benefits attributable to linguistic simplicity only emerge for speech over a threshold of intelligibility.

Recognising foreign-accented speech of varying intelligibility and linguistic complexity: insights from older listeners with or without hearing loss

Objective The goal of this study was to assess recognition of foreign-accented speech of varying intelligibility and linguistic complexity in older adults. It is important to understand the factors that influence the recognition of this commonly encountered type of speech, in a population that remains understudied in this regard. Design A repeated measures design was used. Listeners repeated back linguistically simple and complex sentences heard in noise. The sentences were produced by three talkers of varying intelligibility: one native American English, one foreign-accented talker of high intelligibility and one foreign-accented talker of low intelligibility. Percentage word recognition in sentences was measured. Study sample Twenty-five older listeners with a range of hearing thresholds participated. Results We found a robust interaction between talker intelligibility and linguistic complexity. Recognition accuracy was higher for simple versus complex sentences, but only for the native and high intelligibility foreign-accented talkers. This pattern was present after effects of working memory capacity and hearing acuity were taken into consideration. Conclusion Older listeners exhibit qualitatively different speech processing strategies for low versus high intelligibility foreign-accented talkers. Differences in recognition accuracy for words presented in simple versus in complex sentence contexts only emerged for speech over a threshold of intelligibility.

Recognition of foreign-accented speech in noise: The interplay between talker intelligibility and linguistic structure.

Foreign-accented speech recognition is typically tested with linguistically simple materials, which offer a limited window into realistic speech processing. The present study examined the relationship between linguistic structure and talker intelligibility in several sentence-in-noise recognition experiments. Listeners transcribed simple/short and more complex/longer sentences embedded in noise. The sentences were spoken by three talkers of varying intelligibility: one native, one high-, and one low-intelligibility non-native English speakers. The effect of linguistic structure on sentence recognition accuracy was modulated by talker intelligibility. Accuracy was disadvantaged by increasing complexity only for the native and high intelligibility foreign-accented talkers, whereas no such effect was found for the low intelligibility foreign-accented talker. This pattern emerged across conditions: low and high signal-to-noise ratios, mixed and blocked stimulus presentation, and in the absence of a major cue to prosodic structure, the natural pitch contour of the sentences. Moreover, the pattern generalized to a different set of three talkers that matched the intelligibility of the original talkers. Taken together, the results in this study suggest that listeners employ qualitatively different speech processing strategies for low- versus high-intelligibility foreign-accented talkers, with sentence-related linguistic factors only emerging for speech over a threshold of intelligibility. Findings are discussed in the context of alternative accounts.

Effect of mode of delivery and background noise on speech characteristics of talkers in a classroom environment

Global and acoustic–phonetic correlates of speech intelligibility are important measures of talker intelligibility. Speech characteristics have been evaluated using speech samples that involve reading pre-administered passages under controlled environments. In classrooms, lecturing is the mode of speech delivery. The presence of noise from ceiling fans and other external noise hinder the speech communication process. This study evaluated the talkers’ speech characteristics utilizing recordings from graduate students under reading and lecturing modes and in the presence and absence of noise generated by ceiling fans. The acoustical conditions under which talkers delivered their speech were characterized using octave band U50 values. Global and acoustic–phonetic correlates of talker intelligibility were measured and the variation in correlates of speech intelligibility was statistically analyzed. The results revealed that talkers significantly modified their speech characteristics relevant to intelligibility across modes of speech delivery and in the presence of noise. Fundamental frequency measures such as F0-mean and F0-SD and durational measures such as speech and pause rates were all found to be higher for lecture mode of delivery. Talkers showed similar vowel-articulatory changes under the two modes of delivery. When lecturing in the presence of noise, talkers significantly reduced the length of pauses and also utilized a combination of vowel-articulatory strategies to overcome the presence of noise. The results suggest the need to investigate talkers’ speech adaptation in real classroom environments in terms of correlates of speech intelligibility and to reconsider classroom acoustical guidelines in view of both listener and talker intelligibilities.

Effects of noise and talker intelligibility on judgments of accentedness.

The goal of this study was to determine how noise affects listeners' subjective judgments of foreign-accented speech and how those judgments relate to the intelligibility of foreign-accented talkers. Fifty native English listeners heard native Mandarin speakers and native English speakers producing English sentences in quiet and in three levels of noise. Participants judged the accent of each speaker on a scale from 1 (native-like) to 9 (foreign). The results show that foreign-accented talkers were rated as less accented in the presence of noise, and that, while lower talker intelligibility was generally associated with higher (more foreign) accent ratings, the presence of noise significantly attenuated this relationship. In contrast, noise increased accent ratings and strengthened the relationship between intelligibility and accent ratings for native talkers. These findings indicate that, by obscuring the acoustic-phonetic cues that listeners use to judge accents, noise makes listeners less confident about the foreign (or native) status of a given talker.

Sentence recognition in noise: The interplay between talker intelligibility, familiarity, and linguistic complexity

Foreign-accented speech recognition is typically tested with linguistically simple sentences, which offer a limited window into speech processing. In this study, participants transcribed simple (i.e. mono-clausal, canonical declarative syntax) and complex (i.e. multi-clausal, non-canonical syntax, and/or passive voice) sentences in noise at two signal-to-noise ratios. The sentences were spoken by three talkers: one native, one high and one low intelligibility L2 English speaker. Unsurprisingly, recognition accuracy dropped with decreasing intelligibility and signal-to-noise ratio. Interestingly, accuracy dropped with increasing linguistic complexity only for the native and high intelligibility talkers. For the low intelligibility talker, sentence complexity did not affect recognition accuracy. This pattern of findings was present in mixed and blocked experimental designs, indicating no effect of familiarity with the talker. We argue that listeners employ qualitatively different speech processing strategies for low versus high intelligibility talkers, and suggest that this difference may be driven by the talker’s realization of prosodic structure. While native and high intelligibility talkers effectively encode the contrast between complex and simple sentences, low intelligibility talkers may not convey sufficient prosodic information, resulting in the recognition of both sentence types as non-hierarchically organized word sequences. [Work supported by Knowles Hearing Center, NIH grants R01-DC012289 and R01-DC005794.]

Intelligibility of Noise-Adapted and Clear Speech in Child, Young Adult, and Older Adult Talkers.

This study examined intelligibility of conversational and clear speech sentences produced in quiet and in noise by children, young adults, and older adults. Relative talker intelligibility was assessed across speaking styles. Sixty-one young adult participants listened to sentences mixed with speech-shaped noise at -5 dB signal-to-noise ratio. The analyses examined percent correct scores across conversational, clear, and noise-adapted conditions and the three talker groups. Correlation analyses examined whether talker intelligibility is consistent across speaking style adaptations. Noise-adapted and clear speech significantly enhanced intelligibility for young adult listeners. The intelligibility improvement varied across the three talker groups. Notably, intelligibility benefit was smallest for children's speaking style modifications. Listeners also perceived speech produced in noise by older adults to be less intelligible compared to the younger talkers. Talker intelligibility was correlated strongly between conversational and clear speech in quiet, but not for conversational speech produced in quiet and in noise. Results provide evidence that intelligibility variation related to age and communicative barrier has the potential to aid clinical decision making for individuals with speech disorders, particularly dysarthria.

Effects of talker intelligibility and noise on judgments of accentedness

The damaging effect of background noise on the intelligibility of foreign-accented speech has been well documented, but little is known about the effect of noise on listeners’ subjective judgments of accents. Noise adds distortion to speech, which may cause it to sound more “foreign.” On the other hand, noise may reduce perceived foreignness by masking cues to the accent. In this study, 40 native English speakers listened to 14 English-speaking native Mandarin speakers in four levels of noise: -4 dB, 0 dB, + 4 dB, and quiet. Participants judged each speaker on a scale from 1 (native-like) to 9 (foreign). The results showed a significant decrease in perceived accentedness as noise level increased. They also showed a significant interaction between noise and intelligibility: intelligibility (which had been measured for the same talkers in a previous study) had the greatest effect on perceived accentedness in quiet, and a reduced effect with increasing noise levels. These findings indicate that listeners’ decreased access to acoustic-phonetic cues in the presence of background noise also reduces their sensitivity to phonetic variation arising from foreign accents. Furthermore, the link between intelligibility and accentedness is weakened by the presence of noise.

Effect of noise on foreign-accent adaptation

Previous work indicates that listeners are able to adapt quickly to changes in accent [Clarke & Garrett, 2004; Bradlow & Bent, 2008] and that adaptation is faster for familiar accents [Witteman et al., 2013; Blanco et al., 2014]. The present study examines how listeners adapt to more and less familiar accents presented in noise as compared to quiet, since noisy listening environments are more perceptually challenging [Vaden et al., 2013; Gilbert et al., 2014] but also more closely represent typical listening conditions. Native English listeners heard blocks of sentences produced by native- or foreign-accented talkers (Korean, Spanish) and responded to a visual probe. Listener accuracy and reaction times were compared across accent and speaker blocks. Results showed that, unlike in quiet, listeners were unable to adapt to the less-familiar Korean accent. Processing of the familiar Spanish accent was successful in noise and quiet. Adaptation in noise was furthermore correlated with talker intelligibility: less intelligible talkers elicited lower accuracy and slower response; in quiet, there was no talker effect. The results revealed that the ability to adapt to a novel foreign accent is disrupted, as the extra perceptual effort in noise prevents listeners from generalizing about systematic variability of foreign-accented features.

Talker intelligibility across clear and sinewave vocoded speech

Talker intelligibility was compared across clear and sinewave vocoded speech. Ten talkers (5 female) from the Midwest and Western dialect regions recorded samples of 210 meaningful IEEE sentences, 206 semantically anomalous sentences, and 300 MRT words. Ninety-three normal hearing participants provided open set transcriptions of the materials presented in the clear over headphones. Forty-one different normal hearing participants provided open set transcriptions of the materials processed with an eight-channel sinewave vocoder. Transcription accuracy was highest for clear speech compared to vocoded speech, and for meaningful sentences, followed by anomalous sentences and words for both conditions. Weak talker effects were observed for the meaningful sentences in the clear (ranging from 97.7% to 98.2%), but were more pronounced for vocoded versions (68.5% to 85.5%). Weak talker effects were observed for semantically anomalous sentences in the clear (89.4%-93.3%), but more variability was observed across talkers in the vocoded condition (54.4%–73.7%). Finally, stronger talker effects were observed for clear and vocoded MRT words (83.8%–95.6%, 46.3%–59.0%, respectively). Talker rankings differed across stimulus conditions, as well as across processing conditions, but significant positive correlations between conditions were observed for meaningful and anomalous sentences, but not MRT words. Acoustic and dialect influences on intelligibility will be discussed.

Perception of Speech Produced by Native and Nonnative Talkers by Listeners with Normal Hearing and Listeners with Cochlear Implants

PURPOSE The aim of this study was to evaluate the understanding of English sentences produced by native (English) and nonnative (Spanish) talkers by listeners with normal hearing (NH) and listeners with cochlear implants (CIs). METHOD Sentence recognition in noise was measured in adult subjects with CIs and subjects with NH, all of whom were native talkers of American English. Test sentences were from the Hearing in Noise Test (HINT) database and were produced in English by four native and eight nonnative talkers. Subjects also rated the intelligibility and accent for each talker. RESULTS The speech recognition thresholds in noise of subjects with CIs and subjects with NH were 4.23 dB and 1.32 dB poorer with nonnative talkers than with native talkers, respectively. Performance was significantly correlated with talker intelligibility and accent ratings for subjects with CIs but only correlated with talker intelligibility ratings for subjects with NH. For all subjects, performance with individual nonnative talkers was significantly correlated with talkers' number of years of residence in the United States. CONCLUSION CI users exhibited a larger deficit in speech understanding with nonnative talkers than did subjects with NH, relative to native talkers. Nonnative talkers' experience with native culture contributed strongly to speech understanding in noise, intelligibility ratings, and accent ratings of both listeners with NH and listeners with CIs.

Developmental and gender-related trends of intra-talker variability in consonant production

This study investigates the effect of age and gender on the internal structure, cross-category distance, and discriminability of phonemic categories for two contrasts varying in fricative place of articulation (/s/-/∫/) and stop voicing (/b/-/p/) in word-initial tokens spoken by adults and normally developing children aged 9-14 yr. Vast between- and within-talker variability was observed with 16% of speakers exhibiting some degree of overlap between phonemic categories-a possible contribution to the range of talker intelligibility found in the literature. Females of all ages produced farther and thus more discriminable categories than males, although gender-marking for fricative between-category distance did not emerge until approximately 11 yr of age. Children produced farther yet also much more dispersed categories than adults with increasing discriminability with age, such that by age 13, children's categories were no less discriminable than those of adults. However, children's ages did not predict category distance or dispersion, indicating that convergence on adult-like category structure must occur later in adolescence.