Non-native Listeners Research Articles

Speaker-listener neural coupling correlates with semantic and acoustic features of naturalistic speech.

Recent research has extensively reported the phenomenon of inter-brain neural coupling between speakers and listeners during speech communication. Yet, the specific speech processes underlying this neural coupling remain elusive. To bridge this gap, this study estimated the correlation between the temporal dynamics of speaker-listener neural coupling with speech features, utilizing two inter-brain datasets accounting for different noise levels and listener's language experiences (native vs. non-native). We first derived time-varying speaker-listener neural coupling, extracted acoustic feature (envelope) and semantic features (entropy and surprisal) from speech, and then explored their correlational relationship. Our findings reveal that in clear conditions, speaker-listener neural coupling correlates with semantic features. However, as noise increases, this correlation is only significant for native listeners. For non-native listeners, neural coupling correlates predominantly with acoustic feature rather than semantic features. These results revealed how speaker-listener neural coupling is associated with the acoustic and semantic features under various scenarios, enriching our understanding of the inter-brain neural mechanisms during natural speech communication. We therefore advocate for more attention on the dynamic nature of speaker-listener neural coupling and its modeling with multilevel speech features.

On Non-Native Listeners’ Ability to Identify Prominence and Pitch Accents in EnglishMonologic Speech

A comparative perceptual study involving two experimental groups with different native languages (Russian and Czech) shows that phonologically trained non-native speakers of English are good at identifying basic suprasegmental features of the English language, namely prominence (sentence stress) and accent types, which potentially makes it possible to use their prosodic annotations when validating cross-language intonation research. The occasional failure of both experimental groups to identify certain accent types is explained in the study by the annotators’ mother tongue’s prosodic interference: Czech and Russian speakers rely on different acoustic cues when identifying prosodic features in their native languages and transfer this habit to the discrimination of English prosodic characteristics. The study demonstrates that when a prosodic cue is not marked in the speaker’s mother tongue, it will likely be ignored in the foreign language.

Audiovisual perception of interrupted speech by nonnative listeners.

The purpose of the present study was to examine the influence of visual cues in audiovisual perception of interrupted speech by nonnative English listeners and to identify the role of working memory, long-term memory retrieval, and vocabulary knowledge in audiovisual perception by nonnative listeners. The participants included 31 Mandarin-speaking English learners between 19 and 41 years of age. The perceptual stimuli were noise-filled periodically interrupted AzBio and QuickSIN sentences with or without visual cues that showed a male speaker uttering the sentences. In addition to sentence recognition, the listeners completed a semantic fluency task, verbal (operation span) and visuospatial (symmetry span) working memory tasks, and two vocabulary knowledge tests (Vocabulary Level Test and Lexical Test for Advanced Learners of English). The results revealed significantly better speech recognition in the audio-visual condition than the audio-only condition, but the magnitude of visual benefit was substantially attenuated for sentences that had limited semantic context. The listeners' vocabulary size in English played a key role in the restoration of missing speech information and audiovisual integration in the perception of interrupted speech. Meanwhile, the listeners' verbal working memory capacity played an important role in audiovisual integration especially for the difficult stimuli with limited semantic context.

The development of English language connected speech perception skills: an empirical study on Chinese EFL children.

Connected speech processes (CSPs) occur randomly in everyday conversations of native speakers; however, such phonological variations can bring about challenges for non-native listeners. Looking at CSP literature, there seems to be very few studies that involved young foreign language learners. Therefore, the present study aimed to explore the development of connected speech perception skills by focusing on 201 9- to 12-year-old Chinese EFL children. It also incorporated systematic error analysis to further probe into the specific perceptual difficulties. The results indicate that: (1) Despite a significantly ascending trend for the overall growth of perception skills, no significant differences were found between 11 and 12 year olds in elision and contraction, which suggests that the developmental trend varied depending on different CSP types; (2) Although random errors decreased with age, the number of lexicon and syntax errors gradually increased, and the distribution of perceptual errors shifted from the level of words and syllables to that of phonemes; (3) The primary types of errors resulting in the perception difficulties for elision and contraction were consonant errors, grammatical errors and morphology errors. Ergo, this study enhances the understanding of connected speech perception among EFL children and provides some implications for EFL/ESL listening instructions.

Arabic speech intelligibility: Perception of spoken Arabic by native and non-native speakers

Previous studies have shown that native language backgrounds of both talkers and listeners affect speech intelligibility. This study investigated the interlanguage speech intelligibility benefit (ISIB) that is also known as the advantage in understanding second language (L2) speech that non-native listeners have over native listeners when both groups listen to speakers with the same first language (L1). More specifically, it looked into the ISIB in relation to the Arabic language spoken by both native Arabic (NA) and English speakers. To this end, 15 NA and 15 native English (NE) subjects listened to Arabic produced by two groups of talkers (5 NA talkers and 5 NE talkers) and were asked to identify the words they heard. Results showed evidence for the interlanguage speech intelligibility for listeners (i.e. NE listeners were more accurate than NA listeners at identifying English-accented Arabic speech). However, no evidence for the ISIB for talkers was found. That is, NE listeners did not find English-accented speech more intelligible than NA speech. By examining L2 learners’ recognition of L2 consonant contrasts, the study contributes to the body of knowledge on L2 sound acquisition as well as the ISIB literature. It also provides some insight into the problem of adult L2 learners’ ability to learn novel L2 consonants.

The Effect of Pitch Accent on the Perception of English Lexical Stress: Evidence from English and Mandarin Chinese Listeners

The relative weighting of f0 and vowel reduction in English spoken word recognition at the sentence level were investigated in one two-alternative forced-choice word identification experiment. In the experiment, an H* pitch-accented or a deaccented word fragment (e.g., AR- in the word archive) was presented at the end of a carrier sentence for identification. The results of the experiment revealed differences in the cue weighting of English lexical stress perception between native and non-native listeners. For native English listeners, vowel quality was a more prominent cue than f0, while native Mandarin Chinese listeners employed both vowel quality and f0 in a comparable fashion. These results suggested that (a) vowel reduction is superior to f0 in signaling initial stress in the words and (b) f0 facilitates the recognition of word initial stress, which is modulated by first language.

Discrimination of Degrees of Foreign Accent across Different Speakers

Second-language learners often encounter communication challenges due to a foreign accent (FA) in their speech, influenced by their native language (L1). This FA can affect rhythm, intonation, stress, and the segmental domain, which consists of individual language sounds. This study looks into the segmental FA aspect, exploring listeners’ perceptions when Spanish interacts with English. Utilizing the SIAEW corpus, which replaces segments of English words with anticipated Spanish-accented realizations, we assess the ability of non-native listeners to discriminate degrees of accent across male and female voices. This research aims to determine the impact of voice consistency on detecting accentedness variations, studying participants from Japanese and Spanish. Results show that, while listeners are generally able to discriminate degrees of foreign accent across speakers, some segmental transformations convey a more clear distinction depending on the phonological representations of the native and accented realisations on the listener’s system. Another finding is that listeners tend to better discriminate degrees of accent when words are more native-like sounding.

Lexical Stress Identification in Cochlear Implant-Simulated Speech by Non-Native Listeners.

This study investigates whether a presumed difference in the perceptibility of cues to lexical stress in spectro-temporally degraded simulated cochlear implant (CI) speech affects how listeners weight these cues during a lexical stress identification task, specifically in their non-native language. Previous research suggests that in English, listeners predominantly rely on a reduction in vowel quality as a cue to lexical stress. In Dutch, changes in the fundamental frequency (F0) contour seem to have a greater functional weight than the vowel quality contrast. Generally, non-native listeners use the cue-weighting strategies from their native language in the non-native language. Moreover, few studies have suggested that these cues to lexical stress are differently perceptible in spectro-temporally degraded electric hearing, as CI users appear to make more effective use of changes in vowel quality than of changes in the F0 contour as cues to linguistic phenomena. In this study, native Dutch learners of English identified stressed syllables in CI-simulated and non-CI-simulated Dutch and English words that contained changes in the F0 contour and vowel quality as cues to lexical stress. The results indicate that neither the cue-weighting strategies in the native language nor in the non-native language are influenced by the perceptibility of cues in the spectro-temporally degraded speech signal. These results are in contrast to our expectations based on previous research and support the idea that cue weighting is a flexible and transferable process.

The perception of vowelless words in Tashlhiyt

This study examines the perceptual mechanisms involved in the processing of words without vowels, a lexical form that is common in Tashlhiyt but highly dispreferred cross-linguistically. In Experiment 1, native Tashlhiyt and non-native (English-speaking) listeners completed a paired discrimination task where the middle segment of the different-pair was either a vowel (e.g., fan vs. fin), consonant (e.g., ʁbr vs. ʁdr), or vowelless vs. voweled contrast (e.g., tlf vs. tuf). Experiment 2 was a word-likeness ratings task of tri-segmental nonwords constructed to vary in the sonority of the middle segment. We find that vowelless words containing different types of sonority profiles were generally highly discriminable by both native and non-native listeners. This can be explained by the phonetic and acoustic properties of vowelless words: Since Tashlhiyt exhibits low consonant-to-consonant coarticulation, the presence of robust consonantal cues in the speech signal means that the internal phonological structure of vowelless words is recoverable by listeners. At the same time, word-likeness ratings of nonwords indicated that listeners relied on their native-language experience to judge the wellformedness of new words: Tashlhiyt listeners were most likely to accept obstruent-centered vowelless words; meanwhile, English listeners’ preferences increased with higher sonority values of the word center. Across both experiments, speech style variation provided further evidence as to how the phonetic implementation of vowelless words makes them perceptually stable. Thus, our findings provide an overview of the low-level acoustic-phonetic and higher-level phonological processing mechanisms involved in the perception of vowelless words. Our results can inform understandings of the relationship between language-specific phonetic variation and phonotactic patterns, as well as how auditory processing mechanisms shape phonological typology.

Who said it? Native and non-native listeners' source memory for object-speaker associations

Memory for who said what can help native and non-native listeners identify critical information for use in conversations. In two experiments, source memory for object-speaker associations was tested for listeners varying in language proficiency. Native and non-native participants of English first heard speakers name objects with prototypical color (e.g., broccoli) before they had to indicate who had previously named each object. Speakers either consistently named objects of one color, or they randomly named objects of different colors. While both listener groups associated objects with speakers successfully, in Experiment 1, with black and white images, native listeners showed an advantage in source memory in the color consistent condition over the random condition, while non-native listeners had no such advantage. The finding for non-native listeners was replicated in Experiment 2 with color images. The results confirm the role of source memory representations in communication, but also suggest that encoding object-speaker association in L2 may be less sensitive to consistent patterns in the input.

Performance of single-channel speech enhancement algorithms on Mandarin listeners with different immersion conditions in New Zealand English

Speech enhancement (SE) is a widely used technology to improve the quality and intelligibility of noisy speech. So far, SE algorithms were designed and evaluated on native listeners only, but not on non-native listeners who are known to be more disadvantaged when listening in noisy environments. This paper investigates the performance of five widely used single-channel SE algorithms on early-immersed New Zealand English (NZE) listeners and native Mandarin listeners with different immersion conditions in NZE under negative input signal-to-noise ratio (SNR) by conducting a subjective listening test in NZE sentences. The performance of the SE algorithms in terms of speech intelligibility in the three participant groups was investigated. The result showed that the early-immersed group always achieved the highest intelligibility. The late-immersed group outperformed the non-immersed group for higher input SNR conditions, possibly due to the increasing familiarity with the NZE accent, whereas this advantage disappeared at the lowest tested input SNR conditions. The SE algorithms tested in this study failed to improve and rather degraded the speech intelligibility, indicating that these SE algorithms may not be able to reduce the perception gap between early-, late- and non-immersed listeners, nor able to improve the speech intelligibility under negative input SNR in general. These findings have implications for the future development of SE algorithms tailored to Mandarin listeners, and for understanding the impact of language immersion on speech perception in noise.

Influence of Test Room Acoustics on Non-Native Listeners’ Standardized Test Performance

Understanding the impact of room acoustics on non-native listeners is crucial, particularly in standardized English as a foreign language (EFL) proficiency testing environments. This study aims to elucidate how acoustics influence test scores, considering variables overlooked in prior research such as seat position and baseline language proficiency. In this experiment, 42 Japanese university students’ performance on standardized EFL listening tests was assessed in two rooms with distinct acoustic qualities, as determined by the speech transmission index (STI) and reverberation time (RT). The rooms differed significantly in their STI values and RT measurements, with one exhibiting high speech intelligibility qualities of ≥0.66 STI and RT0.5–2kHz < 0.7 s and the other falling below these benchmarks. The findings revealed that listening test scores were consistently higher in the acoustically favorable room across all participants. Notably, the negative effect of poor acoustics was more pronounced for students with lower baseline language proficiency. No significant score differences were observed between front- and rear-seat positions, suggesting that overall room acoustics may be more influential than individual seating locations. The study concludes that acoustics play a significant role in the standardized EFL test performance, particularly for lower-proficiency learners. This highlights the necessity of standardized testing environments to be more carefully selected in order to ensure the fair and reliable assessment of language proficiency.

Performance of speech enhancement algorithms on the speech intelligibility of native Mandarin listeners immersed in English-speaking environment

Speech enhancement algorithms have been developed to improve speech intelligibility for listeners under noisy conditions. However, all existing algorithms were evaluated only by native listeners, the performance of such algorithms on non-native listeners was rarely investigated. This study conducts a subjective listening test on native New Zealand English listeners and native Mandarin listeners who have been immersed in New Zealand English-speaking environment for more than one year. The participants were asked to transcribe noisy English sentences processed by five widely used single-channel speech enhancement algorithms. The speech intelligibility of the two groups was quantified and compared to investigate the effectiveness of the speech enhancement algorithms on non-native listeners who are familiar with the target language.

Effect of room acoustics on speech intelligibility under noise between native and non-native listeners

The current study investigates the effect of room acoustics on the perception of speech between native and non-native listeners of English. The study particularly focuses on how the benefit of spatial release from masking (SRM) is affected by room acoustics for the different listener groups. The result of subjective listening test shows that SRM is observed both from native and non-native listeners, however the drop in the speech intelligibility is more significant in the non-native listeners when the sound sources are located farther away from the listener.

When speaking clearly does not enhance comprehension: Comparing intelligibility of hard-of-hearing- and non-native-directed speech for native and non-native listeners

While clear speech is more intelligible than casual speech, some prior work indicates that the clear speech benefit is reduced for non-native listeners. It is unclear, however, how intelligibility for native and non-native listeners might differ across clear styles directed towards different imagined interlocutor types. If clear speech enhancements benefit the intended listeners, then (1) for non-native listeners, non-native-directed speech should be more intelligible than hard-of-hearing-directed speech; (2) native and non-native listeners should benefit equally from non-native-directed speech, while the advantage of hard-of-hearing-directed speech should be greater for native listeners. Native English speakers were recorded producing casual, hard-of-hearing, and non-native-directed speech to imagined interlocutors. Results from a speech-perception-in-noise task indicate that (1) native listeners had higher transcription accuracy than non-native listeners; (2) hard-of-hearing-directed speech was more intelligible than both non-native-directed and casual speech; (3) non-native-directed and casual speech did not differ in intelligibility; and (4) native and non-native listeners did not differ in the relative benefit of hard-of-hearing-directed speech. Contra our hypothesis, non-native-directed speech did not benefit non-native listeners despite being hyper-articulated compared to casual speech. This study highlights that speakers’ expectations of what will be helpful for listeners do not always provide the intended benefit.

The impact of speech type on listening effort and intelligibility for native and non-native listeners.

Listeners are routinely exposed to many different types of speech, including artificially-enhanced and synthetic speech, styles which deviate to a greater or lesser extent from naturally-spoken exemplars. While the impact of differing speech types on intelligibility is well-studied, it is less clear how such types affect cognitive processing demands, and in particular whether those speech forms with the greatest intelligibility in noise have a commensurately lower listening effort. The current study measured intelligibility, self-reported listening effort, and a pupillometry-based measure of cognitive load for four distinct types of speech: (i) plain i.e. natural unmodified speech; (ii) Lombard speech, a naturally-enhanced form which occurs when speaking in the presence of noise; (iii) artificially-enhanced speech which involves spectral shaping and dynamic range compression; and (iv) speech synthesized from text. In the first experiment a cohort of 26 native listeners responded to the four speech types in three levels of speech-shaped noise. In a second experiment, 31 non-native listeners underwent the same procedure at more favorable signal-to-noise ratios, chosen since second language listening in noise has a more detrimental effect on intelligibility than listening in a first language. For both native and non-native listeners, artificially-enhanced speech was the most intelligible and led to the lowest subjective effort ratings, while the reverse was true for synthetic speech. However, pupil data suggested that Lombard speech elicited the lowest processing demands overall. These outcomes indicate that the relationship between intelligibility and cognitive processing demands is not a simple inverse, but is mediated by speech type. The findings of the current study motivate the search for speech modification algorithms that are optimized for both intelligibility and listening effort.

Second Language Learners Listening to their Peers: Is There a Shared L1 Effect for L2 Comprehensibility and Accentedness?

Abstract Previous research on the perception of second-language speech has suggested that non-native listeners may benefit from sharing the speakers’ first language, e.g. speakers of Spanish find Spanish-accented English easy to understand. In the present study, L2 English speech samples elicited from L1 Finnish and L1 Finland-Swedish speakers were rated for comprehensibility and accentedness by English-speaking listeners and the speakers’ peers, seeking to explore whether there is a difference between the listener groups. In addition, the speakers’ overall spoken proficiency (A2, B1, B2 on the CEFR scale) was considered in the analysis to find out if the possible shared L1 effect is connected to the speakers’ proficiency. The results were mixed, finding that L1 Finland-Swedish listeners were more lenient towards their peers’ English than the English-speaking listeners were, whereas L1 Finnish listeners gave comprehensibility ratings equal to those given by English-speaking listeners, and accentedness ratings that were stricter. The finding supports earlier suggestions on the effects of sharing the speakers’ L1 for L2 speech perception being L1 dependent. As for the influence of the speakers’ proficiency, the results demonstrate a greater difference between English-speaking listeners and listeners who share the speakers’ L1 regarding low-proficiency speakers.

Perception of Dutch vowels by Cypriot Greek listeners: To what extent can listeners’ patterns be predicted by acoustic and perceptual similarity?

There have been numerous studies investigating the perception of non-native sounds by listeners with different first language (L1) backgrounds. However, research needs to expand to under-researched languages and incorporate predictions conducted under the assumptions of new speech models. This study aimed to investigate the perception of Dutch vowels by Cypriot Greek adult listeners and test the predictions of cross-linguistic acoustic and perceptual similarity. The predictions of acoustic similarity were formed using a machine-learning algorithm. Listeners completed a classification test, which served as the baseline for developing the predictions of perceptual similarity by employing the framework of the Universal Perceptual Model (UPM), and an AXB discrimination test; the latter allowed the evaluation of both acoustic and perceptual predictions. The findings indicated that listeners classified each non-native vowel as one or more L1 vowels, while the discrimination accuracy over the non-native contrasts was moderate. In addition, cross-linguistic acoustic similarity predicted to a large extent the classification of non-native sounds in terms of L1 categories and both the acoustic and perceptual similarity predicted the discrimination accuracy of all contrasts. Being in line with prior findings, these findings demonstrate that acoustic and perceptual cues are reliable predictors of non-native contrast discrimination and that the UPM model can make accurate estimations for the discrimination patterns of non-native listeners.

Visual-auditory perception of prosodic focus in Japanese by native and non-native speakers.

Speech communication is multi-sensory in nature. Seeing a speaker's head and face movements may significantly influence the listeners' speech processing, especially when the auditory information is not clear enough. However, research on the visual-auditory integration speech processing has left prosodic perception less well investigated than segmental perception. Furthermore, while native Japanese speakers tend to use less visual cues in segmental perception than in other western languages, to what extent the visual cues are used in Japanese focus perception by the native and non-native listeners remains unknown. To fill in these gaps, we test focus perception in Japanese among native Japanese speakers and Cantonese speakers who learn Japanese, using auditory-only and auditory-visual sentences as stimuli. Thirty native Tokyo Japanese speakers and thirty Cantonese-speaking Japanese learners who had passed the Japanese-Language Proficiency Test with level N2 or N3 were asked to judge the naturalness of 28 question-answer pairs made up of broad focus eliciting questions and three-word answers carrying broad focus, or contrastive or non-contrastive narrow focus on the middle object words. Question-answer pairs were presented in two sensory modalities, auditory-only and visual-auditory modalities in two separate experimental sessions. Both the Japanese and Cantonese groups showed weak integration of visual cues in the judgement of naturalness. Visual-auditory modality only significantly influenced Japanese participants' perception when the questions and answers were mismatched, but when the answers carried non-contrastive narrow focus, the visual cues impeded rather than facilitated their judgement. Also, the influences of specific visual cues like the displacement of eyebrows or head movements of both Japanese and Cantonese participants' responses were only significant when the questions and answers were mismatched. While Japanese participants consistently relied on the left eyebrow for focus perception, the Cantonese participants referred to head movements more often. The lack of visual-auditory integration in Japanese speaking population found in segmental perception also exist in prosodic perception of focus. Not much foreign language effects has been found among the Cantonese-speaking learners either, suggesting a limited use of facial expressions in focus marking by native and non-native Japanese speakers. Overall, the present findings indicate that the integration of visual cues in perception of focus may be specific to languages rather than universal, adding to our understanding of multisensory speech perception.

Non-native listeners perceiving foreign accentedness of English speech: new evidence from Chinese and Spanish listeners

As an international language, English is an important tool for people of different native languages to communicate with each other. Many empirical studies, proving that foreign accents are prevalent in L2 learners’ oral English, involved native speakers as raters of foreign accents. This study compared the rating patterns of listeners from Chinese, Spanish and English language backgrounds with different English proficiencies for foreign accents in English speech and investigated the acoustical cues used by these listeners. Results showed that Chinese listeners, sharing language backgrounds with talkers, tended to be more tolerant for the Chinese accent in English speech. Spanish listeners had a stricter criterion for mild Chinese accents, proving that the influence of listeners’ L1 on their perception of foreign accents can be different for different degrees of accents. The effects of listeners’ L2 proficiency was found for Chinese listeners’ perception of accents and the number of acoustic predictors used by non-native English listeners. The study provides more evidence about the mechanism of native and non-native listeners’ perception of foreign accent and offers implications for the recruitment of raters for the assessment of oral English.