Non-native Listeners Research Articles

Prosodic Focus Interpretation in Spectrotemporally Degraded Speech by Non-Native Listeners.

This study assesses how spectrotemporal degradations that can occur in the sound transmission of a cochlear implant (CI) may influence the ability of non-native listeners to recognize the intended meaning of utterances based on the position of the prosodically focused word. Previous research suggests that perceptual accuracy and listening effort are negatively affected by CI processing (or CI simulations) or when the speech is presented in a non-native language, in a number of tasks and circumstances. How these two factors interact to affect prosodic focus interpretation, however, remains unclear. In an online experiment, normal-hearing (NH) adolescent and adult native Dutch learners of English and a small control group of NH native English adolescents listened to CI-simulated (eight-channel noise-band vocoded) and non-CI-simulated English sentences differing in prosodically marked focus. For assessing perceptual accuracy, listeners had to indicate which of four possible context questions the speaker answered. For assessing listening effort, a dual-task paradigm was used with a secondary free recall task. The results indicated that prosodic focus interpretation was significantly less accurate in the CI-simulated condition compared with the non-CI-simulated condition but that listening effort was not increased. Moreover, there was no interaction between the influence of the degraded CI-simulated speech signal and listening groups in either their perceptual accuracy or listening effort. Non-native listeners are not more strongly affected by spectrotemporal degradations than native listeners, and less proficient non-native listeners are not more strongly affected by these degradations than more proficient non-native listeners.

The role of lexical context and language experience in the perception of foreign-accented segments

Abstract When faced with intelligibility problems, listeners resort to contextual information. The present study explores the use of lexical context by listeners when identifying segments with various degrees of foreign accent. Native English listeners identified words into which a single Spanish-accented segment from a 5-step continuum had been inserted. Listeners also identified vowel-consonant or consonant-vowel sequences containing the same accented segments. While lexical context helped, the lexical advantage was largely independent of degree of foreign accent, with a slight benefit only for the most accented consonants. To examine the influence of listeners’ first language on the usefulness of lexical context, a second experiment was carried out with Spanish, Japanese and Czech non-native listeners. As was the case for native listeners, there was little evidence that a lexical context helps more for foreign-accented than native segments. Normalised for word familiarity, overall non-native identification patterns were comparable to native listeners’ perceptions. Listeners’ first language phonetic inventory had an effect on identification levels, particularly in the case of vowels. Lexical context benefits for vowel identification can be explained by their generally less categorical processing, their realisational variability in English, and symbol mapping issues.

Perception and production of Mandarin-Accented English: The effect of degree of Accentedness on the Interlanguage Speech Intelligibility Benefit for Listeners (ISIB-L) and Talkers (ISIB-T)

Previous research on the Interlanguage Speech Intelligibility Benefit (ISIB) indicates nonnative listeners may have an advantage at understanding nonnative speech of talkers with the same first language (L1) due to shared interlanguage knowledge. The present study offers a comprehensive analysis of various factors that may modulate this advantage, including the proficiency of both the listeners and the talkers, the mapping of phonemes between the L1 and second language (L2), and the acoustic properties of the phones. Accuracy scores on a lexical decision task were used to investigate both native English listeners’ and native Mandarin learners’ of English perception of native English and Mandarin-accented English speech. Results show clear ISIB-L and ISIB-T effects and demonstrate the dynamic nature of ISIB effects, with both being modulated by speaker and listener proficiency. More striking ISIB effects typically occur at the most extreme ends of accentedness. Additionally, an advantage for common-phoneme over unique-phoneme words in nonnative speech was observed. While nonnative productions of common-phoneme words are more accurate than those of unique-phoneme words, for the most accented productions, nonnative listeners are faster to respond to these unique, often mispronounced, productions.The nonnative listener advantage at perceiving nonnative speech depends on various factors, including listener proficiency, speaker proficiency, phoneme characteristics, and the acoustics of specific speech tokens.

Co-text, context, and listening proficiency as crucial variables in intelligibility among nonnative users of English

AbstractResearch on intelligibility in international encounters has long focused on issues of pronunciation to the detriment of factors such as linguistic co-text and extralinguistic context, which are comparatively well-studied variables in intelligibility research concerning L1 listeners. This paper seeks to expand the scope of international intelligibility research in this respect by reporting on a large-scale study involving 423 nonnative listeners at different proficiency levels, who transcribed words spoken with another nonnative accent under four conditions that varied in the availability of syntactic, semantic, and schematic cues. The results suggest that co-text and context as well as listening proficiency are crucial variables that ought to receive greater attention in research on international intelligibility. The pedagogical implications of these findings are addressed as well.

Characterization of the Intelligibility of Vowel–Consonant–Vowel (VCV) Recordings in Five Languages for Application in Speech-in-Noise Screening in Multilingual Settings

The purpose of this study is to characterize the intelligibility of a corpus of Vowel–Consonant–Vowel (VCV) stimuli recorded in five languages (English, French, German, Italian and Portuguese) in order to identify a subset of stimuli for screening individuals of unknown language during speech-in-noise tests. The intelligibility of VCV stimuli was estimated by combining the psychometric functions derived from the Short-Time Objective Intelligibility (STOI) measure with those derived from listening tests. To compensate for the potential increase in speech recognition effort in non-native listeners, stimuli were selected based on three criteria: (i) higher intelligibility; (ii) lower variability of intelligibility; and (iii) shallower psychometric function. The observed intelligibility estimates show that the three criteria for application in multilingual settings were fulfilled by the set of VCVs in English (average intelligibility from 1% to 8% higher; SRT from 4.01 to 2.04 dB SNR lower; average variability up to four times lower; slope from 0.35 to 0.68%/dB SNR lower). Further research is needed to characterize the intelligibility of these stimuli in a large sample of non-native listeners with varying degrees of hearing loss and to determine the possible effects of hearing loss and native language on VCV recognition.

Language proficiency modulates listeners’ selective attention to a talker’s mouth: A conceptual replication of Birulés et al. (2020)

AbstractThis study presents a conceptual replication of Birulés et al.’s (2020, Experiment 2) investigation of native and nonnative listeners’ selective attention to a talker’s mouth with the goal of better understanding the potentially modulating role of proficiency in listeners’ reliance on audiovisual speech cues. Listeners’ eye gaze was recorded while watching two short videos. Findings from one of the videos replicated results from the original study, showing greater attention to the talker’s mouth among L2 than L1 listeners. In both videos, L2 proficiency modulated attention, with more fixations on the mouth among lower proficiency listeners, an effect predicted but not observed in the original study. Collectively, these laboratory-based findings highlight the role of visual speech cues in L2 listening and present evidence that listeners with more limited proficiency may be especially reliant on such cues. These observations warrant future investigations of the benefits of visual speech cues in instructional and assessment contexts.

Perceptual effects of formant enhancement with the factors of phonetic type, listening conditions, and language experience of listeners

The second formant (F2) enhancement is a technique that aims to improve speech perception in adverse noise by amplifying the F2 of speech signals. The current study was to investigate whether F2 enhancement would improve speech identification with the factors of phonetic type (e.g., vowel and consonant), listening conditions (e.g., speech and nonspeech noise at moderately and challenging SNRs), and language experience of listeners (e.g., native and nonnative listeners), and whether the amount of perceptual benefit was dependent on these factors. Two groups of participants, English native and nonnative listeners, were recruited in this study. Identification of English vowels and consonants with and without F2 enhancement were measured in quiet, long-term speech shaped noise (LTSSN) and six-talker babble (6-TB) at the signal-to-noise ratios (SNRs) of −10 dB and −15 dB. Overall, significant improvements from F2 enhancement were found in both vowel and consonant identification for both native and nonnative listeners in various listening conditions. Furthermore, greater improvement was found at the SNR of -15 dB than at the SNR of −10 dB, as well as for nonnative listeners than native listeners in vowel identification. Meanwhile, the amount of benefit was generally comparable speech and nonspeech noise. These results indicate that F2 enhancement could improve phonetic identification in noise for native and nonnative listeners, showing a potential as a speech enhancement algorithm in challenging noise.

Effects of auditory context on nonnative recognition of reduced speech: Does meaning explain it all?

Previous studies have shown that acoustic cues in the sentential context improve native (L1) listeners’ recognition of reduced speech (Janse & Ernestus, 2011). That is, when hearing a reduced target in casual speech (e.g., too into /twntə/), L1 listeners would be more accurate in recognizing it in its surrounding speech (e.g., she’s too into computers) than if they merely see the text of its context (e.g., she’s ____ computers), even though semantic/syntactic information is equivalent in both conditions. It is unclear, however, whether nonnative (L2) listeners make use of the acoustic cues in the context to facilitate reduced speech recognition to the same extent as L1 listeners do. This study investigates this question by comparing L1 and L2 listeners of English in their recognition of reduced words that were aurally presented in three conditions—Isolation, Textual (with surrounding text given), and Auditory (with both text and auditory forms of context)—in a repetition task. Results showed that L2 listeners were significantly more accurate in the Auditory than Textual condition, and such difference was not distinguishable from L1 listeners. These findings lend support to the hypothesis that the facilitative effects of context acoustic cues are language general.

Multilingual spoken word recognition: A megastudy approach

Bilingualism research has primarily focused on the perception and processing of individual sounds or word learning. Many studies have investigated how bilingual listeners perceive sound contrasts that don't create lexical distinctions in their native language. There is substantially less research that has investigated how word-level properties impact L2 auditory processing. The present study examines how auditory lexical processing differs between monolingual and bilingual listeners with different language backgrounds. We collected lexical decision accuracies and latencies for 26,793 words and 9600 pseudowords from the Massive Auditory Lexical Decision database from native and non-native listener responses. We compare the language backgrounds of our 1028 listeners and group them into four groups: native speakers, early, early-late, and late bilinguals. We report the findings of an analysis investigating how language background and proficiency modulate lexical effects to understand how language background influences spoken word recognition. We find differences in the effects of lexical frequency, phonological neighborhood density, and phonological uniqueness point between the different listener groups. We discuss the impact of bilingualism in a word recognition task and how these results inform models of spoken word recognition and second language acquisition.

Non-native talkers and listeners and the perceptual benefits of clear speech.

This study examined the intelligibility benefit of native and non-native clear speech for native and non-native listeners when the first language background of non-native talkers and listeners is matched. All four combinations of talkers and listeners were tested: native talker-native listener, non-native talker-native listener, native talker-non-native listener, and non-native talker-non-native listener. Listeners were presented with structurally simple but semantically anomalous English sentences produced clearly or casually and mixed with speech-shaped noise at 0 dB signal-to-noise ratio and asked to write down what they heard. Results showed that native English speech was more intelligible than non-native speech and that native English listeners recovered speech better than non-native listeners did. Clear speech was significantly more intelligible than casual speech. There were no interactions between speaking style and native language background indicating that clear speech intelligibility benefit was not significantly different across distinct combinations of talkers and listeners. In particular, shared first language background among non-native speakers and listeners did not provide an extra benefit in either the overall speech intelligibility or the intelligibility gains of clear speech.

Recognizing non-native spoken words in background noise increases interference from the native language.

Listeners frequently recognize spoken words in the presence of background noise. Previous research has shown that noise reduces phoneme intelligibility and hampers spoken-word recognition - especially for non-native listeners. In the present study, we investigated how noise influences lexical competition in both the non-native and the native language, reflecting the degree to which both languages are co-activated. We recorded the eye movements of native Dutch participants as they listened to English sentences containing a target word while looking at displays containing four objects. On target-present trials, the visual referent depicting the target word was present, along with three unrelated distractors. On target-absent trials, the target object (e.g., wizard) was absent. Instead, the display contained an English competitor, overlapping with the English target in phonological onset (e.g., window), a Dutch competitor, overlapping with the English target in phonological onset (e.g., wimpel, pennant), and two unrelated distractors. Half of the sentences was masked by speech-shaped noise; the other half was presented in quiet. Compared to speech in quiet, noise delayed fixations to the target objects on target-present trials. For target-absent trials, we observed that the likelihood for fixation biases towards the English and Dutch onset competitors (over the unrelated distractors) was larger in noise than in quiet. Our data thus show that the presence of background noise increases lexical competition in the task-relevant non-native (English) and in the task-irrelevant native (Dutch) language. The latter reflects stronger interference of one's native language during non-native spoken-word recognition under adverse conditions.

Assessment of speech transmission index and reverberation time in standardized English as a foreign language test rooms

Today, millions of standardized English as a foreign language proficiency tests are administered globally each year. A large portion of this is conducted as a paper-based test in which the listening section is commonly delivered through loudspeakers to groups of test takers, a method in which the audio signals are exposed to the acoustic tendencies of each particular venue. As it is well-established in the literature that non-native listeners are more susceptible to adverse listening conditions compared to their native counterparts, there is a need for an objective examination of the acoustic quality of such environments. This study examined the speech transmission index for public address systems (STIPA) for three types of sound sources (wall-mounted speakers, radio cassette player, and amplified speaker) and reverberation time (RT) in 10 unoccupied classrooms commonly used as test rooms at a university in Japan. The results revealed that STI was found to be statistically significantly different for the amplified speaker compared to both or one other sound source in eight out of 10 rooms. The amplified speaker also recorded the highest STI among the three sound sources in eight out of 10 rooms and the most rooms with STI entirely above 0.66, a minimum target value prescribed in IEC 60268–16:2020 as exhibiting high speech intelligibility. Additionally, ≥ 0.66 STI was consistently observed in rooms with RT0.5-2kHz≤ 0.7 s. Further observations are discussed to better understand the current conditions under which these tests are administered

Performance evaluation of speech masking design among listeners with varying language backgrounds

Speech masking is a technique used for protecting speech confidentiality, which is realised by adding maskers to cover target speech. Numerous studies have demonstrated the native and non-native differences in perceiving masked speech, but few have compared the effect of masker types from the perspective of effectiveness of speech privacy and their annoyance levels. This paper reports on the result of an English word perception task by native and non-native listeners where six types of maskers, both speech-like and non-speech-like, are implemented, as well as the annoyance levels of each masker. Natives, non-natives residing in an English-speaking country, and non-natives residing in a non-English-speaking country were tasked to write down sentences when masking sound was present, and also to rate the annoyance levels of the maskers. Results showed that language background significantly affected the performance of unmasked speech, the difference between the two non-native listeners became minimal in the presence of maskers, and maskers annoyed the listener groups differently, calling for a discussion in the selection of a universally effective masker to increase speech privacy.

Vowel convergence does not affect auditory speaker discriminability in humans and machine in a case study on Swiss German dialects

In this study, we examined whether the convergence in interlocutors’ vowel acoustics leads to decreasing discriminability between interlocutors’ voices. Ten pairs of Grison and Zürich German speakers produced lexical items before and after dialogue interactions with evidence of vowel convergence in post-dialogue productions. In Experiment 1, native and non-native Swiss German listeners discriminated pairs of speakers whose speech was obtained pre- and post-dialogue. Results showed that listeners’ sensitivity (A’) was higher for native than non-native listeners, but comparable for pre- and post-dialogue recordings. The observed negative correlation between voice discrimination and the acoustic distance in formant space was mainly driven by a single speaker pair. In Experiment 2, the speaker recognition performance of an i-vector-based software was compared in pre- and post-dialogue speech. Results revealed no difference in the system performance between the two conditions. The findings suggest that vowel convergence does not compromise voice discriminability under the given experimental conditions.

Perceptual effects of formant enhancement with the factors of phonetic type, listening conditions, and language experience of listeners

The second formant (F2) enhancement is a technique that aims to improve speech perception in adverse noise by amplifying the F2 of speech signals. The current study was to investigate whether F2 enhancement would improve speech identification with the factors of phonetic type (e.g., vowel and consonant), listening conditions (e.g., speech and nonspeech noise at moderately and challenging SNRs), and language experience of listeners (e.g., native and nonnative listeners), whether the amount of perceptual benefit was dependent on these factors. Two groups of participants, English native and nonnative listeners, were recruited in this study. Identification of English vowels and consonants with and without F2 enhancement were measured in quiet, long-term speech-shaped noise (LTSSN) and six-talker babble (6-TB) at the signal-to-noise ratios (SNRs) of −10 dB and −15 dB. Overall, significant improvements from F2 enhancement were found in both vowel and consonant identification for both native and nonnative listeners in various listening conditions. Furthermore, greater improvement was found at the SNR of −15 dB than at the SNR of −10 dB, as well as for nonnative listeners than native listeners in vowel identification. Meanwhile, the amount of benefit was generally comparable in speech and nonspeech noise. These results indicate that F2 enhancement could improve phonetic identification in noise for native and nonnative listeners, showing a potential as a speech enhancement algorithm in challenging noise.

Effects of semantic predictability and speaking style on phonetic variation

Many linguistic factors contribute to phonetic reduction in «easy» linguistic contexts and phonetic enhancement in «hard» linguistic contexts. However, previous findings are mixed as to whether interactions among these factors are simply additive and as to whether their effects are differentially exhibited across acoustic domains. To better understand phonetic variation processes, we explored the interaction between two factors, semantic predictability and listener-driven speaking style, on phonetic variation in duration, f0, and vowel formant frequencies. Talkers read aloud sentences differing in the semantic predictability of the sentence-final word (high versus low predictability) to purported listeners with different linguistic competence (a native listener versus a non-native listener with high proficiency versus a non-native listener with low proficiency). Talkers produced phonetic enhancement for less predictable words and for the low-proficiency non-native listener relative to the native and high-proficiency non-native listeners across measures, but the effects of predictability and speaking style did not interact. Together, these results suggest that phonetic variation due to semantic predictability and speaking style may be additive, independent processes, rather than the result of a single listener-oriented process.

Linguistic fine-tuning in the spectra of the frequency-following responses to Mandarin tones

Our understanding of the meaningful linguistic information encoded in neural signals such as FFR is still rather limited. Previous FFR tone studies mostly have focused on F0, and it is unclear how the other rich phonetic cues are encoded in the spectra. In this study, we analyzed the spectra of the frequency following responses to four Mandarin lexical tones from 13 native Mandarin listeners and 22 native English listeners. The stimuli consisted of four 250 ms synthetic syllable /yi/ with Mandarin four lexical tones minimally distinguished by F0 contours. In general, the harmonic structure of the FFR signals for native Mandarin listeners is much better defined than for non-native listeners, and the advantage is especially clear for higher-frequency harmonics. This result suggests that native listeners can hear the pitch contours more clearly and precisely not only because of a clearer f0 representation but also because of better-defined overtones. Moreover, for native listeners, harmonic representation varies by tonal targets. For a higher-pitched tone (Tone 1), H2 is the most prominent harmonic; for a lower-pitched tone (e.g., Tone 3 and Tone 2), H1 is the most prominent harmonic. These results suggest that pitch tracking strategies may vary by tonal targets.

Assimilation patterns predict L2 identification accuracy of English initial consonants

Abstract A large range of studies has generated predictions of second language (L2) speech learning problems from perceptual assimilation patterns of nonnative sounds to native (L1) categories. The present study extends our knowledge of assimilation patterns as a predictor of L2 speech perception by examining the relationship between perceptual assimilation and L2 identification. Specifically, we examine how nonnative listeners’ perceptual assimilation of the full range of English initial consonants predicts their identification accuracy. Four groups of non-native listeners, differing orthogonally with regard to their L1 (Danish, Finnish) and English-language immersion experience (immersion, non-immersion) participated. As predicted, assimilation patterns interacted with L2 experience in accounting for how the L2 learners identified English consonants for some assimilation types, while the predictions were less successful for other assimilation types.

The effect of simultaneous exposure on the attention selection and integration of segments and lexical tones by Urdu-Cantonese bilingual speakers.

In the perceptual learning of lexical tones, an automatic and robust attention-to-phonology system enables native tonal listeners to adapt to acoustically non-optimal speech, such as phonetic conflicts in daily communications. Previous tone research reveals that non-native listeners who do not linguistically employ lexical tones in their mother tongue may find it challenging to attend to the tonal dimension or integrate it with the segmental features. However, it is unknown whether the attentional interference initially caused by a maternal attentional system would continue influencing the non-optimal tone perception for simultaneous bilingual teenagers. From an endpoint in the age of language acquisition, we investigate whether the tone-specific attention mechanism developed by the Urdu-Cantonese simultaneous bilinguals is automatic enough to assist them in adapting to a phonetically-conflicting environment. Three groups of teenagers engaged in a four-condition ABX task: Urdu-Cantonese simultaneous bilinguals, Cantonese native listeners, and Urdu-speaking, late learners of Cantonese. The results showed that although the simultaneous bilinguals could phonologically process Cantonese tones in a Cantonese-like way under a conflict-free listening condition, they still failed in adapting to the phonetic conflicts, especially the segment-induced ones. It thus demonstrated that the simultaneous exposure and years of regular education in Hong Kong local schools still could not automatically guarantee simultaneous bilingual processing of Cantonese tones. In interpreting the findings, it hypothesized that, except for simultaneous exposure, the development of a tone-specific attention mechanism is also likely to be L1-inhibitory, tone experience-driven, and language-specific for simultaneous bilinguals.

Are Serbian and English listeners insensitive to lexical pitch accents in Serbian?

The paper investigated possible perceptual insensitivity effects in the perception of lexical pitch accents by native and non-native listeners, that is, by Serbian and English listeners, respectively. The objective of the study was to explore which word-prosodic categories listeners used when they were required to contrast and recall sequences of lexical pitch accents. To that effect, Serbian and English listeners performed a Sequence Recall Task (SRT) in which they contrasted pairs of non-words with different Serbian lexical pitch accent types, and recalled the sequences of these non-words under different memory load conditions. Listeners' answers were coded correct and incorrect and the accuracy scores between the groups were compared and analyzed. Both groups had almost identical levels of accuracy and they performed well above chance level on each contrast. Neither group exhibited any effects of perceptual insensitivity to lexical pitch accents. English (non-native) listeners did not differ in their performance from native Serbian listeners, which, contrary to what previous research suggested, implied that one's native language word-prosodic category inventory did not preclude the encoding of non-native word-prosodic categories. Instead, non-native listeners were capable of deploying different prosodic resources such as post-lexical pitch accents to recall the sequences.