Non-native Speech Perception Research Articles

Sketching the Landscape of Speech Perception Research (2000-2020): A Bibliometric Study.

Based on 6,407 speech perception research articles published between 2000 and 2020, a bibliometric analysis was conducted to identify leading countries, research institutes, researchers, research collaboration networks, high impact research articles, central research themes and trends in speech perception research. Analysis of highly cited articles and researchers indicated three foundational theoretical approaches to speech perception, that is the motor theory, the direct realism and the computational approach as well as four non-native speech perception models, that is the Speech Learning Model, the Perceptual Assimilation Model, the Native Language Magnet model, and the Second Language Linguistic Perception model. Citation networks, term frequency analysis and co-word networks revealed several central research topics: audio-visual speech perception, spoken word recognition, bilingual and infant/child speech perception and learning. Two directions for future research were also identified: (1) speech perception by clinical populations, such as hearing loss children with cochlear implants and speech perception across lifespan, including infants and aged population; (2) application of neurocognitive techniques in investigating activation of different brain regions during speech perception. Our bibliometric analysis can facilitate research advancements and future collaborations among linguists, psychologists and brain scientists by offering a bird view of this interdisciplinary field.

Non-native speech perception and the role of phonological memory: A re-examination

Previous laboratory-based research has found phonological memory to significantly affect the perception of non-native speech sounds (Inceoglu, 2019). The present study aims to bolster findings reported in Inceoglu (2019) and provide support for the efficacy of online research by replicating the study’s laboratory-based research in an online setting. Data were collected from 32 native English-speaking learners of French (12 male, 20 female; mean age: 33 years old) online using the Prolific data collection platform (www.prolific.com). The study consisted of two major components: an L2 French nasal vowel identification task (Inceoglu, 2019) and a nonword repetition task designed to assess phonological memory (Anderson, 2012; Grey et al., 2015; Inceoglu, 2019; Kissling, 2014). Results of a simple linear regression revealed that participants with higher scores on the nonword repetition task (indicative of better phonological short-term memory) were significantly likely to perform more target-like on the nasal vowel identification task than those with lower scores (t = 12.57; p < 0.001). These findings not only replicate the original results outlined in Inceoglu (2019), but they also support the efficacy of online research. The latter finding is crucial as research begins to adjust to accommodate a more socially distanced world.

The variably intense vocalizations of affect and emotion (VIVAE) corpus prompts new perspective on nonspeech perception.

The human voice is a potent source of information to signal emotion. Nonspeech vocalizations (e.g., laughter, crying, moans, or screams), in particular, can elicit compelling affective experiences. Consensus exists that the emotional intensity of such expressions matters; however how intensity affects such signals, and their perception remains controversial and poorly understood. One reason is the lack of appropriate data sets. We have developed a comprehensive stimulus set of nonverbal vocalizations, the first corpus to represent emotion intensity from one extreme to the other, in order to resolve the empirically underdetermined basis of emotion intensity. The full set, comprising 1085 stimuli, features eleven speakers expressing 3 positive (achievement/triumph, sexual pleasure, surprise) and 3 negative (anger, fear, physical pain) affective states, each varying from low to peak emotion intensity. The smaller core set of 480 files represents a fully crossed subsample (6 emotions × 4 intensities × 10 speakers × 2 items) selected based on judged authenticity. Perceptual validation and acoustic characterization of the stimuli are provided; the expressed emotional intensity, like expressed emotion, is reflected in listener evaluation and signal properties of nonverbal vocalizations. These carefully curated new materials can help disambiguate foundational questions on the communication of affect and emotion in the psychological and neural sciences and strengthen our theoretical understanding of this domain of emotional experience. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Social attitudes, intelligibility and comprehensibility: The role of the listener in the perception of non-native speech

This study investigates the role of the listener in the perception of non-native speakers and their speech. Our goal is to examine the impact of listener characteristics on their attitudes towards non-native speakers and their speech. In addition, we aim to explore the relationship between listeners’ attitudes and the intelligibility, comprehensibility and perceived foreign accentedness of non-native speech. 126 participants living in Flanders, Belgium, completed a questionnaire probing their attitudes towards non-native speakers of Dutch. An analysis of background variables of these listeners revealed that their age, educational level, extent of contact with non-native speakers and especially political preference could predict their responses to questionnaire items. In a subsequent session, participants performed (a) a speaker/speech evaluation task, (b) a transcription task measuring intelligibility, and (c) a comprehensibility and accentedness rating task. This latter session required participants to transcribe and evaluate speech samples of Dutch produced by speakers of Mandarin Chinese. We found a significant correlation between comprehensibility and accentedness and a number of attitudinal dimensions, such as the perceived status of, and solidarity with, the speaker. The study has implications for language testing, as it demonstrates the impact of listeners’ social attitudes on the assessment of non-native speech.

Overnight changes to dual-memory processes reflected in speech-perceptual performance.

Adults' ability to attain and retain nonnative speech sound categories vary substantially among individuals. While we know that speech-perceptual skills play a role, we know less about how consolidation-related changes in acoustic-phonetic memory contribute to perceptual tasks. The goal of this investigation was to examine contributions of memory and perceptual skills to the perceptual performance on a trained nonnative speech contrast over two days. Twenty-one adult participants (ages 18-24) completed four different experiments. Three of these assessed learning and memory: visual statistical learning (implicit), visual object recognition (explicit), and nonnative (Hindi dental-retroflex) speech-sound training. Participants completed the learning tasks around 8 p.m., and performance was measured shortly after learning and again 12 hours later. On a separate day, participants completed a categorical perception task on a native (/a/-/e/) vowel continuum. Nonnative speech perception was associated with implicit learning performance when both were assessed shortly after learning, and associated with the retention of explicit memory when both were assessed after an overnight delay. Native speech-sounds were at least marginally associated with nonnative speech perception performance on both days, but with a stronger association observed with performance assessed on Day 2. These findings provide preliminary support for the interpretation that speech-sounds are encoded by at least two memory systems in parallel, but that perceptual performance may reflect acoustic-phonetic knowledge learned by different memory systems over time since exposure. Moreover, performance on speech perception tasks in both native and nonnative speech-sounds may rely on similar retrieval mechanisms for long-term storage of speech-sound information.

Supplemental Material for The Variably Intense Vocalizations of Affect and Emotion (VIVAE) Corpus Prompts New Perspective on Nonspeech Perception

Supplemental Material for The Variably Intense Vocalizations of Affect and Emotion (VIVAE) Corpus Prompts New Perspective on Nonspeech Perception

Cross-talker generalization in the perception of nonnative speech: A large-scale replication.

Speech perception depends on the ability to generalize previously experienced input effectively across talkers. How such cross-talker generalization is achieved has remained an open question. In a seminal study, Bradlow & Bent (2008, henceforth BB08) found that exposure to just 5 min of accented speech can elicit improved recognition that generalizes to an unfamiliar talker of the same accent (N = 70 participants). Cross-talker generalization was, however, only observed after exposure to multiple talkers of the accent, not after exposure to a single accented talker. This contrast between single- and multitalker exposure has been highly influential beyond research on speech perception, suggesting a critical role of exposure variability in learning and generalization. We assess the replicability of BB08's findings in two large-scale perception experiments (total N = 640) including 20 unique combinations of exposure and test talkers. Like BB08, we find robust evidence for cross-talker generalization after multitalker exposure. Unlike BB08, we also find evidence for generalization after single-talker exposure. The degree of cross-talker generalization depends on the specific combination of exposure and test talker. This and other recent findings suggest that exposure to cross-talker variability is not necessary for cross-talker generalization. Variability during exposure might affect generalization only indirectly, mediated through the informativeness of exposure about subsequent speech during test: Similarity-based inferences can explain both the original BB08 and the present findings. We present Bayesian data analysis, including Bayesian meta-analyses and replication tests for generalized linear mixed models. All data, stimuli, and reproducible literate (R markdown) code are shared via OSF. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Input quality and speech perception development in bilingual infants' first year of life.

Individual differences in infants' native phonological development have been linked to the quantity and quality of infant-directed speech (IDS). The effects of parental and infant bilingualism on this relation in 131 five- and nine-month-old monolingual and bilingual Spanish and Basque infants (72male; 59 female; from white middle-class background) were investigated. Bilingualism did not affect the developmental trajectory of infants' native and non-native speech perception and the quality of maternal speech. In both language groups, vowel exaggeration in IDS was significantly related to speech perception skills for 9-month-olds (r=-.30), but not for 5-month-olds. This demonstrates that bilingual and monolingual caregivers provide their infants with speech input that assists their task of learning the phonological inventory of one or two languages.

Effect of listeners' language backgrounds on pure tone detection in temporally modulated noise.

Previous studies in our laboratory showed that English-native listeners had greater masking release from the temporal variation in noise than Chinese-native listeners in speech perception tasks. The goal of this study was to examine whether the advantage of temporal dip listening for English-native listeners was also presented in non-speech perception. Thresholds of temporal modulation detection and thresholds of tone detection in stationary and temporally modulated noise were measured for English- and Chinese-native listeners. Results indicated significant group difference in the masking release for tone detection, but with no difference in temporal modulation detection, suggesting the language effects on psychoacoustical temporal processing.

Challenges in the perception of L2 English phonemes by native speakers of Cypriot Greek

Research into non-native speech perception performance suggests that adults encounter difficulties discriminating segmental distinctions that are not used contrastively in their first language. Current theoretical models suggest that these difficulties are related to the acquisition of a native speech system but diverge in whether the perceptual mechanisms operate on acoustic or articulatory information. The present research addresses: (1) the difficulties Cypriot Greek (CGR) listeners of L2 English encounter with English vowels and consonants, and (2) the effect of extra-linguistic (i.e. age, gender, years of learning L2 English, educational setting, age of onset, reported use in the L2, visits to English-speaking countries) and linguistic factors (word frequency, word length, syllable number) on the perception of vowels and consonants in L2 English. A task involving two discrimination tests assessed native CGR listeners’ perception of English contrasts for vowels and consonants. Overall, listeners performed slightly better when dealing with consonants compared to vowels, significant effects have been reported. Concerning the examined factors, age, gender, years of L2 instruction, and years of experience have been reported to have significant effects, especially on speech perception of consonants. Specifically, the findings suggest that acoustic cues such as voicing and vowel length, which are relevant for native speakers of English, may not be for non-native speakers, in this case for CGR L1 speakers.

Training non-native vowel perception: In quiet or noise.

Noise makes speech perception much more challenging for non-native listeners than for native listeners. Training for non-native speech perception is usually implemented in quiet. It remains unclear if background noise may benefit or hamper non-native speech perception learning. In this study, 51 Chinese-native listeners were randomly assigned into three groups, including vowel training in quiet (TIQ), vowel training in noise (TIN), and watching videos in English as an active control. Vowel identification was assessed before (T1), right after (T2), and three months after training (T3) in quiet and various noise conditions. Results indicated that compared with the video watching group, the TIN group improved vowel identification in both quiet and noise significantly more at T2 and at T3. In contrast, the TIQ group improved significantly more in quiet and also in non-speech noise conditions at T2, but the improvement did not hold at T3. Moreover, compared to the TIQ group, the TIN group showed significantly less informational masking at both T2 and T3 and less energetic masking at T3. These results suggest that L2 speech training in background noise may improve non-native vowel perception more effectively than TIQ background only. The implications for non-native speech perception learning are discussed.

Semantic predictability and adaptation to nonnative speech.

Listeners improve their ability to understand nonnative speech through exposure. The present study examines the role of semantic predictability during adaptation. Listeners were trained on high-predictability, low-predictability, or semantically anomalous sentences. Results demonstrate that trained participants improve their perception of nonnative speech compared to untrained participants. Adaptation is most robust for the types of sentences participants heard during training; however, semantic predictability during exposure did not impact the amount of adaptation overall. Results show advantages in adaptation specific to the type of speech material, a finding similar to the specificity of adaptation previously demonstrated for individual talkers or accents.

The Contributions of Crosslinguistic Influence and Individual Differences to Nonnative Speech Perception

Perception of a nonnative language (L2) is known to be affected by crosslinguistic transfer from a listener’s native language (L1), but the relative importance of L1 transfer vis-a-vis individual learner differences remains unclear. This study explored the hypothesis that the nature of L1 transfer changes as learners gain experience with the L2, such that individual differences are more influential at earlier stages of learning and L1 transfer is more influential at later stages of learning. To test this hypothesis, novice L2 learners of Korean from diverse L1 backgrounds were examined in a pretest-posttest design with respect to their perceptual acquisition of novel L2 consonant contrasts (the three-way Korean laryngeal contrast among lenis, fortis, and aspirated plosives) and vowel contrasts (/o/-/ʌ/, /u/-/ɨ/). Whereas pretest performance showed little evidence of L1 effects, posttest performance showed significant L1 transfer. Furthermore, pretest performance did not predict posttest performance. These findings support the view that L1 knowledge influences L2 perception dynamically, according to the amount of L2 knowledge available to learners at that time. That is, both individual differences and L1 knowledge play a role in L2 perception, but to different degrees over the course of L2 development.

Aging and non-native speech perception: A phonetic training study

Cognitive decline is evident in the elderly and it affects speech perception and foreign language learning. A listen-and-repeat training with a challenging speech sound contrast was earlier found to be effective in young monolingual adults and even in advanced L2 university students at the attentive and pre-attentive levels. This study investigates foreign language speech perception in the elderly with the same protocol used with the young adults. Training effects were measured with attentive behavioural measures (N = 9) and with electroencephalography measuring the pre-attentive mismatch negativity (MMN) response (N = 10). Training was effective in identification, but not in discrimination and there were no changes in the MMN. The most attention demanding perceptual functions which benefit from experience-based linguistic knowledge were facilitated through training, whereas pre-attentive processing was unaffected. The elderly would probably benefit from different training types compared to younger adults.

Perceptual consequences of pauses: Credibility and fluency of native and non-native speech

Silent pauses are a natural part of speech and impact perception of speech. However, it is unclear how foreign accentedness influences perception of pauses. Specifically, studies have shown mixed results regarding whether listeners process pauses in native and non-native speech similarly or differently. A possible explanation for these results is that perceptual consequences of pauses differ depending on the type of processing that listeners engage in: a focus on the content/meaning of the speech versus characteristics/form of the speech. Thus, the present study examines the effect of silent pauses on listeners' perception of native and non-native speech in two different tasks, examining perceived credibility of the information conveyed in the speech, and perceived fluency of the speech. Specifically, we ask whether characteristics of silent pauses (e.g., presence or absence of a pause; syntactic location of a pause) influence listeners' perception differently for native versus non-native speech, and whether this pattern differs for perceived credibility and fluency. The results help us better understand how disfluency and foreign accent together impact listeners' perception. Further, this provides insight for how perceptual consequences of certain acoustic properties of the speech differ depending on how listeners are asked to evaluate the speech.

Native phonological and phonetic influences in perceptual assimilation of monosyllabic Thai lexical tones by Mandarin and Vietnamese listeners

A cross tone-language perceptual assimilation study investigated native categorisations and goodness ratings of non-native Thai tones by Thai-naive listeners differing in their native tone systems: Mandarin, Northern Vietnamese and Southern Vietnamese. We derived hypotheses from the Perceptual Assimilation Model (PAM: Best, 1995), which considers both native phonological and phonetic influences on perceptual assimilation of non-native speech contrasts. Mandarin listeners reliably categorised the Thai mid level tone to their single native level tone category, reflecting a native phonological effect, but they also showed high residual phonetic sensitivity to differences between the non-native tone and the native tone it was assimilated to, indicated by low category-goodness ratings. Native phonological and phonetic differences in tones of the two Vietnamese dialects also affected perceptual assimilation of the Thai high level and rising tones. In addition, categorisation responses were faster overall for Categorised than Uncategorised assimilations, revealing the processing cost of perceptual uncertainty due to phonological competition among and/or phonetic discrepancies from multiple native categories. This indicates, furthermore, a more focused and thus stronger native phonological contribution for Categorised than Uncategorised assimilations. PAM principles thus extend to non-native tone assimilations and indicate the importance of both native phonological and phonetic contributions to non-native speech perception.

Emblem Gestures Improve Perception and Evaluation of Non-native Speech.

Traditionally, much of the attention on the communicative effects of non-native accent has focused on the accent itself rather than how it functions within a more natural context. The present study explores how the bodily context of co-speech emblematic gestures affects perceptual and social evaluation of non-native accent. In two experiments in two different languages, Mandarin and Japanese, we filmed learners performing a short utterance in three different within-subjects conditions: speech alone, culturally familiar gesture, and culturally unfamiliar gesture. Native Mandarin participants watched videos of foreign-accented Mandarin speakers (Experiment 1), and native Japanese participants watched videos of foreign-accented Japanese speakers (Experiment 2). Following each video, native language participants were asked a set of questions targeting speech perception and social impressions of the learners. Results from both experiments demonstrate that familiar—and occasionally unfamiliar—emblems facilitated speech perception and enhanced social evaluations compared to the speech alone baseline. The variability in our findings suggests that gesture may serve varied functions in the perception and evaluation of non-native accent.

The role of left vs. right superior temporal gyrus in speech perception: An fMRI-guided TMS study

Debate continues regarding the necessary role of right superior temporal gyrus (STG) regions in sublexical speech perception given the bilateral STG activation often observed in fMRI studies. To evaluate the causal roles, TMS pulses were delivered to inhibit and disrupt neuronal activity at the left and right STG regions during a nonword discrimination task based on peak activations from a blocked fMRI paradigm assessing speech vs. nonspeech perception (N = 20). Relative to a control region located in the posterior occipital lobe, TMS to the left anterior STG (laSTG) led to significantly worse accuracy, whereas TMS to the left posterior STG (lpSTG) and right anterior STG (raSTG) did not. Although the disruption from TMS was significantly greater for the laSTG than for raSTG, the difference in accuracy between the laSTG and lpSTG did not reach significance. The results argue for a causal role of the laSTG but not raSTG in speech perception. Further research is needed to establish the source of the differences between the laSTG and lpSTG.

On the generalization of tones: A detailed exploration of non-speech auditory perception stimuli

The dynamic changes in natural sounds’ temporal structures convey important event-relevant information. However, prominent researchers have previously expressed concern that non-speech auditory perception research disproportionately uses simplistic stimuli lacking the temporal variation found in natural sounds. A growing body of work now demonstrates that some conclusions and models derived from experiments using simplistic tones fail to generalize, raising important questions about the types of stimuli used to assess the auditory system. To explore the issue empirically, we conducted a novel, large-scale survey of non-speech auditory perception research from four prominent journals. A detailed analysis of 1017 experiments from 443 articles reveals that 89% of stimuli employ amplitude envelopes lacking the dynamic variations characteristic of non-speech sounds heard outside the laboratory. Given differences in task outcomes and even the underlying perceptual strategies evoked by dynamic vs. invariant amplitude envelopes, this raises important questions of broad relevance to psychologists and neuroscientists alike. This lack of exploration of a property increasingly recognized as playing a crucial role in perception suggests future research using stimuli with time-varying amplitude envelopes holds significant potential for furthering our understanding of the auditory system’s basic processing capabilities.

Effects of acoustic and semantic cues on listening effort during native and non-native speech perception.

Relative to native listeners, non-native listeners who are immersed in a second language environment experience increased listening effort and a reduced ability to successfully perform an additional task while listening. Previous research demonstrated that listeners can exploit a variety of intelligibility-enhancing cues to cope with adverse listening conditions. However, little is known about the implications of those speech perception strategies for listening effort. The current research aims to investigate by means of pupillometry how listening effort is modulated in native and non-native listeners by the availability of semantic context and acoustic enhancements during the comprehension of spoken sentences. For this purpose, semantic plausibility and speaking style were manipulated both separately and in combination during a speech perception task in noise. The signal to noise ratio was individually adjusted for each participant in order to target 50% intelligibility level. Behavioural results indicated that native and non-native listeners were equally able to fruitfully exploit both semantic and acoustic cues to aid their comprehension. Pupil data indicated that listening effort was reduced for both groups of listeners when acoustic enhancements were available, while the presence of a plausible semantic context did not lead to a reduction in listening effort.