Perception Of Spoken Words Research Articles

Examining listeners' perception of spoken words with different face masks.

The COVID-19 pandemic made face masks part of daily life. While masks protect against the virus, it is important to understand the impact masks have on listeners' recognition of spoken words. We examined spoken word recognition under three different mask conditions (no mask; cloth mask; Kn95 mask) and in both easy (low density, high phonotactic probability) and hard (high density, low phonotactic probability) words in a lexical decision task. In Experiment 1, participants heard all words and nonwords under all three mask conditions. In Experiment 2, participants heard each word and nonword only once under one of the mask conditions. The reaction time and accuracy results were consistent between Experiments 1 and 2. The pattern of results was such that the no mask condition produced the fastest and most accurate responses followed by the Kn95 mask condition and the cloth mask condition, respectively. Furthermore, there was a trend towards a speed-accuracy trade-off with Word Type. Easy words produced faster but less accurate responses relative to hard words. The finding that cloth masks had a more detrimental impact on spoken word recognition than Kn95 masks is consistent with previous research, and the current results further demonstrate that this effect extends to individual word recognition tasks with only audio presentation.

Modulations of right hemisphere connectivity in young children relates to the perception of spoken words

The early school years shape a young brain's capability to comprehend and contextualize words within milliseconds of exposure. Parsing word sounds (phonological interpretation) and word recognition (enabling semantic interpretation) are integral to this process. Yet little is known about the causal mechanisms of cortical activity during these early developmental stages. In this study, we aimed to explore these causal mechanisms via dynamic causal modelling of event-related potentials (ERPs) acquired from 30 typically developing children (ages 6–8 years) as they completed a spoken word-picture matching task. Source reconstruction of high-density electroencephalography (128 channels) was used to ascertain differences in whole-brain cortical activity during semantically “congruent” and “incongruent” conditions. Source activations analyzed during the N400 ERP window identified significant regions-of-interest (pFWE<.05) localized primarily in the right hemisphere when contrasting congruent and incongruent word-picture stimuli. Dynamic causal models (DCMs) were tested on source activations in the fusiform gyrus (rFusi), inferior parietal lobule (rIPL), inferior temporal gyrus (rITG) and superior frontal gyrus (rSFG). DCM results indicated that a fully connected bidirectional model with self-(inhibiting) connections over rFusi, rIPL and rSFG provided the highest model evidence, based on exceedance probabilities derived from Bayesian statistical inferences. Connectivity parameters of rITG and rSFG regions from the winning DCM were negatively correlated with behavioural measures of receptive vocabulary and phonological memory (pFDR<.05), such that lower scores on these assessments corresponded with increased connectivity between temporal pole and anterior frontal regions. The findings suggest that children with lower language processing skills required increased recruitment of right hemisphere frontal/temporal areas during task performance.

Bottom-up and top-down effects on the perception of children's speech

This study examined adult’s perception of spoken words produced by children and by other adults in audio-only (AO) and audiovisual (AV) contexts, to determine whether talker age affects the magnitude of AV benefit. One-syllable words produced by five 4-6 year old child talkers and five adult talkers were presented to 151 adult listeners at three signal-to-noise ratios. The SNRs were determined through pilot testing to be the ones at which adult and child talkers’ productions were recognized equally accurately in the AO condition. The magnitude of AV benefit was statistically equivalent for the adult and child talkers. However, lexical factors affected the perception of children and adults’ speech differently. Word frequency had a larger effect on the perception of children’s speech, perhaps reflecting listeners’ expectation that children would be especially unlikely to use a low frequency word. Lexical factors also affected the nature of listener errors: listeners were less likely to provide a low-frequency word when misperceiving a child than when misperceiving an adult. Listeners were also more likely to guess a word with a different CV structure than the target for children than for adults. Results suggest that listeners apply qualitatively different expectations when listening to adults and children.

Biasing the Perception of Spoken Words with Transcranial Alternating Current Stimulation

Recent neuroimaging evidence suggests that the frequency of entrained oscillations in auditory cortices influences the perceived duration of speech segments, impacting word perception [Kösem, A., Bosker, H. R., Takashima, A., Meyer, A., Jensen, O., & Hagoort, P. Neural entrainment determines the words we hear. Current Biology, 28, 2867-2875, 2018]. We further tested the causal influence of neural entrainment frequency during speech processing, by manipulating entrainment with continuous transcranial alternating current stimulation (tACS) at distinct oscillatory frequencies (3 and 5.5 Hz) above the auditory cortices. Dutch participants listened to speech and were asked to report their percept of a target Dutch word, which contained a vowel with an ambiguous duration. Target words were presented either in isolation (first experiment) or at the end of spoken sentences (second experiment). We predicted that the tACS frequency would influence neural entrainment and therewith how speech is perceptually sampled, leading to a perceptual overestimation or underestimation of the vowel's duration. Whereas results from Experiment 1 did not confirm this prediction, results from Experiment 2 suggested a small effect of tACS frequency on target word perception: Faster tACS leads to more long-vowel word percepts, in line with the previous neuroimaging findings. Importantly, the difference in word perception induced by the different tACS frequencies was significantly larger in Experiment 1 versus Experiment 2, suggesting that the impact of tACS is dependent on the sensory context. tACS may have a stronger effect on spoken word perception when the words are presented in continuous speech as compared to when they are isolated, potentially because prior (stimulus-induced) entrainment of brain oscillations might be a prerequisite for tACS to be effective.

Eye movements reveal fast, voice-specific priming.

In spoken word perception, voice specificity effects are well-documented: When people hear repeated words in some task, performance is generally better when repeated items are presented in their originally heard voices, relative to changed voices. A key theoretical question about voice specificity effects concerns their time-course: Some studies suggest that episodic traces exert their influence late in lexical processing (the time-course hypothesis; McLennan & Luce, 2005), whereas others suggest that episodic traces influence immediate, online processing. We report 2 eye-tracking studies investigating the time-course of voice-specific priming within and across cognitive tasks. In Experiment 1, participants performed modified lexical decision or semantic classification to words spoken by 4 speakers. The tasks required participants to click a red "x" or a blue "+" located randomly within separate visual half-fields, necessitating trial-by-trial visual search with consistent half-field response mapping. After a break, participants completed a second block with new and repeated items, half spoken in changed voices. Voice effects were robust very early, appearing in saccade initiation times. Experiment 2 replicated this pattern while changing tasks across blocks, ruling out a response priming account. In the General Discussion, we address the time-course hypothesis, focusing on the challenge it presents for empirical disconfirmation, and highlighting the broad importance of indexical effects, beyond studies of priming.

Phonological Neighborhood Effects in Spoken Word Perception and Production

Research on spoken word perception and production has identified two hallmarks of spoken word processing: multiple activation of representations of the sound patterns of words in memory and subsequent competition among these patterns. Evidence for this activation-competition process has come, in part, from experimental studies examining the effects of phonological neighborhoods, which are collections of similar-sounding words that are activated in memory during both perception and production. In this article, we review more than 20 years of research on phonological neighborhood effects in spoken word processing that has demonstrated that the speed and accuracy of spoken word perception and production are, in large part, a function of the density and frequency of neighborhoods of spoken words. We conclude our review with a discussion of new avenues of research—based on recent advances in network science—that hold the promise of deepening our understanding of the mental operations involved in our uniquely human capacity for communicating with the spoken word.

Longitudinal performance of spoken word perception in Mandarin pediatric cochlear implant users

ObjectiveThe objective of the present study was to investigate the longitudinal performance on open-set word perception in Mandarin children with cochlear implants (CIs). MethodsProspective cohort study. One hundred and five prelingually deaf children implanted with CIs participated in the study. The Standard-Chinese Version of Monosyllabic Lexical Neighborhood Test (LNT) and Multisyllabic Lexical Neighborhood Test (MLNT) were used as open-set word perception evaluation tools. Evaluations were administrated at 6, 12, 24, 36, 48, 60, 72, and 84 months post CI stimulation, respectively. Results(1) Spoken word perception performance of congenitally deaf children with CIs improved significantly over time. (2) The fastest improvement occurred in the first 36 months after initial activation, then the improvement slowed down and the final peak score of 81.7% correct was achieved at 72 months after initial activation. (3) Early implanted children exhibited better longitudinal performance. (4) Lexical factors affected consistently in each evaluation session. For lexically harder words, such as monosyllabic hard words, there was substantial room for improvement even after long-term use of CI. Conclusions(1) CI continuously provided significant benefits in word perception to children with severe/profound sensorineural hearing loss. (2) Age at implantation and Mandarin lexical factor affected longitudinal performance significantly.

Do We Perceive Others Better than Ourselves? A Perceptual Benefit for Noise-Vocoded Speech Produced by an Average Speaker.

In different tasks involving action perception, performance has been found to be facilitated when the presented stimuli were produced by the participants themselves rather than by another participant. These results suggest that the same mental representations are accessed during both production and perception. However, with regard to spoken word perception, evidence also suggests that listeners’ representations for speech reflect the input from their surrounding linguistic community rather than their own idiosyncratic productions. Furthermore, speech perception is heavily influenced by indexical cues that may lead listeners to frame their interpretations of incoming speech signals with regard to speaker identity. In order to determine whether word recognition evinces similar self-advantages as found in action perception, it was necessary to eliminate indexical cues from the speech signal. We therefore asked participants to identify noise-vocoded versions of Dutch words that were based on either their own recordings or those of a statistically average speaker. The majority of participants were more accurate for the average speaker than for themselves, even after taking into account differences in intelligibility. These results suggest that the speech representations accessed during perception of noise-vocoded speech are more reflective of the input of the speech community, and hence that speech perception is not necessarily based on representations of one’s own speech.

Deaf children with cochlear implants do not appear to use sentence context to help recognize spoken words.

The authors investigated the ability of deaf children with cochlear implants (CIs) to use sentence context to facilitate the perception of spoken words. Deaf children with CIs (n = 24) and an age-matched group of children with normal hearing (n = 31) were presented with lexically controlled sentences and were asked to repeat each sentence in its entirety. Performance was analyzed at each of 3 word positions of each sentence (first, second, and third key word). Whereas the children with normal hearing showed robust effects of contextual facilitation-improved speech perception for the final words in a sentence-the deaf children with CIs on average showed no such facilitation. Regression analyses indicated that for the deaf children with CIs, Forward Digit Span scores significantly predicted accuracy scores for all 3 positions, whereas performance on the Stroop Color and Word Test, Children's Version (Golden, Freshwater, & Golden, 2003) predicted how much contextual facilitation was observed at the final word. The pattern of results suggests that some deaf children with CIs do not use sentence context to improve spoken word recognition. The inability to use sentence context may be due to possible interactions between language experience and cognitive factors that affect the ability to successfully integrate temporal-sequential information in spoken language.

Fronto-temporal connectivity is preserved during sung but not spoken word listening, across the autism spectrum.

Co-occurrence of preserved musical function with language and socio-communicative impairments is a common but understudied feature of Autism Spectrum Disorders (ASD). Given the significant overlap in neural organization of these processes, investigating brain mechanisms underlying speech and music may not only help dissociate the nature of these auditory processes in ASD but also provide a neurobiological basis for development of interventions. Using a passive-listening functional magnetic resonance imaging paradigm with spoken words, sung words and piano tones, we found that 22 children with ASD, with varying levels of functioning, activated bilateral temporal brain networks during sung-word perception, similarly to an age and gender-matched control group. In contrast, spoken-word perception was right-lateralized in ASD and elicited reduced inferior frontal gyrus (IFG) activity which varied as a function of language ability. Diffusion tensor imaging analysis reflected reduced integrity of the left hemisphere fronto-temporal tract in the ASD group and further showed that the hypoactivation in IFG was predicted by integrity of this tract. Subsequent psychophysiological interactions revealed that functional fronto-temporal connectivity, disrupted during spoken-word perception, was preserved during sung-word listening in ASD, suggesting alternate mechanisms of speech and music processing in ASD. Our results thus demonstrate the ability of song to overcome the structural deficit for speech across the autism spectrum and provide a mechanistic basis for efficacy of song-based interventions in ASD.

Lip Reading Without Awareness

Lip Reading Without Awareness

Selective attention to phonology dynamically modulates initial encoding of auditory words within the left hemisphere

Selective attention to phonology, i.e., the ability to attend to sub-syllabic units within spoken words, is a critical precursor to literacy acquisition. Recent functional magnetic resonance imaging evidence has demonstrated that a left-lateralized network of frontal, temporal, and posterior language regions, including the visual word form area, supports this skill. The current event-related potential (ERP) study investigated the temporal dynamics of selective attention to phonology during spoken word perception. We tested the hypothesis that selective attention to phonology dynamically modulates stimulus encoding by recruiting left-lateralized processes specifically while the information critical for performance is unfolding. Selective attention to phonology was captured by manipulating listening goals: skilled adult readers attended to either rhyme or melody within auditory stimulus pairs. Each pair superimposed rhyming and melodic information ensuring identical sensory stimulation. Selective attention to phonology produced distinct early and late topographic ERP effects during stimulus encoding. Data-driven source localization analyses revealed that selective attention to phonology led to significantly greater recruitment of left-lateralized posterior and extensive temporal regions, which was notably concurrent with the rhyme-relevant information within the word. Furthermore, selective attention effects were specific to auditory stimulus encoding and not observed in response to cues, arguing against the notion that they reflect sustained task setting. Collectively, these results demonstrate that selective attention to phonology dynamically engages a left-lateralized network during the critical time-period of perception for achieving phonological analysis goals. These findings suggest a key role for selective attention in on-line phonological computations. Furthermore, these findings motivate future research on the role that neural mechanisms of attention may play in phonological awareness impairments thought to underlie developmental reading disabilities.

Phonological Neighborhood Effects in Spoken Word Perception and Production

Phonological Neighborhood Effects in Spoken Word Perception and Production

The socially weighted encoding of spoken words: a dual-route approach to speech perception

Spoken words are highly variable. A single word may never be uttered the same way twice. As listeners, we regularly encounter speakers of different ages, genders, and accents, increasing the amount of variation we face. How listeners understand spoken words as quickly and adeptly as they do despite this variation remains an issue central to linguistic theory. We propose that learned acoustic patterns are mapped simultaneously to linguistic representations and to social representations. In doing so, we illuminate a paradox that results in the literature from, we argue, the focus on representations and the peripheral treatment of word-level phonetic variation. We consider phonetic variation more fully and highlight a growing body of work that is problematic for current theory: words with different pronunciation variants are recognized equally well in immediate processing tasks, while an atypical, infrequent, but socially idealized form is remembered better in the long-term. We suggest that the perception of spoken words is socially weighted, resulting in sparse, but high-resolution clusters of socially idealized episodes that are robust in immediate processing and are more strongly encoded, predicting memory inequality. Our proposal includes a dual-route approach to speech perception in which listeners map acoustic patterns in speech to linguistic and social representations in tandem. This approach makes novel predictions about the extraction of information from the speech signal, and provides a framework with which we can ask new questions. We propose that language comprehension, broadly, results from the integration of both linguistic and social information.

Famous talker effects in spoken word recognition

Previous work has demonstrated that talker-specific representations affect spoken word recognition relatively late during processing. However, participants in these studies were listening to unfamiliar talkers. In the present research, we used a long-term repetition-priming paradigm and a speeded-shadowing task and presented listeners with famous talkers. In Experiment 1, half the words were spoken by Barack Obama, and half by Hillary Clinton. Reaction times (RTs) to repeated words were shorter than those to unprimed words only when repeated by the same talker. However, in Experiment 2, using nonfamous talkers, RTs to repeated words were shorter than those to unprimed words both when repeated by the same talker and when repeated by a different talker. Taken together, the results demonstrate that talker-specific details can affect the perception of spoken words relatively early during processing when words are spoken by famous talkers.

On the tolerance of spectral blur in the perception of spoken words

How does a listener resolve linguistic properties conveyed by speech? Many descriptions of perception attribute a causal role to brief spectral details in narrow frequency ranges. Perceptual standards allow far more variety, revealed by the robustness of perception of many kinds of distorted speech. The present study considered the effects of spectral blur on the recognition of spoken words. Listeners heard successive presentations of noise-vocoded easy and hard words. The number of spectral channels composing the word increased with each presentation, reducing blur within a trial. Four conditions counterbalanced the number of presentations of each word in a trial, 3 or 5, and the severity of initial blur, either 1 or 5 channels. In all conditions, the final presentation had 9 bands, yielding a net blur reduction of 4 or 8 bands. These conditions were also tested with the instruction that the words would become clearer during each trial. A control used two repetitions of each word at 9 spectral bands. Across the tests, exposure to spectral blur impaired the recognition of easy and hard words alike regardless of the listener’s belief during presentation. However, intelligibility of hard words declined sharply when subjects were instructed to attend to the continuity and successive decrease in blur within a trial. The pattern of results exposes the role of attention, uncertainty, and spectral resolution in the phonetic contribution to word identification.

Exploring auditory aging can exclusively explain Japanese adults’ age-related decrease in training effects of American English /r/-/l/

Age-related decrease in training effect was shown by training of American English /r/-/l/ contrasts on Japanese speakers. This study examined whether the decrease can be explained exclusively by auditory aging, or other, compensatory cognitive processing should be taken into account. Japanese speakers aged 60’s participated the experiment. Hearing threshold and spoken word perception test of participants’ first language were used to estimate their auditory aging. The word perception test was composed of low-familiar words, high-familiar words, and mono syllables. The audiograms showed low threshold at high frequencies. The result of the perception test showed that low intelligibility for phonemes with high frequency or short duration, and confusion between contracted sounds and basic sounds. These were particular for low-familiar words and mono syllables. These results suggest that participants had auditory aging emerging as high frequency loss and time-frequency-resolution degradation. Nonetheless, the acoustic features to distinguish /r/ and /l/ have long duration, low frequencies and wide frequency distance which are supposed to be unaffected by these auditory aging. The effect of word familiarity suggested compensatory cognitive processing involved. These suggest that age-related decrease cannot be explained exclusively by auditory aging, compensatory cognitive processing should be taken into account.

Examining the role of variability in emotional tone of voice during online spoken word recognition

Previous studies demonstrate that listeners are faster to recognize words recently spoken by the same talker, relative to a different talker, when processing is relatively slow. The purpose of the present study was to examine intra-talker variability in emotional tone of voice on listeners' online perception of spoken words. Based on previous work, we predicted reduced priming in the mismatch condition, when words are repeated in a different emotional tone of voice (e.g., sad and frightened), relative to the match condition, when words are repeated in the same emotional tone of voice (e.g., frightened)—when processing is relatively slow (experiment 2, hard lexical decision) and equivalent priming in the match and mismatch conditions when processing is relatively fast (experiment 1, easy lexical decision). The present study should provide a greater understanding of the role that variability in emotional tone of voice plays in listeners' online perception of spoken words.

Examining the role of talker‐specific details in bilingual listeners’ perception of spoken words.

Previous studies demonstrate that listeners are faster to recognize words recently spoken by the same talker, relative to a different talker. However, there is evidence that such talker effects are more robust when processing is relatively slow. The purpose of the present study was to examine talker effects in bilingual listeners as a function of whether the listeners were hearing words in their first (L1) or second (L2) language. More specifically, in the present study, conducted in Spanish, we examined whether talker changes would affect bilinguals differently, depending on whether Spanish was their L1 (Spanish‐English bilinguals) or their L2 (English‐Spanish bilinguals). Given that bilinguals typically process their L2 more slowly than their L1, the results were expected to reveal greater talker effects in English‐Spanish than in Spanish‐English bilinguals. The present study should provide a greater understanding of the role that talker variability plays in bilingual listeners’ online perception of spoken words.

Investigating whether ease of lexical processing affects listeners’ ratings of the strength of talkers’ foreign accent.

Many research studies in spoken word recognition have examined the role that different types of surface variability have on listeners’ perception of spoken words (e.g., influence of same versus different talker manipulations). In our ongoing research program, we are investigating the other direction of this relationship. Our initial two experiments investigated whether linguistic complexity affects listeners’ overt subjective impressions of the strength of talkers’ foreign accents. Ease of lexical processing was manipulated in two ways, namely, sentence context and long-term repetition priming. In the sentence context experiment, participants made accent ratings on the last word of each sentence presented to them; this word was either related or unrelated to the sentence context. In the long-term repetition priming experiment, participants made their accent ratings to isolated spoken words, which they either had (primed) or had not (unprimed) heard earlier in the experiment. Stronger accent ratings in the unrelated sentence context and unprimed conditions were predicted. Response time to provide their accent rating responses to sentence context and priming manipulations was also studied. Our results should lead to a greater understanding of the role that lexical processing plays in listeners’ perception of foreign accents and perhaps other types of surface variability.