Spoken Word Processing Research Articles

On French listeners’ ability to use stress during spoken word processing

ABSTRACTPrevious studies have suggested that French listeners experience difficulties when they have to discriminate between words that differ in stress. A limitation is that these studies used stress patterns that do not respect the rules of stress placement in French. In this study, three stress patterns were tested on bisyllabic words (1) the legal stress pattern in French, namely words that were unstressed compared to words that bore primary stress on their last syllable (/ʒuʁi/-/ʒu’ʁi/), (2) an illegal stress location pattern, namely words that bore primary stress on their first syllable compared to words that bore primary stress on their last syllable (/’ʒuʁi/-/ʒu’ʁi/) and (3) an illegal pattern that involves an unstressed word, namely words that were unstressed compared to words that bore primary stress on their first syllable (/ʒuʁi/-/’ʒuʁi/). In an ABX task, participants heard three items produced by three different speakers and had to indicate whether X was identical to A or B. The stimuli A and B varied in stress (/ʒu’ʁi/-/ʒuʁi/-/ʒu’ʁi/), in one phoneme (/ʒu’ʁi/-/ʒu’ʁɔ˜/-/ʒu’ʁi/) or in both stress and one phoneme (/ʒu’ʁi/-/ʒuʁɔ˜/-/ʒu’ʁi/). The results showed that French listeners are fully able to discriminate between two words differing in stress provided that the stress pattern included an unstressed word. More importantly, they suggest that the French listeners’ difficulties mainly reside in locating stress within words.

Effect of early dialectal exposure on adult perception of phonemic vowel length.

Attunement to native phonological categories and the specification of relevant phonological features in the lexicon occur early in development for monolingual and monodialectal speakers. However, few studies have investigated whether and how early exposure to two dialects of a language might influence the development of phonological categories, especially when a phonemic contrast exists only in one dialect. This study compared perceptual sensitivity to mispronunciations in phonemic vowel length in Australian English adult listeners with and without early exposure to another English dialect that did not have this contrast. The results showed that, while both mono- and bi-dialectal groups were sensitive to mispronunciations in vowel length, the bi-dialectal adults were more likely to accept a mispronunciation in vowel length compared to mono-dialectal adults. The bi-dialectal group accepted mispronunciations in vowel length more than in vowel height and backness. These results suggest that the bi-dialectal Australian English adults may employ a more flexible vowel length category for spoken word processing compared to mono-dialectal adults. The findings reveal a complex influence of early exposure to another dialect on the development of phonological categories.

ERP responses to lexical-semantic processing in typically developing toddlers, in adults, and in toddlers at risk for language and learning impairment.

Children begin to establish lexical-semantic representations during their first year of life, resulting in a rapid growth of vocabulary around 18-24 months of age. The neural mechanisms underlying this initial ability to map words onto conceptual representations remain relatively unknown. In the present study, the electrophysiological underpinnings of these mechanisms are explored during the critical phase of lexical acquisition using a picture-word matching paradigm. Event-Related Potentials (ERPs) elicited by words (either congruous or incongruous with the previous picture context) and pseudo-words are investigated in 20-month-old toddlers (N = 20) and compared to those elicited in a sample of adults (N = 20), reflecting the final and efficient system, and a sample of toddlers at familial risk for language and learning impairment (LLI, N = 15). The results suggest that the architecture underlying spoken word representation and processing is constant throughout development, even if some differences between children and adults emerged. Interestingly, children seem to be faster than adults in processing incongruent words, probably because relying on a different and more superficial strategy. This early strategy does not seem to be present in children at risk for LLI. In addition, both groups of children do not show different and specific electrophysiological underpinnings in response to real but incongruent words and unknown words, suggesting that during the critical phase of lexical acquisition any potential word is processed in a similar way. Overall, children at risk for LLI turned out to be sensitive to verbal incongruity of the lexical-semantic context, although some differences from typically developing children emerged, reflecting slower processing and less automatic responses. Taken together, the findings of this study pave the way to further research to investigate these effects in clinical and at-risk populations with the general purpose of disentangling the underlying mechanisms of lexical acquisition, and potentially predicting later language (dis)abilities.

Word length modulates the effect of emotional prosody

Previous work has shown that emotional prosody, independent of the lexical carrier, activates words associated with the emotional information (Kim, 2015; Kim & Sumner, submitted). For example, hearing a non-emotional word (pineapple) uttered with angry prosody facilitates recognition of angry-associated words (mad). Building on this finding, the current study delves into the nature of the affective priming between emotional prosody and emotional words and tests if word length modulates affective priming. Word length is an important dimension in lexical processing, as longer words are shown to produce stronger lexical activation than shorter words (Pitt & Samuel, 2006). I hypothesize that social information shows a stronger effect in spoken word processing when lexical activation is weaker. Then we should find stronger affective priming with shorter words than longer words. This hypothesis was tested with a cross-modal priming experiment. The visual targets were 12 angry related words (e.g., mad, upset). The targets were preceded by two-, three-, or four-syllable non-emotional primes (e.g., atom, envelope, aluminum) spoken with angry prosody. Listeners recognized angry words faster after hearing angry prosody than after hearing neutral prosody when the prime words were short (2 syllables) but not when the prime words were longer (3-4 syllables). The current results provide evidence that social effects in word recognition are modulated by the strength of lexical activation.

Spoken words can make the invisible visible-Testing the involvement of low-level visual representations in spoken word processing.

The notion that processing spoken (object) words involves activation of category-specific representations in visual cortex is a key prediction of modality-specific theories of representation that contrasts with theories assuming dedicated conceptual representational systems abstracted away from sensorimotor systems. In the present study, we investigated whether participants can detect otherwise invisible pictures of objects when they are presented with the corresponding spoken word shortly before the picture appears. Our results showed facilitated detection for congruent ("bottle" → picture of a bottle) versus incongruent ("bottle" → picture of a banana) trials. A second experiment investigated the time-course of the effect by manipulating the timing of picture presentation relative to word onset and revealed that it arises as soon as 200-400 ms after word onset and decays at 600 ms after word onset. Together, these data strongly suggest that spoken words can rapidly activate low-level category-specific visual representations that affect the mere detection of a stimulus, that is, what we see. More generally, our findings fit best with the notion that spoken words activate modality-specific visual representations that are low level enough to provide information related to a given token and at the same time abstract enough to be relevant not only for previously seen tokens but also for generalizing to novel exemplars one has never seen before. (PsycINFO Database Record

The multimodal nature of spoken word processing in the visual world: Testing the predictions of alternative models of multimodal integration

Ambiguity in natural language is ubiquitous, yet spoken communication is effective due to integration of information carried in the speech signal with information available in the surrounding multimodal landscape. Language mediated visual attention requires visual and linguistic information integration and has thus been used to examine properties of the architecture supporting multimodal processing during spoken language comprehension. In this paper we test predictions generated by alternative models of this multimodal system. A model (TRACE) in which multimodal information is combined at the point of the lexical representations of words generated predictions of a stronger effect of phonological rhyme relative to semantic and visual information on gaze behaviour, whereas a model in which sub-lexical information can interact across modalities (MIM) predicted a greater influence of visual and semantic information, compared to phonological rhyme. Two visual world experiments designed to test these predictions offer support for sub-lexical multimodal interaction during online language processing.

Orthographic effects in spoken word recognition: Evidence from Chinese.

Extensive evidence from alphabetic languages demonstrates a role of orthography in the processing of spoken words. Because alphabetic systems explicitly code speech sounds, such effects are perhaps not surprising. However, it is less clear whether orthographic codes are involuntarily accessed from spoken words in languages with non-alphabetic systems, in which the sound-spelling correspondence is largely arbitrary. We investigated the role of orthography via a semantic relatedness judgment task: native Mandarin speakers judged whether or not spoken word pairs were related in meaning. Word pairs were either semantically related, orthographically related, or unrelated. Results showed that relatedness judgments were made faster for word pairs that were semantically related than for unrelated word pairs. Critically, orthographic overlap on semantically unrelated word pairs induced a significant increase in response latencies. These findings indicate that orthographic information is involuntarily accessed in spoken-word recognition, even in a non-alphabetic language such as Chinese.

Electrophysiological Correlates of Emotional Content and Volume Level in Spoken Word Processing.

For visual stimuli of emotional content as pictures and written words, stimulus size has been shown to increase emotion effects in the early posterior negativity (EPN), a component of event-related potentials (ERPs) indexing attention allocation during visual sensory encoding. In the present study, we addressed the question whether this enhanced relevance of larger (visual) stimuli might generalize to the auditory domain and whether auditory emotion effects are modulated by volume. Therefore, subjects were listening to spoken words with emotional or neutral content, played at two different volume levels, while ERPs were recorded. Negative emotional content led to an increased frontal positivity and parieto-occipital negativity—a scalp distribution similar to the EPN—between ~370 and 530 ms. Importantly, this emotion-related ERP component was not modulated by differences in volume level, which impacted early auditory processing, as reflected in increased amplitudes of the N1 (80–130 ms) and P2 (130–265 ms) components as hypothesized. However, contrary to effects of stimulus size in the visual domain, volume level did not influence later ERP components. These findings indicate modality-specific and functionally independent processing triggered by emotional content of spoken words and volume level.

Semantic information mediates visual attention during spoken word recognition in Chinese: Evidence from the printed-word version of the visual-world paradigm.

In the present study, we investigated whether the activation of semantic information during spoken word recognition can mediate visual attention's deployment to printed Chinese words. We used a visual-world paradigm with printed words, in which participants listened to a spoken target word embedded in a neutral spoken sentence while looking at a visual display of printed words. We examined whether a semantic competitor effect could be observed in the printed-word version of the visual-world paradigm. In Experiment 1, the relationship between the spoken target words and the printed words was manipulated so that they were semantically related (a semantic competitor), phonologically related (a phonological competitor), or unrelated (distractors). We found that the probability of fixations on semantic competitors was significantly higher than that of fixations on the distractors. In Experiment 2, the orthographic similarity between the spoken target words and their semantic competitors was manipulated to further examine whether the semantic competitor effect was modulated by orthographic similarity. We found significant semantic competitor effects regardless of orthographic similarity. Our study not only reveals that semantic information can affect visual attention, it also provides important new insights into the methodology employed to investigate the semantic processing of spoken words during spoken word recognition using the printed-word version of the visual-world paradigm.

The influence of second language proficiency on bilingual parallel language activation in Hindi–English bilinguals

ABSTRACTWe investigated the spontaneous activation of phonologically related words in high and low proficient Hindi–English bilinguals during spoken word processing in an eye-tracking study. Participants listened to spoken words in L1/L2 and looked at a display (consisting of line drawings of phonological cohort of the translation equivalent of the spoken word and unrelated distractors). Both the groups were quick in orienting their attention towards the competitor with the onset of the spoken word. Furthermore, high proficient bilinguals showed higher and earlier activation of the competitor compared to low proficient bilinguals. Cross-language activations were higher in the L2–L1 direction for both the groups. The results strongly suggest language non-selective access of translation in Hindi–English bilinguals in both language directions. We discuss the results with regard to the predictions of the bilingual language processing models and the effect of language proficiency on conceptual access during listening in bilinguals.

Phonological Neighborhood Effects in Spoken Word Perception and Production

Research on spoken word perception and production has identified two hallmarks of spoken word processing: multiple activation of representations of the sound patterns of words in memory and subsequent competition among these patterns. Evidence for this activation-competition process has come, in part, from experimental studies examining the effects of phonological neighborhoods, which are collections of similar-sounding words that are activated in memory during both perception and production. In this article, we review more than 20 years of research on phonological neighborhood effects in spoken word processing that has demonstrated that the speed and accuracy of spoken word perception and production are, in large part, a function of the density and frequency of neighborhoods of spoken words. We conclude our review with a discussion of new avenues of research—based on recent advances in network science—that hold the promise of deepening our understanding of the mental operations involved in our uniquely human capacity for communicating with the spoken word.

An fMRI Study of Concreteness Effects during Spoken Word Recognition in Aging. Preservation or Attenuation?

It is unclear whether healthy aging influences concreteness effects (i.e., the processing advantage seen for concrete over abstract words) and its associated neural mechanisms. We conducted an fMRI study on young and older healthy adults performing auditory lexical decisions on concrete vs. abstract words. We found that spoken comprehension of concrete and abstract words appears relatively preserved for healthy older individuals, including the concreteness effect. This preserved performance was supported by altered activity in left hemisphere regions including the inferior and middle frontal gyri, angular gyrus, and fusiform gyrus. This pattern is consistent with age-related compensatory mechanisms supporting spoken word processing.

Early use of orthographic information in spoken word recognition: Event-related potential evidence from the Korean language.

This study examines whether orthographic information is used during prelexical processes in spoken word recognition by investigating ERPs during spoken word processing for Korean words. Differential effects due to orthographic syllable neighborhood size and sound-to-spelling consistency on P200 and N320 were evaluated by recording ERPs from 42 participants during a lexical decision task. The results indicate that P200 was smaller for words whose orthographic syllable neighbors are large in number rather than those that are small. In addition, a word with a large orthographic syllable neighborhood elicited a smaller N320 effect than a word with a small orthographic syllable neighborhood only when the word had inconsistent sound-to-spelling mapping. The results provide support for the assumption that orthographic information is used early during the prelexical spoken word recognition process.

L1 and L2 Spoken Word Processing: Evidence from Divided Attention Paradigm.

The present study aims to reveal some facts concerning first language (L1) and second language (L2) spoken-word processing in unbalanced proficient bilinguals using behavioral measures. The intention here is to examine the effects of auditory repetition word priming and semantic priming in first and second languages of these bilinguals. The other goal is to explore the effects of attention manipulation on implicit retrieval of perceptual and conceptual properties of spoken L1 and L2 words. In so doing, the participants performed auditory word priming and semantic priming as memory tests in their L1 and L2. In a half of the trials of each experiment, they carried out the memory test while simultaneously performing a secondary task in visual modality. The results revealed that effects of auditory word priming and semantic priming were present when participants processed L1 and L2 words in full attention condition. Attention manipulation could reduce priming magnitude in both experiments in L2. Moreover, L2 word retrieval increases the reaction times and reduces accuracy on the simultaneous secondary task to protect its own accuracy and speed.

Feature Statistics Modulate the Activation of Meaning During Spoken Word Processing.

Understanding spoken words involves a rapid mapping from speech to conceptual representations. One distributed feature‐based conceptual account assumes that the statistical characteristics of concepts’ features—the number of concepts they occur in (distinctiveness/sharedness) and likelihood of co‐occurrence (correlational strength)—determine conceptual activation. To test these claims, we investigated the role of distinctiveness/sharedness and correlational strength in speech‐to‐meaning mapping, using a lexical decision task and computational simulations. Responses were faster for concepts with higher sharedness, suggesting that shared features are facilitatory in tasks like lexical decision that require access to them. Correlational strength facilitated responses for slower participants, suggesting a time‐sensitive co‐occurrence‐driven settling mechanism. The computational simulation showed similar effects, with early effects of shared features and later effects of correlational strength. These results support a general‐to‐specific account of conceptual processing, whereby early activation of shared features is followed by the gradual emergence of a specific target representation.

The role of prior language context on bilingual spoken word processing: Evidence from the visual world task

We investigated whether speaking in one language affects cross- and within-language activation when subsequently switching to a task performed in the same or different language. English–French bilinguals (L1 English, n = 29; L1 French, n = 28) were randomly assigned to a prior language context condition consisting of a spontaneous production task in English (the no-switch group) or in French (the switch group). Participants then performed an English spoken language comprehension task using the visual world method. The key result was that the switch group showed less evidence of cross-language competition than the no-switch group, consistent with the notion of an active inhibition of a prior language in the switch group. These data suggest that proficient bilinguals can globally suppress a non-target language, whether it is L1 or L2, though doing so requires cognitive resources that may be diverted from other demands, such as controlling within-language competition.

When semantics aids phonology: A processing advantage for iconic word forms in aphasia

Iconicity is the non-arbitrary relation between properties of a phonological form and semantic content (e.g. “moo”, “splash”). It is a common feature of both spoken and signed languages, and recent evidence shows that iconic forms confer an advantage during word learning. We explored whether iconic forms conferred a processing advantage for 13 individuals with aphasia following left-hemisphere stroke. Iconic and control words were compared in four different tasks: repetition, reading aloud, auditory lexical decision and visual lexical decision. An advantage for iconic words was seen for some individuals in all tasks, with consistent group effects emerging in reading aloud and auditory lexical decision. Both these tasks rely on mapping between semantics and phonology. We conclude that iconicity aids spoken word processing for individuals with aphasia. This advantage is due to a stronger connection between semantic information and phonological forms.

Effects of Auditory Translation Priming under Divided Attention in Unbalanced Persian-English Bilinguals

In bilingual studies, repetition priming across languages or translation priming can be used to examine themental representations of bilingual lexicon and language in memory. Motivated to demonstrate the effects ofdividing attention during implicit retrieval of L2 spoken words, we investigated the nature of the processesinvolved in translation priming. In so doing, we used behavioral measures (i.e. reaction time and accuracy) tostudy 60 Persian-English unbalanced proficient bilinguals performing translation priming in two languagedirections under two attention conditions. The present study compared a divided attention (DA) condition, inwhich participants carried out the priming task in auditory modality while simultaneously performing asecondary task in visual modality, and a full attention (FA) condition, in which participants performed only thepriming task. We also examined secondary tasks costs produced by memory tests. Despite significant primingeffects and symmetrical pattern of translation priming in the FA condition, translation priming effects in L2-to-L1direction were absent in the DA condition. The secondary task was disrupted by memory test in this direction aswell. The paper ends with discussion on the role of attention in L2 spoken word processing and languagedirection in translation priming in light of models of bilingual memory.

Talker variability in spoken word recognition: Evidence from repetition priming

The effect of talker variability on the processing of spoken words is investigated using short-term repetition priming experiments. Prime-target pairs, either repeated (e.g., queen-queen) or unrelated (e.g., bell-queen), were produced by the same or different male speakers. Two interstimulus intervals (ISI, 50 and 250 ms) were used to explore the time course of repetition priming and voice specificity effects. The auditory stimuli were presented to 40 listeners, who completed a lexical decision task followed by a talker voice discrimination task. Results from the lexical decision task showed that the magnitude of priming was attenuated in the different-talker condition, indicating a talker variability effect on spoken word recognition. In contrast, the talker variability effect on priming did not differ between the two ISIs. Talker voice discrimination was faster and more accurate for nonword targets, but not for word targets, indicating a lexical status effect on voice discrimination. Taken together, these results suggest that talker variability affects recognition of spoken words, and that the effect cannot be simply attributed to non-lexical voice discrimination.

Developmental differences in the influence of phonological similarity on spoken word processing in Mandarin Chinese

The developmental trajectory of spoken word recognition has been well established in Indo-European languages, but to date remains poorly characterized in Mandarin Chinese. In this study, typically developing children (N=17; mean age 10; 5) and adults (N=17; mean age 24) performed a picture–word matching task in Mandarin while we recorded ERPs. Mismatches diverged from expectations in different components of the Mandarin syllable; namely, word-initial phonemes, word-final phonemes, and tone. By comparing responses to different mismatch types, we uncovered evidence suggesting that both children and adults process words incrementally. However, we also observed key developmental differences in how subjects treated onset and rime mismatches. This was taken as evidence for a stronger influence of top-down processing on spoken word recognition in adults compared to children. This work therefore offers an important developmental component to theories of Mandarin spoken word recognition.