Process Of Spoken Word Recognition Research Articles

Advancements of phonetics in the 21st century: Theoretical and empirical issues of spoken word recognition in phonetic research

How do listeners understand what they are hearing? Humans hearing speech perform spoken word recognition, recognizing what words they are hearing in a speech stream in order to understand the meaning. Phonetics refers to the properties of the speech at a detailed level, particularly below the level of segmental phonemic distinctions. In order to recognize spoken words, listeners have to extract information from the detailed acoustic signal in some way, but theories differ about whether listeners extract phonemes, whole words, or other units, by what mechanism, and they differ on what kinds of information are stored in the lexicon. The process of spoken word recognition can be affected by any number of situations such as the speaker or listener being a non-native of the language or dialect, being a child, having a speech/hearing disability, hearing speech in noise, the speech itself containing variability, or many other situations. Any of these situations can shed light on theoretical questions by giving a fuller picture of how listeners recognize words. This chapter examines what we have learned in these first ∼21 years of the 21st century about how phonetics interacts with spoken word recognition.

Neighborhood Density and Syntactic Class Effects on Spoken Word Recognition: Specific Language Impairment and Typical Development.

The purpose of the current study was to determine the effect of neighborhood density and syntactic class on word recognition in children with specific language impairment (SLI) and typical development (TD). Fifteen children with SLI (M age = 6;5 [years;months]) and 15 with TD (M age = 6;4) completed a forward gating task that presented consonant-vowel-consonant dense and sparse (neighborhood density) nouns and verbs (syntactic class). On all dependent variables, the SLI group performed like the TD group. Recognition performance was highest for dense words and nouns. The majority of 1st nontarget responses shared the 1st phoneme with the target (i.e., was in the target's cohort). When considering the ranking of word types from easiest to most difficult, children showed equivalent recognition performance for dense verbs and sparse nouns, which were both easier to recognize than sparse verbs but more difficult than dense nouns. The current study yields new insight into how children access lexical-phonological information and syntactic class during the process of spoken word recognition. Given the identical pattern of results for the SLI and TD groups, we hypothesize that accessing lexical-phonological information may be a strength for children with SLI. We also discuss implications for using the forward gating paradigm as a measure of word recognition.

When is a wh-in-situ question identified in standard Persian?

ABSTRACTPrevious literature demonstrated the influential role of prediction in processing speech [Brazil, 1981. The place of intonation in a discourse model. In C. Malcolm & M. Montgomery (Eds.), Studies in discourse analysis (pp. 146–157). London: Routledge & Kegan Paul; Grosjean, 1983. How long is the sentence? Prediction and prosody in the on-line processing of language. Linguistics, 21, 501–529, 1996a. Using prosody to predict the end of sentences in English and French: Normal and brain damaged subjects. Language and Cognitive Processes, 11, 107–134; Snedeker & Trueswell, 2003. Using prosody to avoid ambiguity: Effects of speaker awareness and referential context. Journal of Memory and Language, 48, 103–130], and of prosody in predicting the eventual syntactic structure of ambiguous sentences [e.g. Snedeker & Trueswell, 2003. Using prosody to avoid ambiguity: Effects of speaker awareness and referential context. Journal of Memory and Language, 48, 103–130]. Wh-in-situ questions contain temporary syntactic ambiguity. One of the languages characterised by wh-in-situ questions is Persian. The current research adopted the gating paradigm [Grosjean, 1980. Spoken word recognition processes and the gating paradigm. Perception and Psychophysics, 28, 267–283] to investigate when distinctive prosodic cues of the pre-wh part enable correct identification of wh-in-situ questions in Persian. A perception experiment was designed in which gated stimuli were played to Persian native speakers in a forced-choice sentence identification task. In line with our expectation, correct identification responses were given from the beginning of the sentence. The result is discussed in the context of proposals regarding the need to integrate prosody and prediction into models of language and speech processing [Beach, 1991. The interpretation of prosodic patterns at points of syntactic structure ambiguity: Evidence for cue trading relations. Journal of Memory and Language, 30, 644–663; Grosjean, 1983. How long is the sentence? Prediction and prosody in the on-line processing of language. Linguistics, 21, 501–529, 1996a. Using prosody to predict the end of sentences in English and French: Normal and brain damaged subjects. Language and Cognitive Processes, 11, 107–134].

Context Effects and Spoken Word Recognition of Chinese: An Eye-Tracking Study.

This study examined the time-course of context effects on spoken word recognition during Chinese sentence processing. We recruited 60 native Mandarin listeners to participate in an eye-tracking experiment. In this eye-tracking experiment, listeners were told to listen to a sentence carefully, which ended with a Chinese homophone, and look at different visual probes (Chinese characters or different line-drawing pictures) presented concurrently on the computer screen naturally. Different types of context and probe types were manipulated in the experiment. The results showed that (1) preceding sentence context had an early effect on spoken word recognition processes and (2) phonological information of the distracters had only a negligible effect on the spoken word recognition processes. Finally, the patterns of eye-tracking results seemed to favor an interactive approach in spoken word recognition.

Making long-distance relationships work: Quantifying lexical competition with Hidden Markov Models

Abstract A listener recognizes a stimulus word from acoustic–phonetic input by discriminating that word’s representation from those of other words. The Neighborhood Activation Model (NAM; Luce & Pisoni, 1998) is a long-standing and deeply influential model quantifying how properties of the stimulus word and its competitors influence recognition. The current project incorporates Hidden Markov Models (HMMs) into the NAM’s framework to more flexibly evaluate the influence of multiple lexical properties, thereby allowing us to pose novel questions about the process of spoken-word recognition. Analyses using HMMs’ power to evaluate the stimulus’s “distance” even to very distant words suggest that faraway words still act as competitors, suggesting that a larger subset of the lexicon is activated during recognition than has been previously assumed. Another analysis reveals that the way competition is distributed among other words significantly influences word recognition. HMMs have been widely applied in other domains, and our results demonstrate that they may be similarly suited to quantifying the processes underlying spoken-word recognition.

Meaning Inhibition and Sentence Processing in Chinese.

The present study examined the inhibitory processes of spoken word recognition of Chinese homophones during sentence processing, using a standard cross-modal naming experiment with an innovative design and materials construction. Results confirmed that (1) preceding sentence context has exerted an early effect on disambiguating among different alternative meanings of the homophones; (2) the contextually inappropriate meanings of the ambiguous word were inhibited rapidly during sentence processing; and (3) the present results also demonstrated that the inhibitory mechanism could be sustained to a longer duration following the occurrence of the ambiguous word (homophone). Finally, all these results clearly revealed the dynamics of interaction of context effects and spoken word recognition processes.

The process of spoken word recognition in the face of signal degradation.

Though much is known about how words are recognized, little research has focused on how a degraded signal affects the fine-grained temporal aspects of real-time word recognition. The perception of degraded speech was examined in two populations with the goal of describing the time course of word recognition and lexical competition. Thirty-three postlingually deafened cochlear implant (CI) users and 57 normal hearing (NH) adults (16 in a CI-simulation condition) participated in a visual world paradigm eye-tracking task in which their fixations to a set of phonologically related items were monitored as they heard one item being named. Each degraded-speech group was compared with a set of age-matched NH participants listening to unfiltered speech. CI users and the simulation group showed a delay in activation relative to the NH listeners, and there is weak evidence that the CI users showed differences in the degree of peak and late competitor activation. In general, though, the degraded-speech groups behaved statistically similarly with respect to activation levels.

Speaker sex influences processing of grammatical gender.

Spoken words carry linguistic and indexical information to listeners. ionist models of spoken word recognition suggest that indexical information is stripped away in a process called normalization to allow processing of the linguistic message to proceed. In contrast, exemplar models of the lexicon suggest that indexical information is retained in memory, and influences the process of spoken word recognition. In the present study native Spanish listeners heard Spanish words that varied in grammatical gender (masculine, ending in -o, or feminine, ending in -a) produced by either a male or a female speaker. When asked to indicate the grammatical gender of the words, listeners were faster and more accurate when the sex of the speaker “matched” the grammatical gender than when the sex of the speaker and the grammatical gender “mismatched.” No such interference was observed when listeners heard the same stimuli, but identified whether the speaker was male or female. This finding suggests that indexical information, in this case the sex of the speaker, influences not just processes associated with word recognition, but also higher-level processes associated with grammatical processing. This result also raises questions regarding the widespread assumption about the cognitive independence and automatic nature of grammatical processes.

Supplemental Material for The Process of Spoken Word Recognition in the Face of Signal Degradation

Supplemental Material for The Process of Spoken Word Recognition in the Face of Signal Degradation

Lexical Selection in Action: Evidence from Spontaneous Punning

Analysis of a corpus of spontaneously produced Japanese puns from a single speaker over a two-year period provides a view of how a punster selects a source word for a pun and transforms it into another word for humorous effect. The pun-making process is driven by a principle of similarity: the source word should as far as possible be preserved (in terms of segmental sequence) in the pun. This renders homophones (English example: band-banned) the pun type of choice, with part-whole relationships of embedding (cap-capture), and mutations of the source word (peas-bees) rather less favored. Similarity also governs mutations in that single-phoneme substitutions outnumber larger changes, and in phoneme substitutions, subphonemic features tend to be preserved. The process of spontaneous punning thus applies, on line, the same similarity criteria as govern explicit similarity judgments and offline decisions about pun success (e.g., for inclusion in published collections). Finally, the process of spoken-word recognition is word-play-friendly in that it involves multiple word-form activation and competition, which, coupled with known techniques in use in difficult listening conditions, enables listeners to generate most pun types as offshoots of normal listening procedures.

The structure of the lexical network influences lexical processing

Network science is an emerging field that uses computational tools from physics, mathematics, computer science, and other fields to examine the structure of complex systems, and explore how that structure might influence processing. In this approach, words in the mental lexicon can be represented as nodes in a network with links connecting words that are phonologically related to each other. Analyses using the mathematical tools of network science suggest that phonological networks from a variety of languages exhibit the characteristics of small-world networks, and share several other structural features. Studies of small-world networks in other domains have demonstrated that such networks are robust to damage, and can be searched very efficiently. Using conventional psycholinguistic tasks, we examined how certain structural characteristics influence the process of spoken word recognition. The findings from these experiments suggest that the lexicon is structured in a non-arbitrary manner, and that this structure influences lexical processing.

Dissociation of tone and vowel processing in Mandarin idioms

Using event-related potentials, this study measured the access of suprasegmental (tone) and segmental (vowel) information in spoken word recognition with Mandarin idioms. Participants performed a delayed-response acceptability task, in which they judged the correctness of the last word of each idiom, which might deviate from the correct word in either tone or vowel. Results showed that, compared with the correct idioms, a larger early negativity appeared only for vowel violation. Additionally, a larger N400 effect was observed for vowel mismatch than tone mismatch. A control experiment revealed that these differences were not due to low-level physical differences across conditions; instead, they represented the greater constraining power of vowels than tones in the lexical selection and semantic integration of the spoken words. Furthermore, tone violation elicited a more robust late positive component than vowel violation, suggesting different reanalyses of the two types of information. In summary, the current results support a functional dissociation of tone and vowel processing in spoken word recognition.

Eye movements during spoken word recognition in Russian children

This study explores incremental processing in spoken word recognition in Russian 5- and 6-year-olds and adults using free-viewing eye-tracking. Participants viewed scenes containing pictures of four familiar objects and clicked on a target embedded in a spoken instruction. In the cohort condition, two object names shared identical three-phoneme onsets. In the noncohort condition, all object names had unique onsets. Coarse-grain analyses of eye movements indicated that adults produced looks to the competitor on significantly more cohort trials than on noncohort trials, whereas children surprisingly failed to demonstrate cohort competition due to widespread exploratory eye movements across conditions. Fine-grain analyses, in contrast, showed a similar time course of eye movements across children and adults, but with cohort competition lingering more than 1 s longer in children. The dissociation between coarse-grain and fine-grain eye movements indicates a need to consider multiple behavioral measures in making developmental comparisons in language processing.

Effects of open-set and closed-set task demands on spoken word recognition.

Closed-set tests of spoken word recognition are frequently used in clinical settings to assess the speech discrimination skills of hearing-impaired listeners, particularly children. Speech scientists have reported robust effects of lexical competition and talker variability in open-set tasks but not closed-set tasks, suggesting that closed-set tests of spoken word recognition may not be valid assessments of speech recognition skills. The goal of the current study was to explore some of the task demands that might account for this fundamental difference between open-set and closed-set tasks. In a series of four experiments, we manipulated the number and nature of the response alternatives. Results revealed that as more highly confusable foils were added to the response alternatives, lexical competition and talker variability effects emerged in closed-set tests of spoken word recognition. These results demonstrate a close coupling between task demands and lexical competition effects in lexical access and spoken word recognition processes.

Tracking speech comprehension in space and time

A fundamental challenge for the cognitive neuroscience of language is to capture the spatio-temporal patterns of brain activity that underlie critical functional components of the language comprehension process. We combine here psycholinguistic analysis, whole-head magnetoencephalography (MEG), the Mismatch Negativity (MMN) paradigm, and state-of-the-art source localization techniques (Equivalent Current Dipole and L1 Minimum-Norm Current Estimates) to locate the process of spoken word recognition at a specific moment in space and time. The magnetic MMN to words presented as rare “deviant stimuli” in an oddball paradigm among repetitive “standard” speech stimuli, peaked 100–150 ms after the information in the acoustic input, was sufficient for word recognition. The latency with which words were recognized corresponded to that of an MMN source in the left superior temporal cortex. There was a significant correlation ( r = 0.7) of latency measures of word recognition in individual study participants with the latency of the activity peak of the superior temporal source. These results demonstrate a correspondence between the behaviorally determined recognition point for spoken words and the cortical activation in left posterior superior temporal areas. Both the MMN calculated in the classic manner, obtained by subtracting standard from deviant stimulus response recorded in the same experiment, and the identity MMN (iMMN), defined as the difference between the neuromagnetic responses to the same stimulus presented as standard and deviant stimulus, showed the same significant correlation with word recognition processes.

The time course of indexical specificity effects in the perception of spoken words

This research investigates the time-course of indexical specificity effects in spoken word recognition by examining the circumstances under which the variability in the speaking rate affects the participant’s perception of spoken words. Previous research has demonstrated that variability has both representational and processing consequences. The current research examines one of the conditions expected to influence the extent to which indexical variability plays a role in spoken word recognition, namely the time-course of processing. Based on our past work, it was hypothesized that indexical specificity effects associated with the speaking rate would only affect later stages of processing in spoken word recognition. The results confirm this hypothesis: Specificity effects are only in evidence when processing is relatively slow. [Research supported (in part) by Research Grant No. R01 DC 0265801 from the National Institute on Deafness and Other Communication Disorders, National Institutes of Health.]

Using behavioral data to model open-set word recognition and lexical organization by pediatric cochlear implant users.

Children who acquire language through a cochlear implant (CI) provide a unique opportunity to study the physical and cognitive bases of speech perception, as the target language these children must acquire is the same as for children without hearing loss, but the input to the nervous system from a CI is very different from normal hearing. Psycholinguistic models of speech perception in adults with normal hearing propose that the phonological lexicon is organized in a phonetic similarity space based on linguistic features and phonemes, and that these units are important for spoken word recognition. Although the physical signal received through a CI is different from aurally perceived speech, we propose that the same abstract units of language are nonetheless a part of the linguistic knowledge of children who learn spoken language through CIs. In this paper, we model performance by children with CIs on tests of open-set word recognition using their performance on closed-set tests of feature identification. Our model of word recognition does not employ any free parameters, and therefore is highly constrained. Modeling the cognitive process of spoken word recognition on the basis of feature identification performance is important to both theoretical and clinical research. The model of word recognition we propose is appropriate for both listeners who use CIs and those with normal hearing. It is therefore a theoretical model of the process of spoken word recognition in general, as well one way to address the question of whether the cognitive process of spoken word recognition is the same for both CI users and those with normal hearing. Comparing observed and predicted open-set word recognition performance from closed-set identification tasks may be particularly useful clinically for the pediatric CI population. If differences between observed and predicted performance can be attributed to particular components of the model, then it may be possible to provide intervention that is maximally beneficial to the child.

Modeling Spoken Word Recognition Performance by Pediatric Cochlear Implant Users using Feature Identification

Computational simulations were carried out to evaluate the appropriateness of several psycholinguistic theories of spoken word recognition for children who use cochlear implants. These models also investigate the interrelations of commonly used measures of closed-set and open-set tests of speech perception. A software simulation of phoneme recognition performance was developed that uses feature identification scores as input. Two simulations of lexical access were developed. In one, early phoneme decisions are used in a lexical search to find the best matching candidate. In the second, phoneme decisions are made only when lexical access occurs. Simulated phoneme and word identification performance was then applied to behavioral data from the Phonetically Balanced Kindergarten test and Lexical Neighborhood Test of open-set word recognition. Simulations of performance were evaluated for children with prelingual sensorineural hearing loss who use cochlear implants with the MPEAK or SPEAK coding strategies. Open-set word recognition performance can be successfully predicted using feature identification scores. In addition, we observed no qualitative differences in performance between children using MPEAK and SPEAK, suggesting that both groups of children process spoken words similarly despite differences in input. Word recognition ability was best predicted in the model in which phoneme decisions were delayed until lexical access. Closed-set feature identification and open-set word recognition focus on different, but related, levels of language processing. Additional insight for clinical intervention may be achieved by collecting both types of data. The most successful model of performance is consistent with current psycholinguistic theories of spoken word recognition. Thus it appears that the cognitive process of spoken word recognition is fundamentally the same for pediatric cochlear implant users and children and adults with normal hearing.

The data are what the data are: Top-down processing in spoken word recognition is necessary

For the last two decades there has been an ongoing debate about the architecture of the word recognition process: Is this process accomplished in a strictly autonomous, bottom-up fashion, or does information at higher levels affect the perception of lower-level representations? I will review results from a number of experimental settings that support the necessity of positing top-down, lexical-to-phonemic effects. I will argue that models that are presented as strictly autonomous must therefore either be at odds with the data, or must not in fact be fully autonomous. The literature indicates that perception of spoken words is (naturally) based primarily on the perceptual input, but that lexical knowledge also plays a significant role: Top-down processing is a necessary component of an accurate model.

Syntagmatic and paradigmatic states in spoken word recognition

The process of spoken word recognition has frequently been characterized as a mapping of information through a multidimensional space, the dimensions of which correspond to time and acoustic-phonetic similarity. Within this framework, the speed and accuracy of spoken word recognition is a direct function of the ease of traversing a path through the space. The traversal of a path may be affected by at least two states of the space at any point in processing: The first, or syntagmatic, state refers to the history and future of the path. The syntagmatic states of a word correspond to its ‘‘probabilistic’’ phonotactics. The second, or paradigmatic, state refers to the degree of activation within the space at any point (or what has previously been referred to as ‘‘neighborhood activation’’). A connectionist instantiation will be presented of this framework that attempts to predict the speed and accuracy of spoken word recognition as function of syntagmatic and paradigmatic states. Also discussed is a number of empirical tests of the model that provide further insights into the roles of phonotactics and neighborhood structure in spoken word recognition. [Work supported by NIH Grant No. DC-00879-01 to the State University of New York at Buffalo.]