Models Of Spoken Word Recognition Research Articles

Surface phonetic or underlying phonological representations: A mismatch negativity study of Mandarin tone assimilation

Phonological alternation, in which a sound changes depending on its phonological environment, poses challenges to spoken word recognition models. Mandarin T3 sandhi is such a phenomenon in which a tone 3 changes into a tone 2 when followed by another T3, which raises questions regarding whether the human brain processes the surface acoustic-phonetic representation or the underlying linguistic representation of Mandarin T3 sandhi words. We conducted a mismatch negativity (MMN) study examining this issue. Participants passively listened to a T2 word [tʂu2 je4] /tʂu2 je4/, a T3 word [tʂu3 ʐən4] /tʂu3 ʐən4/, a Sandhi word [tʂu2 jen3] /tʂu3 jen3/, or a mix of T3 and sandhi words in an odd-ball paradigm. All were interspersed with a T2 word [tʂu2] (deviant). Results showed an MMN only in the T2 and T3 condition but not in the Sandhi or Mix condition. The fact that the surface acoustic information in the T2 and Sandhi conditions was identical, yet yielding disparate MMN results suggests that Mandarin speakers pr...

The influence of 2-hop network density on spoken word recognition.

The influence of 2-hop density on spoken word recognition was investigated. 2-hop density measures the density of connections among the phonological neighbors (i.e., 1-hop neighbors) and phonological neighbors of those neighbors (i.e., 2-hop neighbors) of a target word. In both naming and lexical decision tasks, words with low 2-hop density were recognized more quickly than words with high 2-hop density. Because stimuli were selected such that the number of 1-hop and 2-hop neighbors were matched across both sets of words, the results suggest that spoken word recognition is influenced by the amount of connectivity among distant neighbors of the target word-a result that is not easily accommodated by current models of spoken word recognition. A diffusion of activation framework is proposed to account for the present findings.

Encoding lexical tones in jTRACE: a simulation of monosyllabic spoken word recognition in Mandarin Chinese.

Despite its prevalence as one of the most highly influential models of spoken word recognition, the TRACE model has yet to be extended to consider tonal languages such as Mandarin Chinese. A key reason for this is that the model in its current state does not encode lexical tone. In this report, we present a modified version of the jTRACE model in which we borrowed on its existing architecture to code for Mandarin phonemes and tones. Units are coded in a way that is meant to capture the similarity in timing of access to vowel and tone information that has been observed in previous studies of Mandarin spoken word recognition. We validated the model by first simulating a recent experiment that had used the visual world paradigm to investigate how native Mandarin speakers process monosyllabic Mandarin words (Malins & Joanisse, 2010). We then subsequently simulated two psycholinguistic phenomena: (1) differences in the timing of resolution of tonal contrast pairs, and (2) the interaction between syllable frequency and tonal probability. In all cases, the model gave rise to results comparable to those of published data with human subjects, suggesting that it is a viable working model of spoken word recognition in Mandarin. It is our hope that this tool will be of use to practitioners studying the psycholinguistics of Mandarin Chinese and will help inspire similar models for other tonal languages, such as Cantonese and Thai.

Spoken word recognition and serial recall of words from components in the phonological network.

Network science uses mathematical techniques to study complex systems such as the phonological lexicon (Vitevitch, 2008). The phonological network consists of a giant component (the largest connected component of the network) and lexical islands (smaller groups of words that are connected to each other, but not to the giant component). To determine if the component that a word resided in influenced lexical processing, language-related tasks (naming, lexical decision, and serial recall) were used to compare the processing of words from the giant component and from lexical islands. Results showed that words from lexical islands were recognized more quickly and recalled more accurately than words from the giant component. These findings can be accounted for via the diffusion of activation across a network. Implications for models of spoken word recognition and network science are also discussed.

Priming the representation of Mandarin tone 3 sandhi words

ABSTRACTPhonological alternation poses problems for spoken word recognition. In Mandarin Tone 3 sandhi, a Tone 3 syllable changes to a Tone 2 syllable when followed by another Tone 3 syllable. A traditional phonological account assumes that the initial syllable of Mandarin disyllabic sandhi words is Tone 3 (T3) underlyingly, but becomes Tone 2 (T2) on the surface. In an auditory–auditory priming lexical decision experiment, each disyllabic tone sandhi target word (e.g., chu3-li3) was preceded by one of three monosyllabic primes: a T2 prime (Surface-Tone overlap) (chu2), a T3 prime (Underlying-Tone overlap) (chu3), or a control prime (Baseline condition) (chu1). Results showed that Tone 3 primes (Underlying-Tone) elicited significantly stronger facilitation effects for the sandhi targets than Tone 2 primes (Surface-Tone), with little effect of target frequency. The data are examined in terms of the contribution of underlying representations for models of spoken word recognition.

Conducting spoken word recognition research online: Validation and a new timing method.

Models of spoken word recognition typically make predictions that are then tested in the laboratory against the word recognition scores of human subjects (e.g., Luce & Pisoni Ear and Hearing, 19, 1-36, 1998). Unfortunately, laboratory collection of large sets of word recognition data can be costly and time-consuming. Due to the numerous advantages of online research in speed, cost, and participant diversity, some labs have begun to explore the use of online platforms such as Amazon's Mechanical Turk (AMT) to source participation and collect data (Buhrmester, Kwang, & Gosling Perspectives on Psychological Science, 6, 3-5, 2011). Many classic findings in cognitive psychology have been successfully replicated online, including the Stroop effect, task-switching costs, and Simon and flanker interference (Crump, McDonnell, & Gureckis PLoS ONE, 8, e57410, 2013). However, tasks requiring auditory stimulus delivery have not typically made use of AMT. In the present study, we evaluated the use of AMT for collecting spoken word identification and auditory lexical decision data. Although online users were faster and less accurate than participants in the lab, the results revealed strong correlations between the online and laboratory measures for both word identification accuracy and lexical decision speed. In addition, the scores obtained in the lab and online were equivalently correlated with factors that have been well established to predict word recognition, including word frequency and phonological neighborhood density. We also present and analyze a method for precise auditory reaction timing that is novel to behavioral research. Taken together, these findings suggest that AMT can be a viable alternative to the traditional laboratory setting as a source of participation for some spoken word recognition research.

Distal rhythm influences whether or not listeners hear a word in continuous speech: Support for a perceptual grouping hypothesis

Due to extensive variability in the phonetic realizations of words, there may be few or no proximal spectro-temporal cues that identify a word’s onset or even its presence. Dilley and Pitt (2010) showed that the rate of context speech, distal from a to-be-recognized word, can have a sizeable effect on whether or not a word is perceived. This investigation considered whether there is a distinct role for distal rhythm in the disappearing word effect. Listeners heard sentences that had a grammatical interpretation with or without a critical function word (FW) and transcribed what they heard (e.g., are in Jill got quite mad when she heard there are birds can be removed and Jill got quite mad when she heard their birds is still grammatical). Consistent with a perceptual grouping hypothesis, participants were more likely to report critical FWs when distal rhythm (repeating ternary or binary pitch patterns) matched the rhythm in the FW-containing region than when it did not. Notably, effects of distal rhythm and distal rate were additive. Results demonstrate a novel effect of distal rhythm on the amount of lexical material listeners hear, highlighting the importance of distal timing information and providing new constraints for models of spoken word recognition.

Insights into failed lexical retrieval from network science

Previous network analyses of the phonological lexicon (Vitevitch, 2008) observed a web-like structure that exhibited assortative mixing by degree: words with dense phonological neighborhoods tend to have as neighbors words that also have dense phonological neighborhoods, and words with sparse phonological neighborhoods tend to have as neighbors words that also have sparse phonological neighborhoods. Given the role that assortative mixing by degree plays in network resilience, we examined instances of real and simulated lexical retrieval failures in computer simulations, analysis of a slips-of-the-ear corpus, and three psycholinguistic experiments for evidence of this network characteristic in human behavior. The results of the various analyses support the hypothesis that the structure of words in the mental lexicon influences lexical processing. The implications of network science for current models of spoken word recognition, language processing, and cognitive psychology more generally are discussed.

Speaker sex influences processing of grammatical gender.

Spoken words carry linguistic and indexical information to listeners. ionist models of spoken word recognition suggest that indexical information is stripped away in a process called normalization to allow processing of the linguistic message to proceed. In contrast, exemplar models of the lexicon suggest that indexical information is retained in memory, and influences the process of spoken word recognition. In the present study native Spanish listeners heard Spanish words that varied in grammatical gender (masculine, ending in -o, or feminine, ending in -a) produced by either a male or a female speaker. When asked to indicate the grammatical gender of the words, listeners were faster and more accurate when the sex of the speaker “matched” the grammatical gender than when the sex of the speaker and the grammatical gender “mismatched.” No such interference was observed when listeners heard the same stimuli, but identified whether the speaker was male or female. This finding suggests that indexical information, in this case the sex of the speaker, influences not just processes associated with word recognition, but also higher-level processes associated with grammatical processing. This result also raises questions regarding the widespread assumption about the cognitive independence and automatic nature of grammatical processes.

Early effects of neighborhood density and phonotactic probability of spoken words on event-related potentials

All current models of spoken word recognition propose that sound-based representations of spoken words compete with, or inhibit, one another during recognition. In addition, certain models propose that higher probability sublexical units facilitate recognition under certain circumstances. Two experiments were conducted examining ERPs to spoken words and nonwords simultaneously varying in phonotactic probability and neighborhood density. Results showed that the amplitude of the P2 potential was greater for high probability-density words and nonwords, suggesting an early inhibitory effect of neighborhood density. In order to closely examine the role of phonotactic probability, effects of initial phoneme frequency were also examined. The latency of the P2 potential was shorter for words with high initial-consonant probability, suggesting a facilitative effect of phonotactic probability. The current results are consistent with findings from previous studies using reaction time and eye-tracking paradigms and provide new insights into the time-course of lexical and sublexical activation and competition.

Effects of variation on processing of word-medial consonants

The present study investigates processing of variation in word-medial stops and the role of this variation in phoneme recognition. Variation in word-medial stops has been shown to influence lexical access, but it is unclear whether this variation also affects recognition at the phoneme level [Tucker, J. Phonet., 39, (2011), 312–318]. A phoneme monitoring task is used to investigate the role of production variation in the identification of word-medial stop consonants. Following Warner and Tucker [J. Acoust. Soc. Am. 130, (2012), 1606–1617], the stimuli comprise English nonwords with target consonants [b], [p], [d], [t], [g], [k] in an intervocalic post-stress position produced with a range of production variation. This variation was grouped into three sets including groups at each extremes and a group in the middle of the range. The acoustic characteristics of the stops, which reflect this variation (e.g., consonant duration and intensity difference) were extracted and used to model the reaction times collected during the phoneme monitoring experiment. The reaction times were statistically modeled using a linear mixed-effects regression. Models of spoken word recognition and models of lexical storage are used to interpret the results and the contribution of the results to our understanding of these models is discussed.

Reconsidering the role of temporal order in spoken word recognition.

Models of spoken word recognition assume that words are represented as sequences of phonemes. We evaluated this assumption by examining phonemic anadromes, words that share the same phonemes but differ in their order (e.g., sub and bus). Using the visual-world paradigm, we found that listeners show more fixations to anadromes (e.g., sub when bus is the target) than to unrelated words (well) and to words that share the same vowel but not the same set of phonemes (sun). This contrasts with the predictions of existing models and suggests that words are not defined as strict sequences of phonemes.

An ANN based approach to recognize initial phonemes of spoken words of Assamese language

Initial phoneme is used in spoken word recognition models. These are used to activate words starting with that phoneme in spoken word recognition models. Such investigations are critical for classification of initial phoneme into a phonetic group. A work is described in this paper using an artificial neural network (ANN) based approach to recognize initial consonant phonemes of Assamese words. A self organizing map (SOM) based algorithm is developed to segment the initial phonemes from its word counterpart. Using a combination of three types of ANN structures, namely recurrent neural network (RNN), SOM and probabilistic neural network (PNN), the proposed algorithm proves its superiority over the conventional discrete wavelet transform (DWT) based phoneme segmentation. The algorithm is exclusively designed on the basis of Assamese phonemical structure which consists of certain unique features and are grouped into six distinct phoneme families. Before applying the segmentation approach using SOM, an RNN is used to take some localized decision to classify the words into six phoneme families. Next the SOM segmented phonemes are classified into individual phonemes. A two-class PNN classification is performed with clean Assamese phonemes, to recognize the segmented phonemes. The validation of recognized phonemes is checked by matching the first formant frequency of the phoneme. Formant frequency of Assamese phonemes, estimated using the pole or formant location determination from the linear prediction model of vocal tract, is used effectively as a priori knowledge in the proposed algorithm.

Spoken word recognition without a TRACE.

How do we map the rapid input of spoken language onto phonological and lexical representations over time? Attempts at psychologically-tractable computational models of spoken word recognition tend either to ignore time or to transform the temporal input into a spatial representation. TRACE, a connectionist model with broad and deep coverage of speech perception and spoken word recognition phenomena, takes the latter approach, using exclusively time-specific units at every level of representation. TRACE reduplicates featural, phonemic, and lexical inputs at every time step in a large memory trace, with rich interconnections (excitatory forward and backward connections between levels and inhibitory links within levels). As the length of the memory trace is increased, or as the phoneme and lexical inventory of the model is increased to a realistic size, this reduplication of time- (temporal position) specific units leads to a dramatic proliferation of units and connections, begging the question of whether a more efficient approach is possible. Our starting point is the observation that models of visual object recognition—including visual word recognition—have grappled with the problem of spatial invariance, and arrived at solutions other than a fully-reduplicative strategy like that of TRACE. This inspires a new model of spoken word recognition that combines time-specific phoneme representations similar to those in TRACE with higher-level representations based on string kernels: temporally independent (time invariant) diphone and lexical units. This reduces the number of necessary units and connections by several orders of magnitude relative to TRACE. Critically, we compare the new model to TRACE on a set of key phenomena, demonstrating that the new model inherits much of the behavior of TRACE and that the drastic computational savings do not come at the cost of explanatory power.

Assessing spoken word recognition in children who are deaf or hard of hearing: a translational approach.

Under natural conditions, listeners use both auditory and visual speech cues to extract meaning from speech signals containing many sources of variability. However, traditional clinical tests of spoken word recognition routinely employ isolated words or sentences produced by a single talker in an auditory-only presentation format. The more central cognitive processes used during multimodal integration, perceptual normalization, and lexical discrimination that may contribute to individual variation in spoken word recognition performance are not assessed in conventional tests of this kind. In this article, we review our past and current research activities aimed at developing a series of new assessment tools designed to evaluate spoken word recognition in children who are deaf or hard of hearing. These measures are theoretically motivated by a current model of spoken word recognition and also incorporate "real-world" stimulus variability in the form of multiple talkers and presentation formats. The goal of this research is to enhance our ability to estimate real-world listening skills and to predict benefit from sensory aid use in children with varying degrees of hearing loss.

Temporal Predictive Codes for Spoken Words in Auditory Cortex

SummaryHumans can recognize spoken words with unmatched speed and accuracy. Hearing the initial portion of a word such as “formu…” is sufficient for the brain to identify “formula” from the thousands of other words that partially match [1–6]. Two alternative computational accounts propose that partially matching words (1) inhibit each other until a single word is selected (“formula” inhibits “formal” by lexical competition [7–9]) or (2) are used to predict upcoming speech sounds more accurately (segment prediction error is minimal after sequences like “formu…” [10–12]). To distinguish these theories we taught participants novel words (e.g., “formubo”) that sound like existing words (“formula”) on two successive days [13–16]. Computational simulations show that knowing “formubo” increases lexical competition when hearing “formu…”, but reduces segment prediction error. Conversely, when the sounds in “formula” and “formubo” diverge, the reverse is observed. The time course of magnetoencephalographic brain responses in the superior temporal gyrus (STG) is uniquely consistent with a segment prediction account. We propose a predictive coding model of spoken word recognition in which STG neurons represent the difference between predicted and heard speech sounds. This prediction error signal explains the efficiency of human word recognition and simulates neural responses in auditory regions.

Sublexical Properties of Spoken Words Modulate Activity in Broca's Area but Not Superior Temporal Cortex: Implications for Models of Speech Recognition

Many models of spoken word recognition posit that the acoustic stream is parsed into phoneme level units, which in turn activate larger representations [McClelland, J. L., & Elman, J. L. The TRACE model of speech perception. Cognitive Psychology, 18, 1-86, 1986], whereas others suggest that larger units of analysis are activated without the need for segmental mediation [Greenberg, S. A multitier theoretical framework for understanding spoken language. In S. Greenberg & W. A. Ainsworth (Eds.), Listening to speech: An auditory perspective (pp. 411-433). Mahwah, NJ: Erlbaum, 2005; Klatt, D. H. Speech perception: A model of acoustic-phonetic analysis and lexical access. Journal of Phonetics, 7, 279-312, 1979; Massaro, D. W. Preperceptual images, processing time, and perceptual units in auditory perception. Psychological Review, 79, 124-145, 1972]. Identifying segmental effects in the brain's response to speech may speak to this question. For example, if such effects were localized to relatively early processing stages in auditory cortex, this would support a model of speech recognition in which segmental units are explicitly parsed out. In contrast, segmental processes that occur outside auditory cortex may indicate that alternative models should be considered. The current fMRI experiment manipulated the phonotactic frequency (PF) of words that were auditorily presented in short lists while participants performed a pseudoword detection task. PF is thought to modulate networks in which phoneme level units are represented. The present experiment identified activity in the left inferior frontal gyrus that was positively correlated with PF. No effects of PF were found in temporal lobe regions. We propose that the observed phonotactic effects during speech listening reflect the strength of the association between acoustic speech patterns and articulatory speech codes involving phoneme level units. On the basis of existing lesion evidence, we interpret the function of this auditory-motor association as playing a role primarily in production. These findings are consistent with the view that phoneme level units are not necessarily accessed during speech recognition.

Sizing up the competition: Quantifying the influence of the mental lexicon on auditory and visual spoken word recognition

Much research has explored how spoken word recognition is influenced by the architecture and dynamics of the mental lexicon (e.g., Luce and Pisoni, 1998; McClelland and Elman, 1986). A more recent question is whether the processes underlying word recognition are unique to the auditory domain, or whether visually perceived (lipread) speech may also be sensitive to the structure of the mental lexicon (Auer, 2002; Mattys, Bernstein, and Auer, 2002). The current research was designed to test the hypothesis that both aurally and visually perceived spoken words are isolated in the mental lexicon as a function of their modality-specific perceptual similarity to other words. Lexical competition (the extent to which perceptually similar words influence recognition of a stimulus word) was quantified using metrics that are well-established in the literature, as well as a statistical method for calculating perceptual confusability based on the phi-square statistic. Both auditory and visual spoken word recognition were influenced by modality-specific lexical competition as well as stimulus word frequency. These findings extend the scope of activation-competition models of spoken word recognition and reinforce the hypothesis (Auer, 2002; Mattys et al., 2002) that perceptual and cognitive properties underlying spoken word recognition are not specific to the auditory domain. In addition, the results support the use of the phi-square statistic as a better predictor of lexical competition than metrics currently used in models of spoken word recognition.

Feedforward and feedback in speech perception: Revisiting analysis by synthesis

We revisit the analysis by synthesis (A×S) approach to speech recognition. In the late 1950s and 1960s, Stevens and Halle proposed a model of spoken word recognition in which candidate word representations were synthesised from brief cues in the auditory signal and analysed against the input signal in tightly linked bottom-up/top-down fashion. While this approach failed to garner much support at the time, recent years have brought a surge of interest in Bayesian approaches to perception, and the idea of A×S has consequently gained attention, particularly in the domain of visual perception. We review the model and illustrate some data from speech perception that are well-accounted for in the context of such an architecture. We focus on prediction in speech perception, an operation at the centre of the A×S algorithm. The data reviewed here and the current possibilities to study online measures of speech processing using cognitive neuroscience methods, in our view, add to a provocative series of arguments why A×S should be reconsidered as a contender in speech recognition research, complementing currently more dominant models.

Orthographic Influences When Processing Spoken Pseudowords: Theoretical Implications

When we hear an utterance, is the orthographic representation of that utterance activated when it is being processed? Orthographic influences have been previously examined in relation to spoken pseudoword processing in three different paradigms. Unlike real word processing, no orthographic effects with pseudowords have been observed in a phoneme goodness ratings task, and there is a mixed outcome in studies looking for spelling–sound consistency effects. In contrast, the orthography of spoken pseudohomographs has been shown to be activated, given that they prime their homographic base word. Explanations are sought for the findings in these three paradigms, leading to an exploration of theoretical models of spoken word recognition.