Theories Of Speech Perception Research Articles

The Acoustics of Speech Communication: Fundamentals, Speech Perception Theory, and Technology

All chapters conclude with Summary. Preface. I.ACOUSTIC PHONETICS: SPEECH ENCODING FROM ARTICULATION TO SOUND STREAM. 1.Language, Phonetics, and Speech Production. Introduction: Language and Science. How Significant Is Acoustic Phonetics? Linguistics, Phonetics, and Phonology. General Conditions of Speech Production. Speech Sound Sources. 2.Sounds, Resonance, and Spectrum Analysis. Sound Production and Propagation. Simple Harmonic Motion. Definitions of Sine Wave Characteristics. Resonance. Definition of Resonant Frequency. Spectrum Analysis. Definitions of Spectrum Terms. Spectra of Recurring Resonant Oscillations. Definitions of Harmonics. Resonant Waves, Spectrum Plots, and Speech Waves. Aperiodic Speech Sounds. 3.Vowel Shaping and Vowel Formants. Model of Pharyngeal-Oral Tract. Spectrum of the Neutral Vowel [e]. Definition of Speech Formants. Vowel Formant Locations and Length of Pharyngeal-Oral Tract. Vocal Tract Constrictions and Formant Frequency Locations. Formants of Model Vowels. Central Vowels. 4.The Glottal Sound Source and the Spectra of Vowels. The Glottal Sound Source. The Phonation Mechanism. The Spectrum of the Glottal Sound Source. Source-Filter Theory of Vowel Production. Visualizing Speech Sounds. Anatomy of Spectrogram. Spectrograms of Vowels. Nasalization of Vowels. Phonemic Nasal Vowels. 5.Prosodic and Tonal Features. Introduction: Telling What and How. Parenting Speech. Prosodic Features of Language Forms. Glottal Source Factors in Stress and Intonation. Durational Prosodic Features. Oral Tract Shaping Factor. Intonation in Discourse. Pacing, Rhythm, and Languages. Tone Languages. 6.Consonant Features, Glides, and Stops. Articulatory Features of Consonants. Distinctive Features. Glide Consonants and Diphthongs. Glide and Voiced Stop. Glide and Stop at Middle Pace. Lateral and Retroflex Glides. Effects of Utterance Position. 7.Consonants: Nasal, Stop, and Fricative Manners of Articulation. Nasal Consonants. Nasal-Glide-Stop Differences. Fricative Consonants. 8.Consonants: The Voiced-Unvoiced Contrast. Production of the Voiced-Voiceless Distinction. Acoustics of Consonant Voicing. Voiced versus Unvoiced Final Consonants. Voiced and Unvoiced Fricatives. Physiological Studies of Consonant Voicing. Physiological Studies of Fricative Voicing. 9.Consonants: Features of Place of Articulation. Formant Transitions of Alveolar versus Labial Consonants. Consonant Place: Transition with Different Vowels. Place Features of Nasal Consonants. Place Features of Fricative Consonants. 10.The Flow of Speech. Coarticulation. Model of Speech Motor Programming. The Syllable as Coarticulation Unit. Effects of Rate of Utterance. Assimilation between Adjacent Consonants. II.SPEECH DECODING BY HUMAN AND MACHINE: FROM SOUND STREAM TO WORDS. 11.Acoustic Cues to Speech Perception, Winifred Strange. Perception of Steady-State Vowels. Speaker Normalization in Vowel Perception. Perception of Coarticulated Vowels. 12.Perception of Vowels: Dynamic Constancy, Winifred Strange. Boundaries of Consonant Categories. Categorical Perception of Speech Continua. Perception of Nonspeech Analogs. Integration of Acoustic Cues. Context and Rate Effects on Phonetic Category Boundaries. 13.Auditory Capacities and Phonological Development: Animal, Baby, and Foreign Listeners, Sarah Hawkins. Background: Does Phonological Perception Use Special Auditory Processes? Techniques for Studying Babies' Speech Perception. Categorical Perception by Babies. Speech Sound Classification by Babies: Prototype and the Perceptual Magnet Effect. Speech Sound Classification by Babies: Constancy. Speech Sound Discrimination by Animals. The Effect of Experience on Speech Sound Discrimination. Development Loss or Selective Attention? The Effect of Retraining. 14.Looking for Invariant Correlates of Linguistic Units: Two Classical Theories of Speech Perception, Sarah Hawkins. Defining the Task of Speech Perception. Overview of Acoustic-Phonetic Theories of Speech Perception. Introduction to Two Classical Theories of Speech Perception. The Motor Theory of Speech Perception. Acoustical Invariants: The Quantal Theory of Speech, Relational Acoustic Invariance, and Lexical Access from Features (LAFF). 15.Reevaluating Assumptions about Speech Perception: Interactive and Integrative Theories, Sarah Hawkins. What Have We Learned from the Classical Theories? Invariance in the Percept but Not the Object: The Theory of Direct Realism. A General Auditory Model without Acoustic Variance: Auditory Enhancement Theory. Categories of Sound? Toward a More Comprehensive Theory of Speech Perception. 16.Hearing Loss and the Audibility of Phoneme Cues, Sally G. Revoile. Consonant Acoustic-Cue Use by a Hypothetical Profoundly Hard of Hearing Person. Phoneme Acoustic-Cue Use by a Hypothetical Severely Hard of Hearing Person. Phoneme Acoustic-Cue Use by a Hypothetical Moderately Hard of Hearing Person. 17.Speech Technology. Speech Machines, J. M. Pickett & Juergen Schroeter. Speech Synthesis, Corine Bickley, Ann Syrdal, & Juergen Schroeter. Speech Recognition by Machine, Diane Kewley-Port. Appendixes. A: Experimenting with Speech. B: Sketches of Some Interesting Books for Phoneticians. References. Index.

Phonetic metamorphosis of white noise: A stop or a fricative

The role of spectro-temporal coherence, lexical status, and word position in the perception of speech in acoustic signals containing a mixture of speech and nonspeech sounds was investigated. Stimuli consisted of nine words in which either white noise was inserted only into the silent interval preceding the onset of vocalic transitions ambiguous between /p/ and /f/, or in which white noise was mixed with the entire speech signal. A control condition contained no noise. Ten adults were asked to decide whether they heard a word with a /p/ or /f/ as the phoneme in the initial, medial, or final word position. The subjects perceived /f/’s 85% of the time when noise was inserted into the silent interval, 47% when noise was presented over the entire word, and 1% in the control condition. These results indicate the importance of spectro-temporal coherence for perceiving speech in a mixture of sounds. The lexical status and position of the critical phoneme in the stimuli also appeared to affect subjects’ responses, although to a smaller degree. Theoretically, the results support the auditory scene analysis view of perceiving speech in heterogeneous signals, while they cannot be adequately explained by the motor theory of speech perception.

Perceptual adjustment to foreign-accented English

A central goal of speech research is to understand how the speech perception system handles variability. Previous work on variability due to differences in speakers’ voices has demonstrated that experience with a voice facilitates word intelligibility [L. C. Nygaard and D. B. Pisoni, Percept. Psychophys. 60, 355–376 (1998)]. The present study investigated whether a more abstract feature of voices, such as foreign accent, can also be perceptually learned and utilized in perception. The following question was tested: Does experience with an accent generalize to the perception of a new voice with the same accent? Two groups were trained with English sentences produced by either Spanish- or Chinese-accented voices. Both groups were then given a word intelligibility test in which sentences were presented in noise. Test sentences included both familiar and new Spanish- and Chinese-accented voices. Training improved intelligibility of familiar voices but did not affect perception of new voices with the trained accents. Results indicate that experience with an accent does not improve perception of a new voice with the same accent. Perceptual learning seems to be restricted to individual voices. Results are discussed with respect to current theories of speech perception. [Work supported by NSF.]

Putting ‘input’ in its proper place

Every theory of second language acquisition (SLA) hypothesizes that learners come to know the properties of a language by being exposed to instances of it in meaningful conversation. Within SLA, the standard analysis of the ‘input question’ appears to be that input equals objective properties of the stimulus array less the effects of selective attention (the intake). In this paper, I offer a critique of the standard analysis. I claim that it is not interpretable in the absence of a theory of mental representation, theories of speech perception and language parsing, and a theory of language learning. I show that it cannot account for aspects of signal detection, phonetic or phonological learning. The standard treatment is essentially pretheoretical. I define an alternative position which distinguishes input to speech processing and input to language-learning mechanisms, drawing on the Autonomous Induction Theory – a novel theory of learning which is compatible with a modular view of processing and a view of linguistic cognition based on Universal Grammar (UG).

Probabilistic Phonotactics and Neighborhood Activation in Spoken Word Recognition

Recent work (Vitevitch & Luce, 1998) investigating the role of phonotactic information in spoken word recognition suggests the operation of two levels of representation, each having distinctly different consequences for processing. The lexical level is marked by competitive effects associated with similarity neighborhood activation, whereas increased probabilities of segments and sequences of segments facilitate processing at the sublexical level. We investigated the two proposed levels in six experiments using monosyllabic and specially constructed bisyllabic words and nonwords. The results of these studies provide further support for the hypothesis that the processing of spoken stimuli is a function of both facilitatory effects associated with increased phonotactic probabilities and competitive effects associated with the activation of similarity neighborhoods. We interpret these findings in the context of Grossberg, Boardman, and Cohen's (1997) adaptive resonance theory of speech perception.

Merits of a Gibsonian approach to speech perception

Neurobiologically inspired theories of speech perception such as that proposed by Sussman et al. are useful to the extent that they are able to constrain such theories. If they are simply intended as suggestive analogies, their usefulness is questionable. In such cases it is better to stick with the Gibsonian approach of attempting to isolate invariants in speech and to demonstrate their role for the perceiver in perceptual experiments.

Spectral transition in the perception of English segments

There is a long-standing debate on whether speech is perceived through steady states or transitions [D. Kewley-Port and D. Pisoni, J. Acoust. Soc. Am. 73, 1779–1793 (1983)]. Furui [S. Furui, J. Acoust. Soc. Am. 80, 1016–1025 (1986)], gating Japanese monomoraic syllables, found listeners identified syllables correctly when they heard the point of maximal spectral transition (maximum D, a measure based on cepstral coefficients). In this paper, this work is extended to English, and to a wider variety of transitions. 128 English words, representing transitions between all manners of articulation, were gated at 20-ms intervals. VC, CC, and VV transitions were used in addition to Furui’s CV and CjV syllables. The results show that for almost all transitions (diphones), there is a clear point at which listener’s identification of the second segment becomes highly accurate. For many diphones (54%), this point is within 15 ms of the maximum D. However, for other diphones (especially those with far-reaching perceptual cues), the point of recognition is farther from the maximum D. This paper will discuss reasons for the discrepancies between Furui’s experiment and the current one, including language-specific factors, and will address implications for the dynamic theory of speech perception.

The Development of Speech Perception: The Transition from Speech Sounds to Spoken Words

Introduction: developing theories of speech perception - constraints from developmental data, Judith C. Goodman et al. Part 1 Innate sensory mechanisms and constraints on learning: observations on speech perception, its development, and the search for a mechanism, Joanne L. Miller and Peter D. Eimas the importance of childhood to language acquisition - evidence from American sign language, Rachel I. Mayberry. Part 2 Perceptual learning of phonological systems: cross-language speech perception - developmental change does not involve loss, Janet F. Werker perceptual learning of non-native speech contrasts - implications for theories of speech perception, David B. Pisoni et al the emergence of native-language phonological influences in infants - a perceptual assimilation model, Catherine T. Best. Part 3 Interactions of linguistic levels - influences on perceptual development: infant speech perception and the development of the mental lexicon, Peter W. Jusczyk sentential processes in early child language - evidence from the perception and production of function morphemes, LouAnn Gerken learning to hear speech as spoken language, Howard C. Nusbaum and Judith C. Goodman.

Speech perception as pattern recognition

This work provides theoretical and empirical arguments in favor of an approach to phonetics that is called double-weak. It is so called because it assumes relatively weak constraints both on the articulatory gestures and on the auditory patterns that map phonological elements. This approach views speech production and perception as distinct but cooperative systems. Like the motor theory of speech perception, double-weak theory accepts that phonological units are modified by context in ways that are important to perception. It further agrees that many aspects of such context dependency have their origin in natural articulatory processes. However, double-weak theory sides with proponents of auditory theories of phonetics by accepting that the real-time objects of perception are well-defined auditory patterns. Because speakers find ways to "orderly" output conditions" (Sussman et al., 1995), listeners are able to successfully decode speech using relatively simple pattern-recognition mechanisms. It is suggested that this situation has arisen through a stylization of gestural patterns to accommodate real-time limits of the perceptual system. Results from a new perceptual experiment, involving a four-dimensional stimulus continuum and a 10-category/hVC/response set, are shown to be largely compatible with this framework.

A computerized adaptive testing system for speech discrimination measurement: the Speech Sound Pattern Discrimination Test.

A computerized, adaptive test-delivery system for the measurement of speech discrimination, the Speech Sound Pattern Discrimination Test, is described and evaluated. Using a modified discrimination task, the testing system draws on a pool of 130 items spanning a broad range of difficulty to estimate an examinee's location along an underlying continuum of speech processing ability, yet does not require the examinee to possess a high level of English language proficiency. The system is driven by a mathematical measurement model which selects only test items which are appropriate in difficulty level for a given examinee, thereby individualizing the testing experience. Test items were administered to a sample of young deaf adults, and the adaptive testing system evaluated in terms of respondents' sensory and perceptual capabilities, acoustic and phonetic dimensions of speech, and theories of speech perception. Data obtained in this study support the validity, reliability, and efficiency of this test as a measure of speech processing ability.

Spontaneous imitation of fundamental frequency and speech rate by nonstutterers and stutterers.

Speech can be described either in terms of acoustics, as a perceptual outcome, or as a motor event. Central to theories of speech perception and production is an attempt to describe how these aspects of speech are interrelated. The present experiment investigated how the nonstutterers' and stutterers' reproductions of acoustically presented interrogative sentences were influenced by experimental variations of intonation (sentence initial vs. sentence final) and speech rate (normal vs. time compressed). We studied the effects of these stimulus manipulations on the speech rate and fundamental frequency (F0) of 10 adult German-speaking nonstutterers and seven stutterers. Experimental manipulations of intonation and speech rate significantly influenced the syllable duration and speech rate of both normal speakers and stutterers. The fundamental frequency of the subjects' responses were also significantly influenced by the intonation of the stimulus. But the stutterers' increase in F0 for stressed syllables was generally less pronounced than that of nonstutterers. These results imply that (a) the subject not only extract linguistic meaning from intonation but that they also store extragrammatical speech rate information, and (b) the speakers adopt these speech rate variations for their own productions. Generally, these results demonstrate that speech perception is not limited to extracting linguistically invariant information. The results show that speakers actively generate their own prosody and that this generative process is influenced by the prosodic structure of another speaker's antecedent speech. The implications of these results for theories of speech production are discussed.

In search of an invariant representation for speech: The modulation spectrogram

The ability to understand speech spoken by diverse speakers under a wide range of acoustic environmental conditions presents a central challenge to current theories of speech perception. Reverberation, background noise, and speaker variation all contribute to the heterogeneous acoustic realization of phonetic information from which listeners routinely derive linguistic information. Representations based on the detailed spectro-temporal properties of speech (e.g., the sound spectrogram) are not sufficiently stable under such conditions as to provide a reasonable account of the processes by which the brain decodes spoken language. A new representational format for visualizing speech, based on the distribution of the modulation spectrum below 16 Hz (and which emphasizes modulation frequencies centered around 4 Hz) across critical-bandlike channels, provides a high degree of stability under reverberant and low signal-to-noise-ratio conditions. This modulation-based representation appears to capture many essential properties of neurons in the primary auditory cortex, and is also well matched to the temporal dynamics of speech production. As such, it may provide a more principled basis for understanding the mechanisms by which human listeners decode the speech signal under natural acoustic conditions than finer-grained spectro-temporal representations.

The effects of the attenuation of second and third formant frequencies on the recognition of stop consonant vowel syllables in aphasic and nonaphasic subjects.

The effects of the attenuation of second and third formant frequencies on the recognition of stop consonant vowel syllables (/p,b,t,d,k,g/ paired with the vowel /a/) in fluent and nonfluent aphasic and nonaphasic normal subjects were examined. Statistical analyses revealed significant differences between groups, between formant conditions, and between stop consonants. All groups responded similarly under the formant conditions, but the attenuation of F2 appeared to account for the most striking disparity between consonant recognition scores irrespective of group. A series of confusion matrices showed that confusions made by aphasic subjects involved both place of articulation and voicing errors, particularly for nonfluent aphasics, and involved the substitution of more anterior consonants. Important cues for the voicing of anterior consonants appeared to be carried by F2. The finding that more intact speech production abilities appeared to facilitate more accurate speech recognition was interpreted as support for the motor theory of speech perception.

Critique: Speech perception: Acoustic or articulatory?

In this review of John Ohala’s paper (‘‘Speech perception is hearing sounds, not tongues’’), we are sympathetic to his position that acoustic properties have largely driven the selection of sounds found in speech production. It is very reasonable that listeners need not refer to articulation during speech perception. However, like many human perception theories, it is difficult (if not impossible) to verify experimentally. The arguments Ohala uses to attack the articulatory theories of speech perception do not really disprove them; instead, they simply argue that an acoustically based theory is more plausible. When dealing with plausibility arguments, one must be careful to ponder the evidence for different positions. Thus this paper comments on the weight of Ohala’s arguments and notes whether any theory of the fundamental basis of speech perception can actually be proven or disproven. It also discusses speech perception from the communication point of view.

From Maine de Biran to the ‘Motor Theory’ of speech

Summary The basic idea of the modern Motor Theory of Speech Perception (Liberman et al. 1963) is that “the perception of speech is tightly linked to the feedback from the speaker’s own articulatory movements”. In this paper we try to show how the same idea was already formulated by the French philosopher Maine de Biran (1805) and taken up in the second half of the 19th century by psychologists (like Steinthal) and linguists (like Kruszewski and Paul). However, whereas in the 19th century the articulatory point of view was not only dominant, but also the only one incorporated in a general theory of language, in the 20th century the articulatory perspective is supplemented by the acoustic one (cf. Malmberg 1967). This was only hinted at by Ferdinand de Saussure in the Cours, but fully expressed in Jakobson & Halle (1956). In this respect, Liberman’s Motor Theory is to be considered much less original than it has been claimed.

Early Understanding and Use of Symbols: The Model Model

3. D.W. Massaro, Broadening the domain of the fuzzy logical model of perception, in Cognition: Con ceptual and Methodological Issues, H.L. Pick, Jr., P. Van den Broek, and D.C. Knill, Eds. (American Psy chological Association, Washington, DC, 1992). 4. M.M. Cohen and D.W. Massaro, Synthesis of visible speech, Behavioral Research Methods, In strumentation, & Computers, 22, 260-263 (1990); M.M. Cohen and D.W. Massaro, Modeling coartic ulation in synthetic visual speech, in Models and Techniques in Computer Animation, N.M. Thal mann and D. Thalmann, Eds. (Springer-Verlag, To kyo, 1993); M.M. Cohen and D.W. Massaro, De velopment and experimentation with synthetic visible speech, Behavioral Research Methods, In strumentation, & Computers, 26, 260-265 (1994); F.l. Parke, A model for human faces that allows speech synchronized animation, Computers and Graphics Journal, 7(1), 1-4(1975). 5. B. de Boysson-Bardies, S. de Schonen, P. Jusczyk, P. McNeilage, and ). Morton, Eds., Devel opmental Neurocognition: Speech and Face Pro cessing in the First Year of Life (Kluwer, Dordrecht, The Netherlands, 1993); C.A. Fowler and D.J. Dekle, Listening with eye and hand: Cross-modal contributions to speech perception, Journal of Ex perimental Psychology: Human Perception and Per formance, 17,8} 6-828 (1991 ); Q. Summerfield, Vi sual perception of phonetic gestures, in Modularity and the Motor Theory of Speech Perception, I.G. Mattingly and M. Studdert-Kennedy, Eds. (Erlbaum, Hillsdale, NJ, 1991). 6. A. Liberman and I.C. Mattingly, The motor theory of speech perception revised, Cognition, 21, 1-33 (1985); I.G. Mattingly and M. Studdert Kennedy, Eds., Modularity and the Motor Theory of Speech Perception (Erlbaum, Hillsdale, NJ, 1991). 7. J.W. Ellison and D.W. Massaro, Evaluating and integrating features in the identification of facial affect, unpublished manuscript, University of Cali fornia, Santa Cruz (1994). 8. S.C. Levine, M.T. Banich, and M.P. Koch Weser, Face recognition: A general or specific right hemisphere capacity? Brain and Cognition, 8, 303 325(1988). 9. B. Dodd and R. Campbell, Eds., Hearing by Eye: The Psychology of Lip-Reading (Erlbaum, Hills dale, NJ, 1987).

Constancy and normalization in the perception of Voice Offset Time as a cue for preaspiration

It is a well-established fact that the realization of individual speech segments is heavily dependent on context. One factor, temporal organization, has been shown to affect numerous speech cues, especially those which are defined in the temporal domain. Theories of speech perception have adopted two different views of the nature of these contextual effects. On the one hand it has been hypothesized that the listener normalizes by taking account of the context. On the other hand it has been hypothesized that higher-order invariants, e.g. speech segment ratios, are sufficient cues for the relevant temporal contrasts, thus obviating the need for mechanisms of normalization. The present series of experiments investigates the merits of these opposing theoretical positions. The experiments involve Voice Offset Time as a cue for preaspiration in Icelandic. The results indicate that durational ratios can, for the most part, function as higher-order invariants for the perception of preaspiration.

Access to phonological structure in listening to speech

In a direct‐realist theory of speech perception, listeners are in immediate (in the sense of unmediated) contact with the phonological units of their language when they use structure in acoustic speech signals as information for its causal source—phonological gestures of the vocal tract. In the theory, phonological categories include, minimally, the sets of motor‐equivalent articulatory movements producible by a synergy of the vocal‐tract, each set, thereby, counting as a token of the same phonological gesture for producer/perceivers of speech. Maximally, categories include a set of similar gestures that members of a language community do not distinguish. Categories, thus, are defined gesturally, not acoustically, as for example, research on prototypes in speech have been interpreted as suggesting. Striking behaviors of listeners that index their extraction of information about phonetic gestures from the acoustic speech signal is their parsing of acoustic signals. A literature review suggests that listeners do not hear such unitary acoustic dimensions as fundamental frequency or duration as unitary. Rather, they parse each dimension into its distinct, converging gestural cause. Complementarily, listeners use as information for a phonological unit the constellation of diverse acoustic consequences of the units gestural realization. [Work supported by NICHD.]

Undoing place assimilation

Phonological variation in the form of different phonetic shapes of words pose a challenge for theories of speech perception and language comprehension. Postlexical phonological processes like assimilation and deletion frequently cause the phonetic shape of a word to change in a given phonological context. Such processes can easily lead to the creation of nonwords. In a sentence like ‘‘I detest green bananas,’’ the word green may well be pronounced as [grim] in the context of a following [b]. Listeners, however, apparently have no problems in parsing and identifying the word correctly in ongoing speech. How is the linguistic system organized such that it can accept the sequence [grim] as the word green? In this talk, competing theoretical accounts dealing with such variation will be presented. An account which assumes a highly abstract lexical representation will be presented. This account assumes that the central component of the linguistic system is the lexicon which contains, among other things, a dictionary of uninflected words with their phonological forms, semantic properties, and syntactic properties of various sorts. The phonological form is assumed to have a highly abstract representation on the basis of which the listener accepts or rejects phonetic variance. In addition, a recent hypothesis will be argued against that assumes an on‐line process of phonological inference and an abstract representation will be presented to deal with phonetic variation. The data to support the hypothesis comes from German assimilation processes.

Learnability Constraints on Theories of Speech Perception: From Spoken Words to Speech Sounds Review of “The Development of Speech Perception: The Transition from Speech Sounds to Spoken Words”

Learnability Constraints on Theories of Speech Perception: From Spoken Words to Speech Sounds Review of “The Development of Speech Perception: The Transition from Speech Sounds to Spoken Words”