Initial Formant Transitions Research Articles

Phonological Task Enhances the Frequency-Following Response to Deviant Task-Irrelevant Speech Sounds.

In electroencephalography (EEG) measurements, processing of periodic sounds in the ascending auditory pathway generates the frequency-following response (FFR) phase-locked to the fundamental frequency (F0) and its harmonics of a sound. We measured FFRs to the steady-state (vowel) part of syllables /ba/ and /aw/ occurring in binaural rapid streams of speech sounds as frequently repeating standard syllables or as infrequent (p = 0.2) deviant syllables among standard /wa/ syllables. Our aim was to study whether concurrent active phonological processing affects early processing of irrelevant speech sounds reflected by FFRs to these sounds. To this end, during syllable delivery, our healthy adult participants performed tasks involving written letters delivered on a computer screen in a rapid stream. The stream consisted of vowel letters written in red, infrequently occurring consonant letters written in the same color, and infrequently occurring vowel letters written in blue. In the phonological task, the participants were instructed to press a response key to the consonant letters differing phonologically but not in color from the frequently occurring red vowels, whereas in the non-phonological task, they were instructed to respond to the vowel letters written in blue differing only in color from the frequently occurring red vowels. We observed that the phonological task enhanced responses to deviant /ba/ syllables but not responses to deviant /aw/ syllables. This suggests that active phonological task performance may enhance processing of such small changes in irrelevant speech sounds as the 30-ms difference in the initial formant-transition time between the otherwise identical syllables /ba/ and /wa/ used in the present study.

Spectral Integration of Dynamic Cues in the Perception of Syllable-Initial Stops

The present experiments examine the potential role of auditory spectral integration and spectral center of gravity (COG) effects in the perception of initial formant transitions in the syllables [da]-[ga] and [t^ha]-[k^ha]. Of interest is whether the place distinction for stops in these syllables can be cued by a ‘virtual F3 transition’ in which the percept of a frequency transition is produced by a dynamically changing COG. Listeners perceived the virtual F3 transitions comparably with actual F3 transitions although the former were less salient a cue. However, in a separate experiment, static ‘virtual F3 bursts’ were not as effective as actual F3 bursts in cueing the alveolar-velar place distinction. These results indicate that virtual F3 transitions can provide phonetic information to the perceptual system and that auditory spectral integration (completed by the central auditory system) may play a significant role in speech perception.

Topography of the auditory evoked potential in humans reflects differences between vowels embedded in pseudo-words

To study the processing of vowels embedded in more complex linguistic structures, we compared cortical responses for pseudo-words. Auditory evoked potentials were recorded in 11 right-handed females using a passive oddball paradigm, with /pemu/ and /pomu/ as standard stimuli, differing only with respect to the first syllable. Topographic differences in the N100 were observed between the standards: /pemu/ had larger amplitudes than /pomu/ at more posterior electrode sites whereas a reverse pattern was found at more anterior positions along the midline. This topographic difference can be explained by different generators for the two stimuli. Different vowels and/or the initial formant transition possibly activate different neural populations in the auditory cortex, also when the vowels are embedded in pseudo-words.

Learning to perceive speech: how fricative perception changes, and how it stays the same.

A part of becoming a mature perceiver involves learning what signal properties provide relevant information about objects and events in the environment. Regarding speech perception, evidence supports the position that allocation of attention to various signal properties changes as children gain experience with their native language, and so learn what information is relevant to recognizing phonetic structure in that language. However, one weakness in that work has been that data have largely come from experiments that all use similarly designed stimuli and show similar age-related differences in labeling. In this study, two perception experiments were conducted that used stimuli designed differently from past experiments, with different predictions. In experiment 1, adults and children (4, 6, and 8 years of age) labeled stimuli with natural /f/ and /[see text]/ noises and synthetic vocalic portions that had initial formant transitions varying in appropriateness for /f/ or /[see text]/. The prediction was that similar labeling patterns would be found for all listeners. In experiment 2, adults and children labeled stimuli with initial /s/-like and /[see text]/-like noises and synthetic vocalic portions that had initial formant transitions varying in appropriateness for /s/ or /[see text]/. The prediction was that, as found before, children would weight formant transitions more and fricative noises less than adults, but that this age-related difference would elicit different patterns of labeling from those found previously. Results largely matched predictions, and so further evidence was garnered for the position that children learn which properties of the speech signal provide relevant information about phonetic structure in their native language.

Abnormal Auditory Cortical Activation in Dyslexia 100 msec after Speech Onset

Reading difficulties are associated with problems in processing and manipulating speech sounds. Dyslexic individuals seem to have, for instance, difficulties in perceiving the length and identity of consonants. Using magnetoencephalography (MEG), we characterized the spatio-temporal pattern of auditory cortical activation in dyslexia evoked by three types of natural bisyllabic pseudowords (/ata/, /atta/, and /a a/), complex nonspeech sound pairs (corresponding to /atta/ and /a a/) and simple 1-kHz tones. The most robust difference between dyslexic and non-reading-impaired adults was seen in the left supratemporal auditory cortex 100 msec after the onset of the vowel /a/. This N100m response was abnormally strong in dyslexic individuals. For the complex nonspeech sounds and tone, the N100m response amplitudes were similar in dyslexic and nonimpaired individuals. The responses evoked by syllable /ta/ of the pseudoword /atta/ also showed modest latency differences between the two subject groups. The responses evoked by the corresponding nonspeech sounds did not differ between the two subject groups. Further, when the initial formant transition, that is, the consonant, was removed from the syllable /ta/, the N100m latency was normal in dyslexic individuals. Thus, it appears that dyslexia is reflected as abnormal activation of the auditory cortex already 100 msec after speech onset, manifested as abnormal response strengths for natural speech and as delays for speech sounds containing rapid frequency transition. These differences between the dyslexic and nonimpaired individuals also imply that the N100m response codes stimulus-specific features likely to be critical for speech perception. Which features of speech (or nonspeech stimuli) are critical in eliciting the abnormally strong N100m response in dyslexic individuals should be resolved in future studies.

Magnetic brain responses to second formant transitions in dependency of transition time, signal voicing, and attention

Using an oddball paradigm, magnetoencephalographic (MEG) recordings were performed during perception of synthetic /i/-like signals (frequent stimuli) and interspersed /bi/ or /yi/-like deviants with an initial second formant (F2) transition of 20, 40, 60, or 80 ms. The experiment was run under two different signal conditions: formant-synthesized voiced stimuli or spectrally and loudness-matched unvoiced cognates generated by filtered white noise. Listeners’ attention was directed either to the infrequent stimuli or to a moving geometric pattern (visual distraction task). Left- and right-hemispheric M50, M100, and MMNm dipole strengths were derived from the MEG measurements. The results show a strong bilateral effect of signal voicing on the M50 and M100 responses, irrespective of F2 transition time, being enhanced with attention directed to the acoustic signal. The deviant stimuli evoked a significant mismatch reaction which under directed attention was lateralized to the left hemisphere. The strength of the MMNm increased across transition time, but exhibited a nonlinearity at 40 ms in terms of an interaction with signal voicing. In conclusion, these results suggest signal-related bilateral preprocessing during the M50 and M100 responses and attention-dependent lateralization of mismatch reactions to speech-relevant acoustic features.

Neural dynamics of perceptual order and context effects for variable-rate speech syllables.

How does the brain extract invariant properties of variable-rate speech? A neural model, called PHONET, is developed to explain aspects of this process and, along the way, data about perceptual context effects. For example, in consonant-vowel (CV) syllables, such as /ba/ and /wa/, an increase in the duration of the vowel can cause a switch in the percept of the preceding consonant from /w/ to /b/ (J.L. Miller & Liberman, 1979). The frequency extent of the initial formant transitions of fixed duration also influences the percept (Schwab, Sawusch, & Nusbaum, 1981). PHONET quantitatively simulates over 98% of the variance in these data, using a single set of parameters. The model also qualitatively explains many data about other perceptual context effects. In the model, C and V inputs are filtered by parallel auditory streams that respond preferentially to the transient and sustained properties of the acoustic signal before being stored in parallel working memories. A lateral inhibitory network of onset- and rate-sensitive cells in the transient channel extracts measures of frequency transition rate and extent. Greater activation of the transient stream can increase the processing rate in the sustained stream via a cross-stream automatic gain control interaction. The stored activities across these gain-controlled working memories provide a basis for rate-invariant perception, since the transient-to-sustained gain control tends to preserve the relative activities across the transient and sustained working memories as speech rate changes. Comparisons with alternative models tested suggest that the fit cannot be attributed to the simplicity of the data. Brain analogues of model cell types are described.

A Constraint on the Discrimination of Speech by Young Infants

Three- and four-month-old infants were tested on their ability to discriminate the third-formant transitions sufficient to signal the syllable-medial stops [t] and [k]. The stimulus patterns consisted of an initial fricative [s], 20 or 100 msec of silence, and the vowel [a], whose initial formant transitions were appropriate for [t] or [k]. Discrimination only occurred when the duration of silence was 100 msec. This constraint on discrimination is discussed in terms of a psychoacoustic explanation based on forward masking and in terms of the hypothesis that the processing of speech signals involves a species-specific system dedicated to deriving a phonetic message.

Macaques show context effects in speech perception

The phonetic distinction between the stop consonant /b/ and semivowel /w/ is cued primarily by duration of the initial formant transition. Syllable duration, however, is a context that influences human adults' and infants' perception of this distinction. For example, when overall syllable duration is long (296 ms), infants show poor discrimination of syllables with transition durations of 16 and 40 ms, but good discrimination of syllables with transition durations of 40 and 64 ms. However, when overall syllable duration is short (80 ms), the opposite is true [P. D. Eimas and J. L. Miller, Science 209, 1140–1141 (1980)]. The current study addresses whether rhesus and Japanese macaques' perception of the /b-w/ distinction is also influenced by syllable duration: Six CV syllables used in the Eimas and Miller (1980) study were presented to five macaques using a same-different discrimination paradigm. Results showed that macaques' pattern of discrimination was similar to the pattern shown by infants, thus supporting an auditory-based mechanism for the effect of syllable duration in phonetic perception. [Work supported by NSF and NIH.]

Effects of stress and final consonant voicing on vowel articulation and formant patterns

Stress level and voicing of a postvocalic consonant are two factors known to influence vowel duration. In the present study, vowel durations, jaw and lower lip movement data, and formant data were examined for a set of b‐vowel‐consonant utterances differing in stress and final consonant voicing. As expected, vowel durations were greater for stressed utterances and for utterances containing voiced final consonants (voiced utterances). Articulator lowering and raising gestures were more rapid and more extensive for stressed versus unstressed utterances. At the acoustic level, stressed uterances demonstrated more rapid initial formant transitions and more extreme steady‐state frequencies than unstressed utterances. In contrast, increases in vowel duration due to voicing were not accompanied by increases in the velocity or magnitude of articulatory gestures. In addition, voiced utterances did not demonstrate steeper formant transitions or more extreme steady‐state frequencies than voiceless utterances. The results demonstrate that stress‐related and voicing‐related changes in vowel duraton are accomplished by separate and distinct transformations with observable consequences at both the articulatory and acoustic level. [Work supported by NINCDS and the Stuttering Center, Baylor College of Medicine.]

Limits of later-occurring rate information for phonetic perception.

It has been shown that when listeners identify monosyllabic target words as beginning with /b/ versus /w/ on the basis of the duration of the initial formant transitions, they do so in relation to the rate of the word. Specifically, as speaking rate decreases such that the word becomes longer, the transition duration required to perceive /w/ as opposed to /b/ also becomes longer. We investigated whether the rate of speech subsequent to the target word also influences perception of the word-initial /b/—/w/ contrast. In a series of studies focusing on the monosyllabic target words bill and will, we found no evidence for the influence of this later-occurring rate information on identification. In each case, however, we did find the expected influence of the rate of the word itself. These results indicate that the domain of the relevant later-occurring rate information is limited. Possible temporal, structural, and lexical factors underlying this limitation are discussed.

Extending formant transitions may not improve aphasics' perception of stop consonant place of articulation

Synthetic speech stimuli were used to investigate whether aphasics' ability to perceive stop consonant place of articulation was enhanced by the extension of initial formant transitions in CV syllables. Phoneme identification and discrimination tests were administered to 12 aphasic patients, 5 fluent and 7 nonfluent. There were no significant differences in performance due to the extended transitions, and no systematic pattern of performance due to aphasia type. In both groups, discrimination was generally high and significantly better than identification, demonstrating that auditory capacity was retained, while phonetic perception was impaired; this result is consistent with repeated demonstrations that auditory and phonetic processes may be dissociated in normal listeners. Moreover, significant rank order correlations between performances on the Token Test and on both perceptual tasks suggest that impairment on these tests may reflect a general cognitive rather than a language-specific deficit.

Some effects of speaking rate on the production of /b/ and /w/.

One of the acoustic properties distinguishing the syllable-initial stop consonant /b/ from the semivowel /w/ is the duration of the initial formant transitions; syllables beginning with /b/ have shorter transitions than those beginning with /w/. This experiment investigated the way in which the transition durations of /b/ and /w/ change as a function of speaking rate by examining tokens of /ba/ and /wa/ produced by four male speakers. At any given speaking rate the /wa/ transitions were, on average, longer than the /ba/ transitions, although pooled across rates, the distributions of transition duration for /ba/ and /wa/ were overlapping. In addition, the magnitude of the difference between average /ba/ and /wa/ transition durations increased with decreases in speaking rate. This is because as rate of speech decreased so that syllable duration increased, there was little change in the initial transition duration of /ba/, but a considerable increase in the initial transition duration of /wa/. Given the overall pattern of results, the transition duration that could optimally distinguish /ba/ from /wa/ was not constant, but increased with syllable duration. This is in accord with Miller and Liberman's (1979) finding that when listeners identify /ba/ and /wa/ on the basis of transition duration, they do so in relation to the duration of the syllable.

Speaking rate affects perception of good exemplars of a phonetic category

An important property distinguishing syllable‐initial stop consonant, /b/, from semivowel, /w/, is the duration of the initial formant transitions: Relatively short transitions specify /b/, whereas longer transitions specify /w/. We reported earlier [J. L. Miller and A.M. Liberman, J. Acoust. Soc. Am. 68, S31 (1978)] that the transition duration at the /b/‐/w/ phonetic boundary is not absolute. Instead, as the syllables become longer, an increasingly longer transition is required to hear /w/. We proposed that the alteration in syllable duration specifies a change in speaking rate, and that the shift in boundary location reflects the listener's adjustment to that change. In the present experiment we asked whether the effect of syllable duration is limited to stimuli near the /b/‐/w/ boundary or extends to the processing of good exemplars of the phonetic categories. A selective adaptation procedure was used to assess perception of two series of syllables that varied in transition duration and ranged from /bæ/ to /wæ/; the series differed from each other only in overall syllable duration. On the basis of the adaptation data we were able to infer that both the location of the /b/‐/w/ boundary and the location of the prototypical or “best” /wæ/ were located at a longer transition duration for the long syllables, as compared to the short syllables. These data indicate that a change in speaking rate affects the way in which acoustic information is mapped onto phonetic categories, in terms of both the boundary between categories and the best exemplars of a given category. [Supported by NIH.]

Some Effects of Speaking Rate on Phonetic Perception

This paper reviews a series of studies on the effects of variation in speaking rate on phonetic perception, in particular, on the identification of /b/ and /w/ in syllable-initial position. The major finding of these studies was that in a variety of tasks listeners adjusted for speaking rate when using the duration of the initial formant transitions to distinguish between /b/ and /w/: At slower rates of speech, a longer transition was needed to hear /w/ rather than /b/, and conversely at faster rates of speech. This effect occurred when rate was specified by the duration and acoustic-phonetic structure of the syllable containing the target consonant and, to a more limited extent, when it was specified by the duration of a subsequent syllable. Moreover, the influence of syllable duration on the processing of transition duration was also evident in young, prearticulate infants, suggesting that at least the rudiments of a system that provides perceptual constancy across rate is part of the linguistic endowment of the infant.

Some effects of acoustic attributes of speech on the processing of phonetic feature information.

How do acoustic attributes of the speech signal contribute to feature-processing interactions that occur in phonetic classification? In a series of five experiments addressed to this question, listeners performed speeded classification tasks that explicitly required a phonetic decision for each response. Stimuli were natural consonant-vowel syllables differing by multiple phonetic features, although classification responses were based on a single target feature. In control tasks, no variations in nontarget features occurred, whereas in orthogonal tasks nonrelevant feature variations occurred but had to be ignored. Comparison of classification times demonstrated that feature information may either be processed separately as independent cues for each feature or as a single integral segment that jointly specifies several features. The observed form on processing depended on the acoustic manifestations of feature variation in the signal. Stop-consonant place of articulation and voicing cues, conveyed independently by the pattern and excitation source of the initial formant transitions, may be processed separately. However, information for consonant place of articulation and vowel quality, features that interactively affect the shape of initial formant transitions, are processed as an integral segment. Articulatory correlates of each type of processing are discussed in terms of the distinction between source features that vary discretely in speech production and resonance features that can change smoothly and continuously. Implications for perceptual models that include initial segmentation of an input utterance into a phonetic feature representation are also considered.

Perceptual equivalence of cues for a phonetic trading relation: Primacy of phonetic over psychoacoustic effects

Recently a phonetic trading relation was found for two different cues to the identification of [split] versus [slit], namely, silent closure duration versus initial formant transition differences for the [plit]-[lit] contrast. Within limits, varying either cue produces the phonetic contrast; an extra amount of one compensates for lack of the other. [Erickson, Fitch, Halwes, and Liberman, J. Acoust. Soc. Am. Suppl. 1 61, S46(A) (1977)]. Subsequently, a three-way discrimination test was devised to more clearly show that the phonetic effects of the independent cue manipulations are perceptually equivalent [Fitch, Halwes, Erickson, and Liberman, Haskins Laboratories: Status Report (1979)]. The researchers hypothesized that the trading relations and perceptual equivalence phenomena reflect perceptual specialization that takes account of the common origin of cues in single articulatory gestures. But the phonetic generality of the paired phenomena needs assessment; therefore, we used their procedures to test the simpler [stay]-[say] contrast. Both phenomena were replicated and virtually identical in magnitude to those reported earlier. To test the phonetic specificity of the phenomena, we ran a perceptual equivalence discrimination test with nonspeech sine-wave analogs of the stimuli, using naive subjects. Results suggest that, in perceiving stimuli, speech sharply increases the effectiveness and shifts the absolute value, of a weak psychoacoustic boundary. [Work supported by NICHD.]

Some effects of later-occurring information on the perception of stop consonant and semivowel.

In three experiments, we determined how perception of the syllable-initial distinction between the stop consonant [b] and the semivowel [w], when cued by duration of formant transitions, is affected by parts of the sound pattern that occur later in time. For the first experiment, we constructed four series of syllables, similar in that each had initial formant transitions ranging from one short enough for [ba] to one long enough for [wa], hut different in overall syllable duration. The consequence in perception was that, as syllable duration increased, the [b-w] boundary moved toward transitions of longer duration. Then, in the second experiment, we increased the duration of the sound by adding a second syllable, [da], (thus creating [bada-wada]), and observed that lengthening the second syllable also shifted the perceived [b-w] boundary in the first syllable toward transitions of longer duration; however, this effect was small by comparison with that produced when the first syllable was lengthened equivalently. In the third experiment, we found that altering the structure of the syllable had an effect that is not to be accounted for by the concomitant change in syllable duration: lengthening the syllable by adding syllable-final transitions appropriate for the stop consonant [d] (thus creating [bad-wad]) caused the perceived [b-w] boundary to shift toward transitions of shorter duration, an effect precisely opposite to that produced when the syllable was lengthened to the same extent by adding steady-state vowel. We suggest that, in all these cases, the later-occurring information specifies rate of articulation and that the effect on the earlier-occurring cue reflects an appropriate perceptual normalization.

Anchoring effects on a synthetic consonant continuum

Anchoring effects on several acoustic continua were studied by manipulating the probabilities of occurrence of end-point stimuli. Two six-step double continua were constructed by simultaneously varying both the initial formant transitions of a CV syllable and either its pitch or its vowel. Thus, the first continuum ranged from low-pitched /bae/ (114 Hz) to high-pitched /dae/ (150 Hz); the second from /bae/ to /dɛ/. Two random-order test tapes were prepared for each double continuum: in one (control) all seven stimuli occurred equally often, in the other (anchor) an end-point stimulus (either /bae/ — “low” or /bae/ occurred four times as often as each of the other six stimuli. Nine subjects listened to each tape twice, once judging the initial consonant of each syllable (/b/, /d/), once judging either its pitch (low, high) or its vowel (/ae/, /ɛ/). On the first test, the mean category boundary was significantly shifted toward the anchoring stimulus on the pitch judgment, but not on the consonant judgment. On the second test, the mean category boundary was significantly shifted toward the anchor stimulus on both vowel and consonant judgments. Judgments of synthetic consonants are thus not always immune to the psychophysical effects of anchoring. [Work supported by NICHD.]

Second Formant Transitions of Selected Consonant-Vowel Combinations in the Speech of Deaf and Normal-Hearing Children

Although it is well known that the speech produced by the deaf is generally of low intelligibility, the sources of this low speech intelligibility have generally been ascribed either to aberrant articulation of phonemes or inappropriate prosody. This study was designed to determine to what extent a nonsegmental aspect of speech, formant transitions, may differ in the speech of the deaf and of the normal hearing. The initial second formant transitions of the vowels /i/ and /u/ after labial and alveolar consonants (/b, d, f/) were compared in the speech of six normal-hearing and six hearing-impaired adolescents. In the speech of the hearing-impaired subjects, the second formant transitions may be reduced both in time and in frequency. At its onset, the second formant may be nearer to its eventual target frequency than in the speech of the normal subjects. Since formant transitions are important acoustic cues for the adjacent consonants, reduced F2 transitions may be an important factor in the low intelligibility of the speech of the deaf.