Natural Speech Stimuli Research Articles

Consonant recognition in noise with temporal cues: II. Effects of temporal envelope enhancement on response times

In an effort to evaluate the importance of envelope cues in speech recognition, response times and percentages of correct responses were measured simultaneously in normal-hearing and hearing-impaired listeners, using a forced-choice identification paradigm. Naturally produced /aCa/ disyllables presented in a white noise background (+6 dB signal-to-noise ratio) were used to create temporal-only stimuli having instantaneous amplitudes identical to the natural speech stimuli, but flat spectra. The temporal envelope of the /aCa/ disyllables was either left intact or enhanced by raising it to the power 2. In the first experiment, 13 young normal-hearing listeners (24–27 years) were tested. The results show that envelope enhancement yields a general improvement in both percent correct identification (4.75%) and response times (63 ms). In the second experiment, nine hearing-impaired listeners (53–71 years) were tested at a comfortable listening level. Overall, temporal expansion fails to show better consonant identification in terms of percentage of correct responses, but yields a significant improvement (66 ms) in terms of response time. These results suggest that: (1) the perceptual effects of speech-envelope enhancement cannot be completely accounted for by percent correct-responses measurements, and (2) response times seem to be more sensitive to the effects of envelope enhancement.

Perception of dynamic information for vowels in syllable onsets and offsets.

It has been demonstrated using the "silent-center" (SC) syllable paradigm that there is sufficient information in syllable onsets and offsets, taken together, to support accurate identification of vowels spoken in both citation-form syllables and syllables spoken in sentence context. Using edited natural speech stimuli, the present study examined the identification of American English vowels when increasing amounts of syllable onsets alone or syllable offsets alone were presented in their original sentence context. The stimuli were /d/-vowel-/d/ syllables spoken in a short carrier sentence by a male speaker. Listeners attempted to identify the vowels in experimental conditions that differed in the number of pitch periods presented and whether the pitch periods were from syllable onsets or syllable offsets. In general, syllable onsets were more informative than syllable offsets, although neither onsets nor offsets alone specified vowel identity as well as onsets and offsets together (SC syllables). Vowels differed widely in ease of identification; the diphthongized long vowels /e/, /ae/, /o/ were especially difficult to identify from syllable offsets. Identification of vowels as "front" or "back" was accurate, even from short samples of the syllable; however, vowel "height" was quite difficult to determine, again, especially from syllable offsets. The results emphasize the perceptual importance of time-varying acoustic parameters, which are the direct consequence of the articulatory dynamics involved in producing syllables.

Synthetic stimuli attenuate the effect of attention on the dichotic right-ear advantage

This study assessed attentional effects on the right-ear advantage (REA) for a dichotic-listening task that used synthetic-speech syllables. Presenting subjects with monaural tone cues at various intervals prior to dichotic pairs of natural-speech syllables, T. A. Mondor and M. P. Bryden 1991 (The influence of attention on the dichotic REA. Neuropsychologia, 29, 1179–1190) found a reduced REA with longer intervals. This suggested that tone cues at longer intervals helped overcome a right-ear attentional bias. Despite sufficient statistical power, in the present study no reduction in the REA was found with longer intervals between tones and synthetic-speech syllables. As synthetic-speech stimuli tend to fuse better into the percept of a single stimulus than do natural-speech stimuli, attentional effects on the REA may be reduced with dichotic stimuli that fuse.

Categorical Perception as a Function of Stimulus Quality

A number of experiments were carried out in order to test the hypothesis that categorical perception of speech stimuli is a function of synthetis quality – specifically, that the greater complexity of more natural speech stimuli makes it difficult for listeners to focus on particular stimulis parameters as psychoacoustic cues. The results show that there is an increase in categorical perception as synthesis quality improves from a simple synthesis by rule, via LPC synthesis, to a much more complex type called sinewave generation.

Ability to perceive non-native contrasts with natural and synthetic speech stimuli

The perception of the distinction between /r/ and /l/ by native speakers of American English and of Japanese was studied using natural and synthetic speech. The American subjects were all nearly perfect at recognizing the natural speech sounds while there was substantial variation among the Japanese subjects in their accuracy of recognizing naturally produced /r/ and /l/. A logit model, which additively combined the acoustic information conveyed by F1-transition duration and by F3-onset frequency, provided a good fit to the perception of synthetic /r/ and /l/ by the American subjects. There was substantial variation among the Japanese subjects in whether the F1 and F3 cues had a significant effect on their classifications of the synthetic speech; greater use of the F1 and F3 cues in classifying the synthetic speech sounds was positively related to accuracy in recognizing natural /r/ and /l/. These results show that tests using natural and synthetic speech tap into the same variation of ability in perceiving the non-native /r/–/l/ contrast by Japanese speakers. The relation between performance on natural and synthetic speech also provides external validation of the logit model by showing that it predicts performance beyond the data to which it was fit.

Categorical perception as a function of stimulus quality.

A number of experiments were carried out in order to test the hypothesis that the degree of categorical perception of speech stimuli is a function of synthesis quality; specifically, that the greater complexity of more natural speech stimuli makes it difficult for listeners to focus on particular stimulus parameters such as psychoacoustic cues. The results show that there is some increase in categorical perception as synthesis quality improves from a simple synthesis by rule, via LPC synthesis, to a much more complex type called sine-wave generation.

Temporal cues for consonant recognition: training, talker generalization, and use in evaluation of cochlear implants.

Limited consonant phonemic information can be conveyed by the temporal characteristics of speech. In the two experiments reported here, the effects of practice and of multiple talkers on identification of temporal consonant information were evaluated. Naturally produced /aCa/disyllables were used to create "temporal-only" stimuli having instantaneous amplitudes identical to the natural speech stimuli, but flat spectra. Practice improved normal-hearing subjects' identification of temporal-only stimuli from a single talker over that reported earlier for a different group of unpracticed subjects [J. Acoust. Soc. Am. 82, 1152-1161 (1987)]. When the number of talkers was increased to six, however, performance was poorer than that observed for one talker, demonstrating that subjects had been able to learn the individual stimulus items derived from the speech of the single talker. Even after practice, subjects varied greatly in their abilities to extract temporal information related to consonant voicing and manner. Identification of consonant place was uniformly poor in the multiple-talker situation, indicating that for these stimuli consonant place is cued via spectral information. Comparison of consonant identification by users of multi-channel cochlear implants showed that the implant users' identification of temporal consonant information was largely within the range predicted from the normal data. In the instances where the implant users were performing especially well, they were identifying consonant place information at levels well beyond those predicted by the normal-subject data. Comparison of implant-user performance with the temporal-only data reported here can help determine whether the speech information available to the implant user consists of entirely temporal cues, or is augmented by spectral cues.

The role of F1 amplitude in producing nasal percepts

A decrease in F1 amplitude is one of the most consistent acoustic consequences of nasal coupling. Despite the consistency of this effect, its importance for the perception of nasality has been given little attention. Listener judgments of the nasality of naturally produced oral and contextually nasalized vowels were evaluated in light of a time‐varying analysis of relative F1 amplitude (A1‐H1). Both average A1‐H1, and change in A1‐H1 over time, appear to correlate with nasality judgments. The nasalized vowels sometimes judged as oral were those that had larger A1‐H1 values overall, more like those of oral vowels. On the other hand, the oral items sometimes judged as nasal did not have lower overall A1‐H1 values; rather, they showed a marked decrease in A1‐H1 over the course of the vowel, a pattern that was not observed for the oral items consistently judged as oral. Comparable experiments were designed using synthetic stimuli, which provide a control for pitch, vowel quality, and vowel duration differences that might have been present in the natural speech stimuli. Our results highlight the role of dynamic information in the perception of nasality. [Work supported by NIH.]

Right hemisphere EEG sensitivity to speech

Recent speech perception work with normals and aphasics suggests that the right hemisphere may be more adept than the left at making the voicing discrimination, and the reverse for place of articulation. We examined this right hemisphere voicing effect with natural speech stimuli: stop consonants in pre-, mid-, and postvocalic contexts. Using a neuroelectric event-related potential paradigm, we found numerous effects indicating bilateral components reflecting the voicing and place contrast and unique right hemisphere discrimination of both voicing and place of articulation.

Speaker identity verification of spectrally ambiguous and distinguishable speech

Feustel et al. [J. Acoust. Soc. Am. Suppl. 1 83, S55 (1988)] evaluated performance of humans in a speaker identity verification (SIV) task with the intent of improving the performance of a LPC‐cepstral based verification algorithm. This study uses the general methodology and signal processing techniques developed by Feustel et al. to further investigate the nature of potential invariant acoustic information (within a speaker) used by humans to verify a speaker's identity. Subjects were presented natural speech or modified natural speech stimuli drawn from a corpus of spectrally distinguishable and spectrally ambiguous utterances. The results suggest that humans can use information from both the glottal (source) waveform and spectral envelope. Overall, the glottal waveform provides more reliable information. The results will be discussed in this context.

Recognition of English sound contrasts by Arabs

It seems to be widely accepted that there is a biologically determined critical stage for language acquisition in humans during childhood [Slobin (1956, 1971)], indirectly providing support for the observations of language teachers that it is almost difficult to acquire a second language thoroughly after the age of 12 or 13. Such conclusions, however, raise the question of the extent to which remedial training can reverse the diminution of discriminative ability in the course of second language acquisition. Natural speech stimuli were presented to two groups of Arabs majoring in English: (1) relatively inexperienced Arabs and (2) experienced Arabs. Our results show that the experienced group acquired better recognition of English sound contrasts than the inexperienced one. Thus we provide support for the findings of Flege (1983) that the more experience the non‐native speakers gain in the second language, the better discriminative ability is noted. [Research supported by Yarmouk University.]

Release from first-formant masking in presbyacusis.

Using synthetic vowels, the effects of first-formant masking on discrimination of the presence of a transition in the second formant were measured under conditions of first-formant attenuation and dichotic presentation. The subjects were 26 listeners between the ages of 61 and 91 with borderline to moderate sensorineural hearing losses. Results indicated: (1) dichotic presentation was significantly superior to monotic presentation; (2) attenuation of the first formant did not result in significant improvement of second-formant transition discrimination. Improved performance in the dichotic conditions may be related to the fact that subjects selected lower most comfortable listening levels than for monotic conditions. Results are compared to those of another study using natural speech stimuli.

Developmental differences in identifying and discriminating CV syllables.

The purpose of this experiment was to determine whether age-related differences would be observed for identification and discrimination of synthesized, five-formant CV syllables among listeners who showed equal performance scores on a standard clinical test of speech understanding. A second question concerned the relations between performance on identification and discrimination tasks as a function of age. Two 13-item continua that varied in the place of articulation feature ([ba, da, ga]) were used; they differed primarily in the presence of absence of a 5-ms noise burst at the consonant onset. Digitized natural speech syllables were also employed in one experimental task. Strong age effects were obtained for the three tasks--identification of syllables, adaptive estimation of "ba, da-" and "da, ga-" boundaries, and discrimination. With the exception of one condition for six-year-olds, only adults showed significant differences between boundaries and just noticeable differences. Minimal differences were obtained in responses to stimuli with and without initial bursts. Across ages there were no significant differences in the subjects' ability to label the synthesized syllables as compared to the natural speech stimuli. Possible explanations for the observed developmental effects are discussed.

Perceiving vowels in isolation and in consonantal context.

Recent studies have shown that vowels tend to be identified more accurately in consonantal context than in isolation. This contextual advantage is often explained perceptually, e.g., by assuming that the formant transitions associated with the consonants convey significant vowel information. In two experiments with stylized synthetic speech patterns, we were unable to replicate the contextual advantage. These negative results were probably due to certain unnatural stimulus characteristics. In another experiment we used natural speech stimuli to assess whether nonperceptual factors associated with the identification task contribute to the contextual advantage. Subjects responded to the test items either by (a) circling written CVC syllables, (b) circling written isolated vowels, or (c) vocally mimicking the items (a task that we assume imposes minimal memory load on subjects). Of these response conditions, only the first yielded an advantage for vowels in context, suggesting that the effect depends on two factors: Stimulus-response compatibility and memory load.

Manner judgments affect the locus of the labial/dental boundary for stops and fricatives

We investigated place of articulation in labial and dental stops and fricatives using both natural and synthetic speech stimuli. Harris (1958) suggested that place perception for [f], [v], [θ], and [ð] is primarily determined by formant transition cues rather than by properties of the frication. Our results agree: with hers, but also show that perception of the formant transitions is affected by manner of articulation. Specifically, when the frication is removed from natural [θa] the truncated syllable is perceived as beginning with a labial stop [b] instead of the expected dental [d]. Experiments with synthetic stimuli confirmed that the labial—dental boundary was indeed further forward for fricatives than for stops. It is unlikely that this shift is caused by masking from the frication, since a parallel shift could be produced for both natural and synthetic stimuli by instructions to label the stimuli as fricatives even when no frication was present. Since perception of acoustically identical stimuli varied with the phonetic manner class to which they were attributed, we conclude that the effect is phonetic rather than psychoacoustic. [Supported by NICHD and NSERCC.]

Some factors affecting the perceived ordering of natural speech stimuli.

Primary auditory stream segregation--the perceptual segregation of acoustically related elements of a continuous auditory sequence into spatially distinct streams--disrupts recovery of the relative temporal order of repeated sequences of consonant and vowel syllables. Three experiments were performed to determine why the apparent temporal order of natural speech is not similarly disrupted. Exp. 1 (N =24) showed that the disruption is not dependent on repetition of the 32 experimental sequences of consonant and vowel syllables. Exps. 2 (N = 48) and 3 (N = 20) showed that when 4 English monosyllables are used as stimuli and syntactic and intonational structure is present then the temporal integrity of the acoustic signal is preserved perceptually. Despite accurate resolution of order for the 40 experimental sequences, errors of words were common. These errors often imposed a syntactic organization on the resulting sequence.

Cardiac indices of infant speech perception: orienting and burst discrimination.

The present study investigated burst cue discrimination in 3- to 4-month-old infants with the natural speech stimuli [bu] and [gu]. The experimental stimuli consisted of either a [bu] or a [gu] burst attached to the formants of the [bu], such that the sole difference between the two stimuli was the initial burst cue. Infants were tested using a cardiac orienting response (OR) paradigm which consisted of 20 tokens of one stimulus (e.g. [bu]) followed by 20 tokens of the second syllable (20/20 paradigm). An OR to the stimulus change revealed that young infants can discriminate burst cue differences in speech stimuli. Discussion of the results focused on asymmetries observed in the data and the relationship of these findings to our previous failure to demonstrate burst discrimination using the habituation/dishabituation cardiac measure generally employed with older infants.

Preferences of infants for regular and distorted natural speech stimuli

The schema hypothesis proposed by Kagan ( Science, 1970 , 170, 826–832) and Lewis ( Developmental Psychology, 1969 , 1, 75–86) was used to make predictions concerning the preferences of infants 3 to 14 months old for speech stimuli. An operant response method was used in determining the infants' preferences for inflected, monotone, and scrambled natural speech stimuli. Although the infants' preferences did not change with age as predicted, the infants produced clear preference orderings for the three stimuli. The speech preferences were interpreted as being based on stimulus variables (e.g., word order, inflection, and speech rate) in addition to the realism variables assumed by the schema hypothesis.

Syllabic vs segmental perception: On the inability to ignore “irrelevant” stimulus parameters

C. C. Wood and R S. Day have shown that reaction times (RT) to target segments within a syllable are significantly longer when the unattended segment in the syllable (either stop consonant or vowel) is varied randomly than when it is held constant. This has been taken as evidence for the perceptual unity of a stop con sonant-vowel syllable. However, this conclusion could be accepted with much more confidence if it could be shown that random variation in unattended segments does not cause an increase in RT if the unattended segments are not in the same syllable as the target segment. The present study, using bisyllabic natural speech stimuli, (1) reconfirms Wood and Day’s results, but (2) finds that random variation in unattended nontautosyllabic segments also results in significantly longer RTs than without nontautosyllabic variation. Segmental vs syllabic speech perception thus remains an open question.

On the inability to selectively ignore “irrelevant” stimulus parameters

C. C. Wood and R. S. Day [Percept. Psychophys. 17.4, 346–50] have shown that reaction times (RT) to target segments within a syllable are significantly longer when the unattended segment in the syllable (either stop consonant or vowel) is varied randomly than when it is held constant. This has been taken as evidence for the perceptual unity of a stop‐consonant‐vowel syllable. However, this conclusion could be accepted with much more confidence if it could be shown that random variation in unattended segments does not cause an increase in RT if the unattended segments are not in the same syllable as the target segment. The present study, using bisyllabic natural speech stimuli, (1) reconfirms Wood and Day's results but (2) finds that random variation in unattended nontautosyllabic segments also results in significantly longer RT's than without nontautosyllabic random variation. These results will be discussed within the context of general models of the speech perception process. [Work supported by NIH.]