Formant Discrimination Research Articles

Perceptual formant discrimination during speech movement planning.

Evoked potential studies have shown that speech planning modulates auditory cortical responses. The phenomenon's functional relevance is unknown. We tested whether, during this time window of cortical auditory modulation, there is an effect on speakers' perceptual sensitivity for vowel formant discrimination. Participants made same/different judgments for pairs of stimuli consisting of a pre-recorded, self-produced vowel and a formant-shifted version of the same production. Stimuli were presented prior to a "go" signal for speaking, prior to passive listening, and during silent reading. The formant discrimination stimulus /uh/ was tested with a congruent productions list (words with /uh/) and an incongruent productions list (words without /uh/). Logistic curves were fitted to participants' responses, and the just-noticeable difference (JND) served as a measure of discrimination sensitivity. We found a statistically significant effect of condition (worst discrimination before speaking) without congruency effect. Post-hoc pairwise comparisons revealed that JND was significantly greater before speaking than during silent reading. Thus, formant discrimination sensitivity was reduced during speech planning regardless of the congruence between discrimination stimulus and predicted acoustic consequences of the planned speech movements. This finding may inform ongoing efforts to determine the functional relevance of the previously reported modulation of auditory processing during speech planning.

Auditory Models for Formant Frequency Discrimination of Vowel Sounds

As formant frequencies of vowel sounds are critical acoustic cues for vowel perception, human listeners need to be sensitive to formant frequency change. Numerous studies have found that formant frequency discrimination is affected by many factors like formant frequency, speech level, and fundamental frequency. Theoretically, to perceive a formant frequency change, human listeners with normal hearing may need a relatively constant change in the excitation and loudness pattern, and this internal change in auditory processing is independent of vowel category. Thus, the present study examined whether such metrics could explain the effects of formant frequency and speech level on formant frequency discrimination thresholds. Moreover, a simulation model based on the auditory excitation-pattern and loudness-pattern models was developed to simulate the auditory processing of vowel signals and predict thresholds of vowel formant discrimination. The results showed that predicted thresholds based on auditory metrics incorporating auditory excitation or loudness patterns near the target formant showed high correlations and low root-mean-square errors with human behavioral thresholds in terms of the effects of formant frequency and speech level). In addition, the simulation model, which particularly simulates the spectral processing of acoustic signals in the human auditory system, may be used to evaluate the auditory perception of speech signals for listeners with hearing impairments and/or different language backgrounds.

Effects of word context on formant discrimination

Studies of phonetic variation identify systematic, but often very small, acoustic differences in the realization of vowels. However, it is not known whether such differences are perceptually relevant in realistic contexts. We examine perceptibility of incrementally manipulated differences in F1 and F2 in vowels embedded in real words of English (experiment 1), and vowels in isolation (experiment 2), via an AX discrimination task. Forty listeners in each experiment, recruited through Amazon Mechanical Turk, heard 27 monosyllabic words or vowels in pairs containing a no-change token and a token edited to differ up to ±100 Hz in F1 or ±150 Hz in F2 relative to the original and were asked to say whether tokens were same or different. Discrimination increased with bigger vowel differences (both F1 and F2) in both experiments; however, even maximally different tokens were correctly discriminated less than half the time. F2 discrimination was better in isolated vowels (48%) than in whole words (33%). Compared to previous results in optimized difference limen studies (e.g., Kewley-Port and Watson, 1994), our results show that untrained listeners in more typical listening conditions (non-synthetic speech heard in non-laboratory conditions in real word contexts) require much bigger differences to be perceptible.

Basic auditory processing and emotion recognition in individuals at clinical high risk for psychosis

The ability to understand others' emotions from subtle cues is a critical skill for navigating the complex social world, with increasing demands placed on this capability as children enter adolescence. Individuals at clinical high-risk (CHR) of developing psychosis exhibit social isolation with specific impairments in social cognition (Thompson et al., 2011) and emotion recognition (Addington et al., 2008; Amminger et al., 2011; Corcoran et al., 2015). The degree of social impairment and the severity of emotion recognition deficits are greater in individuals who later convert to psychosis than in those who do not (Cornblatt et al., 2011). Emotion recognition deficits thus represent promising targets for improving early identification and intervention strategies (Addington et al., 2008; Amminger et al., 2011; Cornblatt et al., 2011; Kee et al., 2003; Niemi et al., 2005). A significant challenge remains however, to identify a precursor of emotional recognition deficits for use as a salient diagnostic tool and therapeutic target. Impairments in basic auditory processes may underlie the auditory emotion identification deficits and social functioning deficits associated with psychotic disorders and at-risk states. The auditory properties of speech convey emotional content (Banse and Scherer, 1996; Hammerschmidt and Jurgens, 2007), and the ability to identify emotion from these vocal cues is impaired in schizophrenia (Addington et al., 2008; Amminger et al., 2011; Bach et al., 2009; Corcoran et al., 2015; Cornblatt et al., 2011; Dickey et al., 2010; Dickey et al., 2008; Kee et al., 2003; Leitman et al., 2007; Leitman et al., 2010; Niemi et al., 2005; Thompson et al., 2011), schizotypal personality disorder (Dickey et al., 2010; Dickey et al., 2008), and CHR subjects (Corcoran et al., 2015; Thompson et al., 2011). We recently reported that individuals with schizophrenia exhibit wide ranging basic auditory processing deficits, and that additional basic auditory processing abilities, including formant discrimination, sinusoidal amplitude modulation detection, and duration discrimination are associated with the ability to recognize emotion in voice (Kraus et al., 2019). In this study, we have aimed to determine whether CHR individuals exhibit similar widespread auditory processing deficits associated with impaired emotion recognition.

Compensation to Altered Auditory Feedback in Children With Developmental Language Disorder and Typical Development.

Purpose Developmental language disorder (DLD), an unexplained problem using and understanding spoken language, has been hypothesized to have an underlying auditory processing component. Auditory feedback plays a key role in speech motor control. The current study examined whether auditory feedback is used to regulate speech production in a similar way by children with DLD and their typically developing (TD) peers. Method Participants aged 6-11 years completed tasks measuring hearing, language, first formant (F1) discrimination thresholds, partial vowel space, and responses to altered auditory feedback with F1 perturbation. Results Children with DLD tended to compensate more than TD children for the positive F1 manipulation and compensated less than TD children in the negative shift condition. Conclusion Our findings suggest that children with DLD make atypical use of auditory feedback.

Domain-general auditory processing determines success in second language pronunciation learning in adulthood: A longitudinal study

AbstractIn this study, we propose a hypothesis that domain-general auditory processing, a perceptual–cognitive anchor of first language (L1) acquisition, can serve as an important deciding factor for successful postpubertal second language (L2) pronunciation learning. To examine this hypothesis, samples of spontaneous speech were elicited from a total of 30 L1 Chinese L2 English learners at two points (outset and endpoint) during an 8-month study-abroad period in the United Kingdom. The participants were tested on three different components of auditory processing ability (formant, pitch, and duration discrimination) using behavioral instruments. The auditory processing scores were then linked to the segmental, prosodic, and fluency dimensions of their L2 pronunciation proficiency development throughout the project. Overall, most learners’ speech became smoother, faster, and more fluent (fewer pauses, faster articulation rate, and more optimal perceived tempo). Certain learners with high-level auditory processing ability (more precise formant discrimination) appeared to further attain more correct pronunciation of individual sounds and words (greater segmental and word stress accuracy), leading to more advanced L2 phonological skills (fluent and accurate). The findings suggest that auditory processing abilities can be a root of language learning throughout the life span and may apply to the initial- to midphase of naturalistic L2 pronunciation learning in adulthood.

Lack of association between contralateral inhibition of otoacoustic emissions and vowel formant discrimination in noise

Purpose The medial olivocochlear (MOC) reflex enhances neural encoding of signals in noise, and measurement of its function may hold clinical utility. Previous research on how the reflex aids speech-in-noise perception has been equivocal. Motivated by animal work, we examined associations between MOC reflex activity and formant discrimination in noise in humans to better understand how the MOC reflex contributes to audition. We hypothesised that participants with stronger MOC reflex activity would have better formant discrimination in noise abilities. Method Twenty-six normal-hearing listeners met all inclusion and exclusion criteria (mean age = 21.5 years), with data from 25 participants included in the final analysis. Transient-evoked otoacoustic emissions (TEOAEs) were measured in right ears. MOC reflex activity was assessed using a contralateral inhibition paradigm in which the change in TEOAE amplitude without versus with a contralateral MOC reflex elicitor was computed. Formant discrimination thresholds for a synthetic vowel /ɛ/ were obtained in right ears using a two-alternative forced-choice procedure that adaptively varied the second formant frequency. Discrimination thresholds were obtained at three signal-to-noise ratios (SNRs). Results TEOAE amplitudes were significantly reduced in the presence of the reflex elicitor (p < .05). Discrimination thresholds decreased significantly with increasing SNR (p < .05 in all cases). No significant correlations were found between contralateral inhibition measures and discrimination thresholds at any SNR (p > .05 in all cases). Conclusion Contrary to hypothesis, no significant associations were found between contralateral inhibition and formant discrimination in noise performance. It is possible that the MOC reflex contributes to formant discrimination but not in a monotonic fashion. Future work should consider investigating how the MOC reflex contributes to other perceptual properties to better characterise the functional relevance of the MOC reflex.

A new measure of authentic auditory emotion recognition: Application to patients with schizophrenia

BackgroundMany social processes such as emotion recognition are severely impaired in patients with schizophrenia. While basic auditory processing seems to play a key role in identifying emotions, research in this field is limited due to the lack of proper assessment batteries. Many of the widely accepted tests utilize actors to portray certain emotions—these batteries are less ecologically and face valid. MethodsThis study utilized a newly developed auditory emotion recognition test that contained natural stimuli from spontaneous displays of emotions to assess 28 patients with schizophrenia and 16 healthy controls. ResultsThe results indicate that the newly developed test, referred to as the INTONATION Test, is more sensitive to the emotion recognition deficits in patients with schizophrenia than previously used measures. The correlations of the INTONATION Test measures with basic auditory processes were similar to established tests of auditory emotion. Particular emotion sub scores from the INTONTATION test, such as happiness, demonstrated the strongest correlations with specific auditory processing skills, such as formant discrimination and sinusoidal amplitude modulation detection (SAM60). ConclusionsThe results from this study indicate that auditory emotion recognition impairments are more pronounced in patients with schizophrenia when perceiving authentic displays of emotion. Understanding these deficits could help specify the nature of auditory emotion recognition deficits in patients with schizophrenia and those at risk.

Vowel discrimination in noise with formant enhancement: Effects of hearing loss and aging

The aim of this study was to investigate whether formant enhancement of F2 would improve vowel formant discrimination in noise for three groups of listeners (i.e., older listeners with hearing loss, older listeners with normal hearing, and young listeners with normal hearing). Thresholds of vowel formant discrimination were examined for the F2 frequencies of three American English vowels: /ʌ/, /ɛ/, and /i/ in long-term speech-shaped noise with signal-to-noise ratio of + 6 dB and + 12 dB. F2 frequencies of three vowels were enhanced by 6 dB. Thresholds of vowel formant discrimination were significantly highest for older listeners with hearing loss, while young listeners with normal hearing had lowest thresholds with the older listeners with normal hearing in between. However, discrimination thresholds were improved significantly for all three groups after F2 enhancement was implemented regardless of low or high SNR of background noise. Hearing loss and aging resulted in higher thresholds of vowel formant discrimination in noise and F2 enhancements could improve discrimination performance for all listeners in noise.

Formant discrimination of speech and non-speech sounds in temporally modulated noise: Effect of language experience

Formant discrimination of speech and non-speech sounds in temporally modulated noise: Effect of language experience

Basic auditory processing deficits and their association with auditory emotion recognition in schizophrenia

BackgroundIndividuals with schizophrenia are impaired in their ability to recognize emotions based on vocal cues and these impairments are associated with poor global outcome. Basic perceptual processes, such as auditory pitch processing, are impaired in schizophrenia and contribute to difficulty identifying emotions. However, previous work has focused on a relatively narrow assessment of auditory deficits and their relation to emotion recognition impairment in schizophrenia. MethodsWe have assessed 87 patients with schizophrenia and 73 healthy controls on a comprehensive battery of tasks spanning the five empirically derived domains of auditory function. We also explored the relationship between basic auditory processing and auditory emotion recognition within the patient group using correlational analysis. ResultsPatients exhibited widespread auditory impairments across multiple domains of auditory function, with mostly medium effect sizes. Performance on all of the basic auditory tests correlated with auditory emotion recognition at the p < .01 level in the patient group, with 9 out of 13 tests correlating with emotion recognition at r = 0.40 or greater. After controlling for cognition, many of the largest correlations involved spectral processing within the phase-locking range and discrimination of vocally based stimuli. ConclusionsWhile many auditory skills contribute to this impairment, deficient formant discrimination appears to be a key skill contributing to impaired emotion recognition as this was the only basic auditory skill to enter a step-wise multiple regression after first entering a measure of cognitive impairment, and formant discrimination accounted for significant unique variance in emotion recognition performance after accounting for deficits in pitch processing.

Auditory Perceptual Abilities Are Associated with Specific Auditory Experience.

The extent to which auditory experience can shape general auditory perceptual abilities is still under constant debate. Some studies show that specific auditory expertise may have a general effect on auditory perceptual abilities, while others show a more limited influence, exhibited only in a relatively narrow range associated with the area of expertise. The current study addresses this issue by examining experience-dependent enhancement in perceptual abilities in the auditory domain. Three experiments were performed. In the first experiment, 12 pop and rock musicians and 15 non-musicians were tested in frequency discrimination (DLF), intensity discrimination, spectrum discrimination (DLS), and time discrimination (DLT). Results showed significant superiority of the musician group only for the DLF and DLT tasks, illuminating enhanced perceptual skills in the key features of pop music, in which miniscule changes in amplitude and spectrum are not critical to performance. The next two experiments attempted to differentiate between generalization and specificity in the influence of auditory experience, by comparing subgroups of specialists. First, seven guitar players and eight percussionists were tested in the DLF and DLT tasks that were found superior for musicians. Results showed superior abilities on the DLF task for guitar players, though no difference between the groups in DLT, demonstrating some dependency of auditory learning on the specific area of expertise. Subsequently, a third experiment was conducted, testing a possible influence of vowel density in native language on auditory perceptual abilities. Ten native speakers of German (a language characterized by a dense vowel system of 14 vowels), and 10 native speakers of Hebrew (characterized by a sparse vowel system of five vowels), were tested in a formant discrimination task. This is the linguistic equivalent of a DLS task. Results showed that German speakers had superior formant discrimination, demonstrating highly specific effects for auditory linguistic experience as well. Overall, results suggest that auditory superiority is associated with the specific auditory exposure.

Vowel dynamics and sentence processing

Throughout Terrance Nearey’s influential academic career, research on the acoustic properties, models and theories of vowels has been a major focus. Studies by Nearey and his colleagues demonstrated the importance of spectral change (VISC) in vowel identification, even for monophthongal vowels. This presentation describes our research on vowel dynamics inspired by VISC research. Starting with syllables, the auditory system can discriminate F1 and F2 formants transition differences very well. Further results using high-fidelity resynthesis with sentences revealed that formant discrimination thresholds are 50% smaller than differences observed for natural syllables. These results support that human perception is very sensitive to spectral information in dynamically changing vowels in sentences. A second line of research examined the role of vowel information in sentences using noise interruption paradigms. Based on traditional segment boundaries, vowels had a two-to-one perceptual advantage over consonants for recognizing words in sentences. Shifting boundaries to favor consonantal transitions revealed a large advantage for even small proportions of vowel information. Recent research has enhanced understanding of the interdependence between vowel dynamics and suprasegmental contours of F0 and amplitude for accurate speech recognition. Thus, dynamic information in vowels, both VISC and suprasegmental, is essential for sentence intelligibility.

Shifting Perceptual Weights in L2 Vowel Identification after Training.

Difficulties with second-language vowel perception may be related to the significant challenges in using acoustic-phonetic cues. This study investigated the effects of perception training with duration-equalized vowels on native Chinese listeners’ English vowel perception and their use of acoustic-phonetic cues. Seventeen native Chinese listeners were perceptually trained with duration-equalized English vowels, and another 17 native Chinese listeners watched English videos as a control group. Both groups were tested with English vowel identification and vowel formant discrimination before training, immediately after training, and three months later. The results showed that the training effect was greater for the vowel training group than for the control group, while both groups improved their English vowel identification and vowel formant discrimination after training. Moreover, duration-equalized vowel perception training significantly reduced listeners’ reliance on duration cues and improved their use of spectral cues in identifying English vowels, but video-watching did not help. The results suggest that duration-equalized English vowel perception training may improve non-native listeners’ English vowel perception by changing their perceptual weights of acoustic-phonetic cues.

English vowel identification and vowel formant discrimination by native Mandarin Chinese- and native English-speaking listeners: The effect of vowel duration dependence

The purpose of this study was to examine the relationship between English vowel identification and English vowel formant discrimination for native Mandarin Chinese- and native English-speaking listeners. The identification of 12 English vowels was measured with the duration cue preserved or removed. The thresholds of vowel formant discrimination on the F2 of two English vowels,/Λ/and/i/, were also estimated using an adaptive-tracking procedure. Native Mandarin Chinese-speaking listeners showed significantly higher thresholds of vowel formant discrimination and lower identification scores than native English-speaking listeners. The duration effect on English vowel identification was similar between native Mandarin Chinese- and native English-speaking listeners. Moreover, regardless of listeners' language background, vowel identification was significantly correlated with vowel formant discrimination for the listeners who were less dependent on duration cues, whereas the correlation between vowel identification and vowel formant discrimination was not significant for the listeners who were highly dependent on duration cues. This study revealed individual variability in using multiple acoustic cues to identify English vowels for both native and non-native listeners.

Vowel formant discrimination: Effects of listeners’ hearing status and language background

The goal of this study was to examine effects of listeners’ hearing status (e.g., normal and impaired hearing) and language background (e.g., native and non-native) on vowel formant discrimination. Thresholds of formant discrimination were measured for F1 and F2 of English vowels at 70 dB SPL for normal- (NH) and impaired-hearing (HI) listeners using a three-interval, two-alternative forced-choice procedure with a two-down, one-up tracking algorithm. Formant thresholds of HI listeners were comparable to those of NH listeners for F1, but significantly higher than NH listeners for F2. Results of a further experiment indicated that an amplification of the F2 peak could markedly improve formant discrimination for HI listeners, but a simple amplification of the sound level did not provide any benefit to them. On the other hand, another experiment showed that vowel density of listeners’ native language appeared to affect vowel formant discrimination, i.e., more crowded vowel space of listeners’ native language, better their vowel formant discrimination. For example, English-native listeners showed significantly lower thresholds of formant discrimination for both English and Chinese vowels than Chinese-native listeners. However, the two groups of listeners had similar psychophysical capacity to discriminate formant frequency changes in non-speech sounds.

Spectral and duration cues of English vowel identification for Chinese-native listeners

This study was to investigate how Chinese-native listeners use spectral and duration cues for English vowel identification. The first experiment was to examine whether Chinese-native listeners’ English vowel perception was related to their sensitivity to the change of vowel formant frequency that is a critical spectral cue to vowel identification. Identification of 12 isolated American English vowels was measured for 52 Chinese college students in Beijing. Thresholds of vowel formant discrimination were also examined for these students. Results showed that there was a significantly moderate correlation between Chinese college students’ English vowel identification and their thresholds of vowel formant discrimination. That is, the lower vowel formant threshold of listeners, the better vowel identification. However, the moderate correlation between vowel identification and formant discrimination suggested some other factors accounting for the individual variability in English vowel identification for Chinese-native listeners. In Experiment 2, vowel identification was measured with and without duration cues, showing that vowel identification was reduced by 5.1% when duration cue was removed. Further analysis suggested that for the listeners who depended less on duration cue, the better thresholds of formant discrimination, the higher scores of vowel identification, but no such correlation for listeners who used duration cues remarkably.

Effects of Signal Level and Spectral Contrast on Vowel Formant Discrimination for Normal-Hearing and Hearing-Impaired Listeners

The aim of this study was to determine whether increasing the overall speech level or the individual spectral contrasts of vowel sounds can improve vowel formant discrimination for listeners both with and without normal hearing. Thresholds of vowel formant discrimination were examined for the F2 frequencies of 3 American English vowels for listeners with and without normal hearing. Spectral contrasts of the F2 were enhanced by 3, 6, and 9 dB. Vowel stimuli were presented at 70 and 90 dB SPL. The thresholds of listeners with hearing impairment were reduced significantly after spectral enhancement was implemented, especially at 90 dB SPL, whereas normal-hearing listeners did not benefit from spectral enhancement. These results indicate that a combination of spectral enhancement of F2 and high speech level is most beneficial to improve vowel formant discrimination for listeners with hearing impairment.

Preliminary comparison of second-formant discrimination thresholds in cochlear implant users and young normal-hearing listeners

Formant discrimination thresholds (FDTs) may provide insights regarding factors limiting vowel perception by cochlear implant (CI) users, but have not been systematically studied in this population. In the present study, estimates of second-formant (F2) FDTs obtained in three CI users were compared to FDTs obtained from three young normal-hearing (YNH) listeners. Procedures and stimuli were modeled after Kewley-Port and Watson [J. Acoust. Soc. Am. 95, 485–496 (1994)] but employed fewer trials and an expanded F2 frequency range. Stimuli were formant-synthesized versions of three target vowels. FDTs were estimated using an adaptive 3AFC task with feedback and based on six consecutive 80-trial stimulus blocks. FDTs for the three YNH listeners were comparable to previously reported FDTs (2.4% of reference frequency versus 1.5% in Kewley-Port and Watson). FDTs for two of the CI users were about 70% larger than the average for the YNH listeners. FDTs for the third CI user approached YNH average values in one frequency region but were enlarged in another region. Data for this CI user could not be explained by place-pitch thresholds (obtained in a previous study) and suggest that CI users’ spectral acuity for complex stimuli may not be directly predictable from measures of spectral acuity for simple stimuli.

Spectral discrimination of consonants for English and Chinese Listeners

Previous work in our laboratory showed that Chinese listeners had significantly higher thresholds for vowel formant discrimination than American English listeners, possibly due to a sparse vowel system in Chinese but a crowded vowel space in American English. Considering that the two languages have similar consonant densities (i.e., similar numbers of consonants), we hypothesized that English and Chinese listeners might have similar thresholds of spectral discrimination of consonant stimuli. Thresholds of spectral shift in the English consonant, /s/, were measured in an isolated form and in a VCV context for both English and Chinese listeners. Preliminary results showed that there was no significant difference in consonant discrimination thresholds between the two groups of listeners for either isolated or VCV consonants. These findings support that phonemic density may play an important role in spectral discrimination of speech sounds.