Formant Estimation Research Articles

Acoustic vowel distances within and across dialects

Regional dialects of American English exhibit variation in the phonetic realization of vowel categories. This variation leads to different degrees of phonetic vowel similarity within and across dialects. This vowel similarity has been captured by measures of formant distance and overlap in the acoustic space, typically with static formant estimates from a single timepoint in the vowel. In contrast, dialect similarity has been assessed using measures of acoustic distance over time. In the current study, dynamic measures of acoustic distance, including dynamic time warping, root mean-square distance, and generalized additive mixed effect models, were used to assess within-dialect vowel category distances and to compare these distances across dialects. The results were generally consistent across the different distance metrics, confirming their respective ability to capture acoustic vowel distance. However, the magnitude of the distances varied across dialects and vowel variables. Whereas some dialect-specific variables showed the expected patterns of within- and across-dialect similarity based on previous descriptive work, other variables did not. These mixed findings have implications for our understanding of regional dialect variation in American English, its effect on acoustic vowel similarity, and the relationship between acoustic vowel similarity and perceptual ambiguity of vowel contrasts within and across dialects.

Refining a deep learning-based formant tracker using linear prediction methods

In this study, formant tracking is investigated by refining the formants tracked by an existing data-driven tracker, DeepFormants, using the formants estimated in a model-driven manner by linear prediction (LP)-based methods. As LP-based formant estimation methods, conventional covariance analysis (LP-COV) and the recently proposed quasi-closed phase forward–backward (QCP-FB) analysis are used. In the proposed refinement approach, the contours of the three lowest formants are first predicted by the data-driven DeepFormants tracker, and the predicted formants are replaced frame-wise with local spectral peaks shown by the model-driven LP-based methods. The refinement procedure can be plugged into the DeepFormants tracker with no need for any new data learning. Two refined DeepFormants trackers were compared with the original DeepFormants and with five known traditional trackers using the popular vocal tract resonance (VTR) corpus. The results indicated that the data-driven DeepFormants trackers outperformed the conventional trackers and that the best performance was obtained by refining the formants predicted by DeepFormants using QCP-FB analysis. In addition, by tracking formants using VTR speech that was corrupted by additive noise, the study showed that the refined DeepFormants trackers were more resilient to noise than the reference trackers. In general, these results suggest that LP-based model-driven approaches, which have traditionally been used in formant estimation, can be combined with a modern data-driven tracker easily with no further training to improve the tracker’s performance.

Formant estimation of high-pitched noisy speech using homomorphic deconvolution of higher-order group delay spectrum

Estimating the formant frequencies of high-pitched speech is essential in many speech processing applications. Unfortunately, most existing methods cannot accurately estimate the formant frequencies from high-pitched speech. Moreover, the available formant estimators do not show noise immunity. In this paper, we propose a higher-order group delay (GD) spectrum-based deconvolution method for formant estimation of high-pitched noisy speech with higher accuracy. Although cepstrum is known to provide a source-filter separation, to some extent, it gets affected by ambient noise. We employ the spectral-root-deconvolution technique on the third-order GD spectrum that yields a noise-robust cepstrum. The resulting cepstrum is found to produce significant improvement when estimating formant frequencies. We evaluated the proposed method on five synthetic vowels and some natural vowels spoken by male and female speakers by calculating the estimation error of the formant frequencies and standard F2–F1 plots, respectively. An utterance from the Texas Instruments and Massachusetts Institute of Technology (TIMIT) database has been utilized to plot the formant contours on the respective spectrogram. We compared the results with the three state-of-the-art methods. Our proposed technique outperforms all approaches, particularly with high-pitched speaking in a noisy environment.

Comparing tools for automation of formant estimation in diverse pediatric populations

Pediatric vowel productions are notoriously hard to reliably quantify. Listener judgments and transcription are subjective, show low intra- and interrater reliability, and are influenced by listener bias. Formant estimation may offer more objectivity, but child speech is fraught with acoustic challenges. Children have less refined articulatory control and they make more production errors. They have high fundamental frequencies, wide formant bandwidths, more variable formant values, and increased subglottal coupling relative to adult speech. Historically, estimation of pediatric formants has been done manually, which is laborious and time-consuming. In recent years, automation tools have been developed to speed up the process. However, these tools have not been widely tested on pediatric speech samples. Even more critically, these tools have not been tested on diverse children. This study uses speech samples from children of color, children with disabilities, and children with both of these identities to compare three automation tools: SpeechMark® (Boyce etal., 2012), Fast Track™ (Barreda, 2021), and a custom Praat® (Boersma & Weenink, 2021) script written by the first author. Outcomes of each tool will be compared and contrasted. The discussion will review the considerations, benefits, and tradeoffs of each automation tool when working with diverse pediatric speech samples.

Mechanical models as ground truth for vowel resonance analysis

It has long been established that LPC analysis results in formant estimates that are not accurate representations of the resonances; they are biased towards the nearest harmonic, and this bias worsens as F0 rises to 200 Hz or more. Manual measurement of formants with the reassigned spectrogram (RS) has been shown to be more accurate, but the ground truth is needed in order to test automatic measurement methods. Here, “vocal tract” models were 3D-printed to allow resonances to be excited replicably. Using the principle of acoustic reciprocity, source signals were played over an external loudspeaker, and the output filtered by the model was recorded via a microphone located in the model’s “glottis.” A white noise source signal was used to determine the resonances. Sawtooth and impulse train source signals (F0s 83–400 Hz) were filtered by the physical model and then analyzed with both LPC and RS. LPC formants were biased towards the nearest harmonic, resulting in errors from 10 Hz at low F0 to 250 at 300 Hz. RS errors ranged from 0 to 20 Hz and were not correlated with F0. These results indicate physical models are useful for creating known resonances for validating accuracy in our measurements.

A strategy for correcting errors in automated formant extraction

Sociophonetic vowel analysis relies heavily on measurements of resonant frequencies, particularly of the first and second formants. Automated formant estimation using linear predictive coding (LPC) algorithms in software like Praat greatly increases efficiency compared to hand measurements and allows researchers to analyze more data than was possible before this technological advancement. However, LPC analysis is prone to certain types of errors (e.g., Di Paolo et al., 2011; Harrison, 2013; Labov et al., 2006; Strelluf, 2019; Styler, 2017). In one common error, which I call “faulty low F2” (FLF2), LPC identifies a spectral peak between F1 and F2 as an F2 measurement. In automated extraction, the real F2 is then recorded as F3. Manually correcting these errors is time-consuming, potentially erasing the gains made by automated extraction. However, a systematic error suggests the possibility of a systematic solution. This presentation describes a method for identifying and correcting FLF2 errors using a script in R. The script identifies possible errors based on expected ranges and trajectories, determines whether the recorded F3 at the measurement point meets the criteria for a legitimate F2, and makes substitutions where appropriate. Results of tests with real data are presented. [Work supported by NSF #XXXXXXX.]

Formant estimation from speech signal using the magnitude spectrum modified with group delay spectrum

Formant estimation from speech signal using the magnitude spectrum modified with group delay spectrum

Acoustic analysis of vowel formant frequencies in genetically-related and non-genetically related speakers with implications for forensic speaker comparison.

The purpose of this study was to explore the speaker-discriminatory potential of vowel formant mean frequencies in comparisons of identical twin pairs and non-genetically related speakers. The influences of lexical stress and the vowels’ acoustic distances on the discriminatory patterns of formant frequencies were also assessed. Acoustic extraction and analysis of the first four speech formants F1-F4 were carried out using spontaneous speech materials. The recordings comprise telephone conversations between identical twin pairs while being directly recorded through high-quality microphones. The subjects were 20 male adult speakers of Brazilian Portuguese (BP), aged between 19 and 35. As for comparisons, stressed and unstressed oral vowels of BP were segmented and transcribed manually in the Praat software. F1-F4 formant estimates were automatically extracted from the middle points of each labeled vowel. Formant values were represented in both Hertz and Bark. Comparisons within identical twin pairs using the Bark scale were performed to verify whether the measured differences would be potentially significant when following a psychoacoustic criterion. The results revealed consistent patterns regarding the comparison of low-frequency and high-frequency formants in twin pairs and non-genetically related speakers, with high-frequency formants displaying a greater speaker-discriminatory power compared to low-frequency formants. Among all formants, F4 seemed to display the highest discriminatory potential within identical twin pairs, followed by F3. As for non-genetically related speakers, both F3 and F4 displayed a similar high discriminatory potential. Regarding vowel quality, the central vowel /a/ was found to be the most speaker-discriminatory segment, followed by front vowels. Moreover, stressed vowels displayed a higher inter-speaker discrimination than unstressed vowels in both groups; however, the combination of stressed and unstressed vowels was found even more explanatory in terms of the observed differences. Although identical twins displayed a higher phonetic similarity, they were not found phonetically identical.

Mel Scale-Based Linear Prediction Approach to Reduce the Prediction Filter Order in CELP Paradigm

This paper proposes a novel method to reduce the order of prediction filter from 10 to 7 in Code Excited Linear Prediction (CELP) coding framework by the inclusion of psychoacoustic Mel scale into Linear Predictive Coding (Mel-LPC). Efficient quantization methods using 2-split Vector Quantization (VQ) for Mel-LPC obtained a reduction of 4 bits/frame and resulted in a total bit gain of 200 bps. A weighting scheme for the Euclidean distance measure gave a reduction of 6 bits/frame that adds up to a total bit gain of 300 bps. A lower Mel-LPC order of 3 has been employed for unvoiced frames by using the perceptual quality as selection criteria and an efficient VQ method using 5 bits is developed which brought down the average bit requirement to 11.5 bits/frame. To incorporate this into Mel-LPC-based CELP encoding scheme, a neural network-based voiced-unvoiced classification algorithm using 5 derived features as input has been constructed and this selection of filter order based on signal statistics provides the benefit of bit reduction by 625 and 325 bps, respectively, for 10th order LPC and 7th order Mel-LPC. In addition to all, the incorporation of Mel-LPC gives a better performance in the estimation of formants.

Fast Track: fast (nearly) automatic formant-tracking using Praat

Abstract Fast Track is a formant tracker implemented in Praat that attempts to automatically select the best analysis from a set of candidates. The best track is selected by modeling smooth formant contours across the entirety of the sound, providing the researcher with rich information about static and dynamic formant properties. Fast Track returns text files containing acoustic information (formant frequencies, formant bandwidths, fundamental frequency, etc.) sampled every 2 ms, generates images showing the winning analysis and comparing alternate analyses, and creates log files detailing analysis information for each file. Fast Track features a modular workflow that allows for analysis steps to be run (and re-run) independently as necessary, and is designed to allow for easy correction of tracking errors by allowing the user to override the automatic analysis, or manually edit tracks where necessary. In addition, Fast Track includes tools to aggregate data across tokens, and to easily create vowel plots of mean values or time-varying formant contours. The design and use of Fast Track are outlined using a re-analysis of the Hillenbrand et al. (1995) dataset, which suggests that Fast Track can be very accurate in cases where signal properties allow for reliable formant estimates.

Power-Weighted LPC Formant Estimation

A power-weighted formant frequency estimation procedure based on Linear Predictive Coding (LPC) is presented. It works by pre-emphasizing the dominant spectral components of an input signal, which allows a subsequent estimation step to extract formant frequencies with greater accuracy. The accuracy of traditional LPC formant estimation is improved by this new power-weighted formant estimator for different classes of synthetic signals and for speech. Power-weighted LPC significantly and reliably outperforms LPC and variants of LPC at the task of formant estimation using the VTR formants dataset, a database consisting of the Vocal Tract Resonance (VTR) frequency trajectories obtained by human experts for the first three formant frequencies. This performance gain is evident over a range of filter orders.

Formant Tracking Using Quasi-Closed Phase Forward-Backward Linear Prediction Analysis and Deep Neural Networks

Formant tracking is investigated in this study by using trackers based on dynamic programming (DP) and deep neural nets (DNNs). Using the DP approach, six formant estimation methods were first compared. The six methods include linear prediction (LP) algorithms, weighted LP algorithms and the recently developed quasi-closed phase forward-backward (QCP-FB) method. QCP-FB gave the best performance in the comparison. Therefore, a novel formant tracking approach, which combines benefits of deep learning and signal processing based on QCP-FB, was proposed. In this approach, the formants predicted by a DNN-based tracker from a speech frame are refined using the peaks of the all-pole spectrum computed by QCP-FB from the same frame. Results show that the proposed DNN-based tracker performed better both in detection rate and estimation error for the lowest three formants compared to reference formant trackers. Compared to the popular Wavesurfer, for example, the proposed tracker gave a reduction of 29%, 48% and 35% in the estimation error for the lowest three formants, respectively.

Vocal Tract Length Estimation Using Accumulated Means of Formants and Its Effects on Speaker-Normalization

Differences in vocal tract lengths (VTLs) in individual speakers cause variations in acoustic features of phonemes. In this paper, a simple method to estimate speaker-specific VTLs and to quantitatively evaluate some speaker-normalization effects of the VTLs is proposed. We employed accumulated means of formant trajectories to estimate the VTLs of speakers ranging from children to adults. For the formant estimation, the inverse-filter control (IFC) system was used. In the system, the decision of analysis order, which means number of formants to be estimated, is automated. Moreover, to evaluate the speaker-normalization effect of VTLs, we proposed the data reduction method, which can reasonably find dense areas of ellipses from distributions in the formant space. Using these ellipse areas, we evaluated the three normalization effects of VTLs: normalization by the mean of all VTLs as the standard, by speaker-categorical means of VTLs, and by individual VTLs. The area reduced from the standard area of the original data by 39.5% and 46.6% in the case of the categorical means and individual VTLs, respectively. As a result, our proposed method was used to provide a “normalized vowel map (NVM)” that visualizes universal vowel-distributions as a core image of linguistic information. Finally, we compared the estimated VTLs with those by another method based on magnetic resonance imaging (MRI) data, using the proposed methods.

The Dynamics of Vowel Hypo- and Hyperarticulation in Swedish Infant-Directed Speech to 12-Month-Olds

Vowel hypo- and hyperarticulation (VHH) was investigated in Swedish infant-directed speech (IDS) to Swedish 12-month-olds using a measure that normalizes across speakers and vowels: the vhh-index. The vhh-index gives the degree of VHH for each individual vowel token, which allows for analysis of the dynamics of VHH within a conversation. Using both the vhh-index and traditional measures of VHH, the degree of VHH was compared between Swedish IDS and ADS. The vowel space area was larger in IDS than in ADS, and the average vhh-index as well as the modal value was higher in IDS than in ADS. Further, the proportion of vowel tokens that were highly hyperarticulated (vhh-index > 75th percentile) were fewer in ADS than in IDS. Vowels in Swedish IDS to 12-month-olds are thus concluded to be hyperarticulated compared to vowels in Swedish ADS, both in terms of degree and frequency. Findings are in line with previous reports on VHH in Swedish IDS as well as on VHH in IDS to infants around 12 months in other languages. The study considers the importance of robust formant estimation, highlights the need for replication of studies on VHH in IDS on previously studied languages and ages, and discusses the benefits of the vhh-index. Those benefits include that it normalizes across speakers and vowels, can be used for dynamic measures within speech samples, and permits analyses on token-level.

Classification of formant estimation methods in transformed auditory feedback experiments using convolutional neural networks

Classification of formant estimation methods in transformed auditory feedback experiments using convolutional neural networks

Effects of sampling rate and type of anti-aliasing filter on linear-predictive estimates of formant frequencies in men, women, and children

The purpose of this study was to assess the effect of downsampling the acoustic signal on the accuracy of linear-predictive (LPC) formant estimation. Based on speech produced by men, women, and children, the first four formant frequencies were estimated at sampling rates of 48, 16, and 10 kHz using different anti-alias filtering. With proper selection of number of LPC coefficients, anti-alias filter and between-frame averaging, results suggest that accuracy is not improved by rates substantially below 48 kHz. Any downsampling should not go below 16 kHz with a filter cut-off centered at 8 kHz.

Exploring vowel formant estimation through simulation-based techniques

AbstractThis paper contributes insight into the sources of variability in vowel formant estimation, a major analytic activity in sociophonetics, by reviewing the outcomes of two simulations that manipulated the settings used for linear predictive coding (LPC)-based vowel formant estimation. Simulation 1 explores the range of frequency differences obtained when minor adjustments are made to LPC settings, and measurement timepoints around the settings used by trained analysts, in order to determine the range of variability that should be expected in sociophonetic vowel studies. Simulation 2 examines the variability that emerges when LPC settings are varied combinatorially around constant default settings, rather than settings set by trained analysts. The impacts of different LPC settings are discussed as a way of demonstrating the inherent properties of LPC-based formant estimation. This work suggests that differences more fine-grained than about 10 Hz in F1 and 15–20 Hz in F2 are within the range of LPC-based formant estimation variability.

Time-varying quasi-closed-phase analysis for accurate formant tracking in speech signals

In this paper, we propose a new method for the accurate estimation and tracking of formants in speech signals using time-varying quasi-closed-phase (TVQCP) analysis. Conventional formant tracking methods typically adopt a two-stage estimate-and-track strategy wherein an initial set of formant candidates are estimated using short-time analysis (e.g., 10–50 ms), followed by a tracking stage based on dynamic programming or a linear state-space model. One of the main disadvantages of these approaches is that the tracking stage, however good it may be, cannot improve upon the formant estimation accuracy of the first stage. The proposed TVQCP method provides a single-stage formant tracking that combines the estimation and tracking stages into one. TVQCP analysis combines three approaches to improve formant estimation and tracking: (1) it uses temporally weighted quasi-closed-phase analysis to derive closed-phase estimates of the vocal tract with reduced interference from the excitation source, (2) it increases the residual sparsity by using the $L_1$ optimization and (3) it uses time-varying linear prediction analysis over long time windows (e.g., 100–200 ms) to impose a continuity constraint on the vocal tract model and hence on the formant trajectories. Formant tracking experiments with a wide variety of synthetic and natural speech signals show that the proposed TVQCP method performs better than conventional and popular formant tracking tools, such as Wavesurfer and Praat (based on dynamic programming), the KARMA algorithm (based on Kalman filtering), and DeepFormants (based on deep neural networks trained in a supervised manner). Matlab scripts for the proposed method can be found at: https://github.com/njaygowda/ftrack

Design of MELPe-Based Variable-Bit-Rate Speech Coding with Mel Scale Approach Using Low-Order Linear Prediction Filter and Representing Excitation Signal Using Glottal Closure Instants

In this paper, we propose a variable-bit-rate speech codec-based on mixed excitation linear prediction enhanced (MELPe) with an average bit rate of 2 kbps and with a better representation of excitation signal. The order of the prediction filter in MELPe coding architecture is reduced from 10 to 7 without affecting the perceptual quality of the decoded speech by using psychoacoustic Mel scale. An efficient two-split vector quantization is developed with weighted Euclidean distance measure for Mel scale-based linear predictive coding (Mel-LPC), and it requires only 18 bits/frame. The instantaneous pitch or epoch that is vital for many speech processing applications is preserved in this codec by including it in the excitation signal used for reconstructing the voiced speech. The quantization scheme developed for glottal closure instants (GCIs) causes an increase in the bit requirement for voiced frames by 4–25 bits depending on the position of GCIs. To compensate for that, the Mel-LPC order for both silence and unvoiced frames has been brought down to 4 without compromising the perceptual quality of reconstructed speech. The lowered bit budget for unvoiced frame is 41 bits/frame, and for silence, it is 31 bits/frame. Further reduction of 10 bits for silence frame is obtained by reducing the number of transmitted parameters and by tuning the quantization bit requirement for each. For categorizing the speech frames at the entry of the encoder, a neural network-based voiced/unvoiced/silence classification algorithm using five-dimensional feature set is created. The experimental results show that the proposed coding scheme operates at an average bit rate of 2 kbps, which is less than the bit rate of MELPe (2.4 kbps), but with a better perceptual score. In addition to all these, the incorporation of Mel-LPC gives a better performance in the estimation of formants and GCIs.

Large-scale acoustic characterization of mid-low vowels across American, British, and Singaporean children

This study compares how American, British and Singaporean children differ in their production of mid-low vowels. Read speech was collected from 6–13 years old American children (140 speakers, 43 406 utterances), British children (82 speakers, 32 542 utterances) and Singaporean children (192 speakers, 34 457 utterances), with a balanced gender ratio. All three corpora were designed to be phonetically balanced and formant estimates were extracted from the vowel tokens using the Praat software. Our large-scale acoustic analyses show that for TRAP-BATH split vowels, (1) British and Singaporean children both produce these vowels with higher F1, suggesting a relatively lowered tongue height; (2) These vowels have higher F2 (suggesting a more fronted tongue position) for both American and Singaporean children. One-way ANOVA tests reached statistical significance for all these differences; F(2,411) = 34.59, p < 0.001 for F1; F(2,411) = 544.13, p < 0.001 for F2. When comparing /æ/ and /ɛ/ productions, British speakers demonstrate the clearest distinction between the two vowels; Singaporean and American speakers both exhibit a higher and more fronted tongue position for /æ/, causing /æ/ to be more acoustically similar to /ɛ/. One-way ANOVA tests and post hoc Tukey’s test demonstrated that these differences are all statistically significant too (p < 0.001).