Formant Location Research Articles

Artificial Bandwidth Extension for Lower Bandwidth Using Sinusoidal Synthesis based on First Formant Location

Artificial Bandwidth Extension for Lower Bandwidth Using Sinusoidal Synthesis based on First Formant Location

The effect of different acoustic noise on speech signal formant frequency location

The presence of noise is one of the major challenges and concerns in speech recognition systems. There are in particular different kinds of noises (pink, white and leopard) that can adversely affect a speech signal in various ways and degrees. In this study, the extent of resistance of a speech signal’s formants or in other words, the displacement of the formants have been measured against being subjected to different conventional noises. The methodology adopted was to apply different noises to the original voice signal, then to measure and to investigate the amount of formant location displacement. In this paper, the mean square movement (MSM) parameter has been introduced. This represents the deviation and displacement amount of the frequencies of the formants caused by applying the various noises. All of the investigations were conducted under three different SNR conditions (5, 10 and 15 dB). This allowed for the assessment of the influence of the signal-to-noise ratio (SNR) on the MSM parameter and the extent of the displacements of the formants. The results indicate that the frequency of the formants under these three SNR amounts was resistant against the machine gun type of noise, whilst white noise caused the most measureable effect and displacement in the frequencies of the formants.

Deriving individualised /r/ targets from the acoustics of children’s non-rhotic vowels

ABSTRACTIn visual-acoustic biofeedback for rhotic errors, learners are guided to match the third formant (F3) location to a visual target on a real-time acoustic spectrum. As the acoustic properties of correct English /r/differ across speakers, this study aimed to improve target selection by investigating the validity of individualised targets derived from children’s non-rhotic vowels. A previously proposed prediction formula was adjusted using data from a child normative sample and tested in two groups of children. Study 1 found that predicted values were unexpectedly higher than actual F3 values in children whose /r/ errors had been remediated. To understand this discrepancy, Study 2 applied the formula to typically developing children and found that predicted values were also higher than actual F3 values, suggesting that different normative data might better represent the current samples. An updated formula is proposed, which can be used to generate individualised targets within acoustic biofeedback applications.

Interpretable parametric voice conversion functions based on Gaussian mixture models and constrained transformations

Voice conversion functions based on Gaussian mixture models and parametric speech signal representations are opaque in the sense that it is not straightforward to interpret the physical meaning of the conversion parameters. Following the line of recent works based on the frequency warping plus amplitude scaling paradigm, in this article we show that voice conversion functions can be designed according to physically meaningful constraints in such manner that they become highly informative. The resulting voice conversion method can be used to visualize the differences between source and target voices or styles in terms of formant location in frequency, spectral tilt and amplitude in a number of spectral bands.

An ANN based approach to recognize initial phonemes of spoken words of Assamese language

Initial phoneme is used in spoken word recognition models. These are used to activate words starting with that phoneme in spoken word recognition models. Such investigations are critical for classification of initial phoneme into a phonetic group. A work is described in this paper using an artificial neural network (ANN) based approach to recognize initial consonant phonemes of Assamese words. A self organizing map (SOM) based algorithm is developed to segment the initial phonemes from its word counterpart. Using a combination of three types of ANN structures, namely recurrent neural network (RNN), SOM and probabilistic neural network (PNN), the proposed algorithm proves its superiority over the conventional discrete wavelet transform (DWT) based phoneme segmentation. The algorithm is exclusively designed on the basis of Assamese phonemical structure which consists of certain unique features and are grouped into six distinct phoneme families. Before applying the segmentation approach using SOM, an RNN is used to take some localized decision to classify the words into six phoneme families. Next the SOM segmented phonemes are classified into individual phonemes. A two-class PNN classification is performed with clean Assamese phonemes, to recognize the segmented phonemes. The validation of recognized phonemes is checked by matching the first formant frequency of the phoneme. Formant frequency of Assamese phonemes, estimated using the pole or formant location determination from the linear prediction model of vocal tract, is used effectively as a priori knowledge in the proposed algorithm.

Vocal Tract Morphology Determines Species-Specific Features in Vocal Signals of Lemurs (Eulemur)

The source-filter theory describes vocal production as a two-stage process involving the generation of a sound source, with its own spectral structure, which is then filtered by the resonant properties of the vocal tract. This theory has been successfully applied to the study of animal vocal signals since the 1990s. As an extension, models reproducing vocal tract resonance can be used to reproduce formant patterns and to understand the role of vocal tract filtering in nonhuman vocalizations. We studied three congeneric lemur species —Eulemur fulvus, E. macaco, E. rubriventer— using morphological measurements to build computational models of the vocal tract to estimate formants, and acoustic analysis to measure formants from natural calls. We focused on call types emitted through the nose, without apparent articulation. On the basis of anatomical measurements, we modeled the vocal tract of each species as a series of concatenated tubes, with a cross-sectional area that changed along the tract to approximate the morphology of the larynx, the nasopharyngeal cavity, the nasal chambers, and the nostrils. For each species, we calculated the resonance frequencies in 2500 randomly generated vocal tracts, in which we simulated intraspecific length and size variation. Formant location and spacing showed significant species-specific differences determined by the length of the vocal tract. We then measured formants of a set of nasal vocalizations (“grunts”) recorded from captive lemurs of the same species. We found species-specific differences in the natural calls. This is the first evidence that morphology of the vocal tract is relevant in generating filter-related acoustic cues that potentially provide receivers with information about the species of the emitter.

Segmentation and Classification of Vowel Phonemes of Assamese Speech Using a Hybrid Neural Framework

In spoken word recognition, one of the crucial points is to identify the vowel phonemes. This paper describes an Artificial Neural Network (ANN) based algorithm developed for the segmentation and recognition of the vowel phonemes of Assamese language from some words containing those vowels. Self-Organizing Map (SOM) trained with a various number of iterations is used to segment the word into its constituent phonemes. Later, Probabilistic Neural Network (PNN) trained with clean vowel phonemes is used to recognize the vowel segment from the six different SOM segmented phonemes. One of the important aspects of the proposed algorithm is that it proves the validation of the recognized vowel by checking its first formant frequency. The first formant frequency of all the Assamese vowels is predetermined by estimating pole or formant location from the linear prediction (LP) model of the vocal tract. The proposed algorithm shows a high recognition performance in comparison to the conventional Discrete Wavelet Transform (DWT) based segmentation.

Formant position based weighted spectral features for emotion recognition

In this paper, we propose novel spectrally weighted mel-frequency cepstral coefficient (WMFCC) features for emotion recognition from speech. The idea is based on the fact that formant locations carry emotion-related information, and therefore critical spectral bands around formant locations can be emphasized during the calculation of MFCC features. The spectral weighting is derived from the normalized inverse harmonic mean function of the line spectral frequency (LSF) features, which are known to be localized around formant frequencies. The above approach can be considered as an early data fusion of spectral content and formant location information. We also investigate methods for late decision fusion of unimodal classifiers. We evaluate the proposed WMFCC features together with the standard spectral and prosody features using HMM based classifiers on the spontaneous FAU Aibo emotional speech corpus. The results show that unimodal classifiers with the WMFCC features perform significantly better than the classifiers with standard spectral features. Late decision fusion of classifiers provide further significant performance improvements.

Constraints on the processes responsible for the extrinsic normalization of vowels.

Listeners tune in to talkers’ vowels through extrinsic normalization. We asked here whether this process could be based on compensation for the long-term average spectrum (LTAS) of preceding sounds and whether the mechanisms responsible for normalization are indifferent to the nature of those sounds. If so, normalization should apply to nonspeech stimuli. Previous findings were replicated with first-formant (F1) manipulations of speech. Targets on a [pt]–[pɛt] (low–high F1) continuum were labeled as [pt] more after high-F1 than after low-F1 precursors. Spectrally rotated nonspeech versions of these materials produced similar normalization. None occurred, however, with nonspeech stimuli that were less speechlike, even though precursor–target LTAS relations were equivalent to those used earlier. Additional experiments investigated the roles of pitch movement, amplitude variation, formant location, and the stimuli's perceived similarity to speech. It appears that normalization is not restricted to speech but that the nature of the preceding sounds does matter. Extrinsic normalization of vowels is due, at least in part, to an auditory process that may require familiarity with the spectrotemporal characteristics of speech.

Bee Threat Elicits Alarm Call in African Elephants

Unlike the smaller and more vulnerable mammals, African elephants have relatively few predators that threaten their survival. The sound of disturbed African honeybees Apis meliffera scutellata causes African elephants Loxodonta africana to retreat and produce warning vocalizations that lead other elephants to join the flight. In our first experiment, audio playbacks of bee sounds induced elephants to retreat and elicited more head-shaking and dusting, reactive behaviors that may prevent bee stings, compared to white noise control playbacks. Most importantly, elephants produced distinctive “rumble” vocalizations in response to bee sounds. These rumbles exhibited an upward shift in the second formant location, which implies active vocal tract modulation, compared to rumbles made in response to white noise playbacks. In a second experiment, audio playbacks of these rumbles produced in response to bees elicited increased headshaking, and further and faster retreat behavior in other elephants, compared to control rumble playbacks with lower second formant frequencies. These responses to the bee rumble stimuli occurred in the absence of any bees or bee sounds. This suggests that these elephant rumbles may function as referential signals, in which a formant frequency shift alerts nearby elephants about an external threat, in this case, the threat of bees.

Non-native speech rhythm: A large-scale study of English pronunciation by Korean learners

Kim, Jong-mi, Suzanne Flynn, and Mira Oh. 2007. Non-native speech rhythm: A large-scale study of English pronunciation by Korean learners. Studies in Phonetics, Phonology and Morphology 13.2. 219-250. To what degree do the rhythmical properties of the native language emerge in the acquisition of a second language? Specifically, would non-native English—a stress-timed language— evidence effects of Korean—a so-called syllable-timed language? In this study, we analyze the speech data from 111 adult native speakers of Korean at an intermediate level learning English as a second language, in comparison with those from 29 adult native speakers of English. The study involves a preand posttest analysis of speech elicited in a reading task. We focus on the acoustic quantification of the vowel duration in relation to F0 trace and formant location in all elicited speech that involve 1) stress alternation (‘John was ‘sick of ‘Fred and ‘Sandy), 2) stress assignment (a ‘blackboard vs. a black ‘board), and 3) stress reduction (add vs. addition). The results indicate that the speech rhythm contrast is manifested in both stress alternation and stress reduction in terms of the pair-wise variability of the vowel duration between the two groups of native and learner speech of English. These results are further supported by independent evidence from the developmental aspect in English pronunciation and perception. (Kangwon National University, Massachusetts Institute of Technology, and Chonnam National University)

Analysis and compensation of speech under stress and noise for environmental robustness in speech recognition

It is well known that the introduction of acoustic background distortion and the variability resulting from environmentally induced stress causes speech recognition algorithms to fail. In this paper, several causes for recognition performance degradation are explored. It is suggested that recent studies based on a Source Generator Framework can provide a viable foundation in which to establish robust speech recognition techniques. This research encompasses three inter-related issues: (i) analysis and modeling of speech characteristics brought on by workload task stress, speaker emotion/stress or speech produced in noise (Lombard effect), (ii) adaptive signal processing methods tailored to speech enhancement and stress equalization, and (iii) formulation of new recognition algorithms which are robust in adverse environments. An overview of a statistical analysis of a Speech Under Simulated and Actual Stress (SUSAS) database is presented. This study was conducted on over 200 parameters in the domains of pitch, duration, intensity, glottal source and vocal tract spectral variations. These studies motivate the development of a speech modeling approach entitled Source Generator Framework in which to represent the dynamics of speech under stress. This framework provides an attractive means for performing feature equalization of speech under stress. In the second half of this paper, three novel approaches for signal enhancement and stress equalization are considered to address the issue of recognition under noisy stressful conditions. The first method employs (Auto:I,LSP:T) constrained iterative speech enhancement to address background noise and maximum likelihood stress equalization across formant location and bandwidth. The second method uses a feature enhancing artificial neural network which transforms the input stressed speech feature set during parameterization for keyword recognition. The final method employs morphological constrained feature enhancement to address noise and an adaptive Mel-cepstral compensation algorithm to equalize the impact of stress. Recognition performance is demonstrated for speech under a range of stress conditions, signal-to-noise ratios and background noise types.

Generating stressed speech from neutral speech using a modified CELP vocoder

The problem of speech modeling for generating stressed speech using a source generator framework is addressed in this paper. In general, stress in this context refers to emotional or task induced speaking conditions. Throughout this particular study, the focus will be limited to speech under angry, loud and Lombard effect (i.e., speech produced in noise) speaking conditions. Source generator theory was originally developed for equalization of speech under stress for robust recognition (Hansen, 1993, 1994). It was later used for simulated stressed training token generation for improved recognition (Bou-Ghazale, 1993; Bou-Ghazale and Hansen, 1994). The objective here is to generate stressed perturbed speech from neutral speech using a source generator framework previously employed for stressed speech recognition. The approach is based on (i) developing a mathematical model that provides a means for representing the change in speech production under stressed conditions for perturbation, and (ii) employing this framework in an isolated word speech processing system to produce emotional/stressed perturbed speech from neutral speech. A stress perturbation algorithm is formulated based on a CELP (code-excited linear prediction) speech synthesis structure. The algorithm is evaluated using four different speech feature perturbation sets. The stressed speech parameter evaluations from this study revealed that pitch is capable of reflecting the emotional state of the speaker, while formant information alone is not as good a correlate of stress. However, the combination of formant location, pitch and gain information proved to be the most reliable indicator of emotional stress under a CELP speech model. Results from formal listener evaluations of the generated stressed speech show successful classification rates of 87% for angry speech, 75% for Lombard effect speech and 92% for loud speech.

Many-valued logic is reflected in formant location and spectral composition

Variation in the truth value of propositions has been shown to yield changes in cognitive event-related potentials (ERPs) [D. A. Pollina and N. K. Squires, Psychophys. 30, S52 (1993)]. Given the connection between the arousal of the central nervous system and peripheral systems, the same paradigm that was used to detect many-valued beliefs via ERPs was used to detect degree of belief by an acoustical analysis of the voice. Degree of belief was manipulated by having subjects solve a murder mystery, after which they learned facts associated with three people: (1) the person they most suspected of committing the murder, (2) the person they judged next most likely to be the murderer, and (3) the person they knew could not be the murderer. Articulatory precision of subjects’ utterances was measured by formant location and spectral composition [F. J. Tolkmitt and K. R. Scherer, J. Exp. Psychol. Human Percept. Perform. 12, 302–313 (1986)]. These measures were related to the degree to which subjects believed each statement to be true.

Formant location from LPC analysis data

The estimation of formant frequencies and bandwidths from the filter coefficients obtained through linear-predictive-coding (LPC) analysis of speech is discussed from several viewpoints. A method for locating roots within the unit circle is derived. This algorithm is particularly well suited to computations carried out in fixed-point arithmetic using specialized signal processing hardware. >

Effect of experimentally induced stress on vocal parameters.

In a factorially designed experiment the factors Mode of Stress (cognitive vs. emotional) and Degree of Stress (low vs. high) were studied in their effect on phonatory and articulatory processes during speech production in groups of male and female students (factor Sex) selected to represent three types of personality (factor Coping Style: low anxiety, high anxiety, anxiety denying). Using digital speech analysis procedures, mean F0, F0 floor, formant location, and spectral energy distribution were extracted from the experimental speech samples. Although mean F0 did not covary with stress manipulation, F0 floor of high-anxious and anxiety-denying subjects increased with stress, probably due to physiologically based changes in muscle tension. For articulatory processes, as measured through formant location and spectral composition, significant changes were found for females. For anxiety-denying female subjects, precision of articulation increased under cognitive stress and decreased under emotional stress. The results are interpreted in terms of differential susceptibility to stress.

Nonlinear auditory coding of vowel formants at high sound pressure levels

To study vowel intelligibility in the absence of the acoustic reflex, ten synthetic vowels whose duration was shorter than the effective latency of the acoustic reflex were presented to normal hearing listeners at levels ranging from 72 to 106 dB SPL. Signal intelligibility varied as a function of both SPL and vowel identity. Vowels with widely spaced formants, i.e., /i/ and /ɪ/, were unaffected by presentation level. Vowels with more proximal formants began to show a decrement in intelligibility at 96 dB and suffered as much as a 40% decrement at 106 dB. Confusion errors were not symmetrical in F1‐F2 space suggesting a systematic distortion in the auditory coding of formant location. These data indicate another source of distortion for many severely hearing‐impaired listeners who commonly listen to speech amplified to greater than 100 dB and who do not have measureable acoustic reflexes. [Research supported by NIH.]

Zero sensitivity properties of the digital lattice filter

The first-order movement of zeros given small perturbations of the PARCOR coefficients in the all-zero digital lattice filter is determined analytically. This is an extension of previous work characterizing sensitivity by integral spectral deviation measures, and is of interest since much is known about the perceptual effects of deviations in formant frequency location. The principal insight is the role that the frequency of the zero plays in its sensitivity and a better understanding of the quite different sensitivity properties of the first and last PARCOR coefficients. Comparison is made to the transversal filter structure, and further interpretation of the effect of preemphasis is given. By examining individual terms of the zero sensitivity equations, an understanding may be gained of the factors responsible for large zero movements and the methods which may be used to decrease these movements. As an example, a computationally simple bilinear type of transformation of the original PARCOR's to a new set of PARCOR's is proposed, which decreases the zero sensitivity at low frequency and increases it at high frequency. The perceptual advantage is estimated by listening tests and spectral deviation plots to be comparable to that of preemphasis, about 2 bits/frame.

The effect of stress and task variation on formant location

The effect of psychological stress and of different task demands on the frequency, amplitude, and band width of formants in words spoken as part of a word recognition task was studied. Subjects were 39 medical assistants who took part in an experiment on the effect of arousal on speech performance. Subjects had to engage in two types of cognitive tasks with different demands on concentration and mathematical ability and had to describe two types of films, one film on pottery making with low stress inducing potential and one film on a surgical operation. Immediately after each of these tasks, subjects had to read out loud words which were presented at relatively short exposure times (to simulate a word recognition task). Vowels within these words were isolated and formant frequencies, amplitude and band width were measured using linear prediction formant estimation routines. The results are interpreted in terms of the inter‐speaker and intra‐speaker variability of formant data across different levels of emotional arousal and task demands.

Automatic Reduction of Vowel Spectra: A Quantitative Analysis of Error Scores

Analysis-by-synthesis procedures for automatic reduction of speech spectra describe the goodness of fit between an input-speech spectrum and an internally generated spectrum by an error score. In vowel analysis, the error score typically is derived from a computed-error curve specified by values corresponding to 24 input filters. Values of error scores from a set of utterances by three talkers are presented and compared to similar scores resulting from manual spectrum-matching procedures. Variations in error score attributable to particular vowels, contexts, and talkers are investigated, and the error-score magnitude is related to accuracy in formant location. [This work was supported in part by the U. S. Army, the U. S. Air Force Office of Scientific Research, and the U. S. Office of Naval Research; in part by the U. S. Air Force (Electronic Systems Division) under Contract AF 19(604)-6102; in part by the National Science Foundation (Grant G-16526); and in part by the National Institutes of Health (Grant MH-04737-02).]