Vocal Tract Characteristics Research Articles

Voice conversion based on Gaussian processes by coherent and asymmetric training with limited training data

Voice conversion (VC) is a technique aiming to mapping the individuality of a source speaker to that of a target speaker, wherein Gaussian mixture model (GMM) based methods are evidently prevalent. Despite their wide use, two major problems remains to be resolved, i.e., over-smoothing and over-fitting. The latter one arises naturally when the structure of model is too complicated given limited amount of training data.Recently, a new voice conversion method based on Gaussian processes (GPs) was proposed, whose nonparametric nature ensures that the over-fitting problem can be alleviated significantly. Meanwhile, it is flexible to perform non-linear mapping under the framework of GPs by introducing sophisticated kernel functions. Thus this kind of method deserves to be explored thoroughly in this paper. To further improve the performance of the GP-based method, a strategy for mapping prosodic and spectral features coherently is adopted, making the best use of the intercorrelations embedded among both excitation and vocal tract features. Moreover, the accuracy in computing the kernel functions of GP can be improved by resorting to an asymmetric training strategy that allows the dimensionality of input vectors being reasonably higher than that of the output vectors without additional computational costs. Experiments have been conducted to confirm the effectiveness of the proposed method both objectively and subjectively, which have demonstrated that improvements can be obtained by GP-based method compared to the traditional GMM-based approach.

Individual identification of Japanese macaques by coo-calls: Pitch or vocal tract characteristics?

Japanese macaques, Macaca fuscata, utter the Smooth Early High (SEH), one of harmonically structured coo-calls, for greeting and locating other individuals. The purpose of this study was to examine acoustical features of SEH for them to identify the caller. Two male Japanese macaques were trained to discriminate SEHs of Macaque A (SEHa) from those of B (SEHb). The fundamental frequencies (pitch) of SEHa and SEHb were different. Those calls were recorded from monkeys unfamiliar to subjects. GO-NOGO paradigm was used for training the animals. SEHa served GO stimuli (S + ) while SEHb served NOGO stimuli (S-). A “Hit” was reinforced with about 1 ml of orange juice. Unfamiliar SEHa and SEHb, which were never used for training sessions, were used as test stimuli. A series of morphed SEHs between SEHa (S + ) and SEHb (S-) with only pitch or vocal tract characteristics varied were also used for test stimuli. Results showed that reaction times (RTs) for unfamiliar SEHa were significantly shorter than those for unfamiliar SEHb. RTs to morphed stimuli shortened as the distance from SEHb (S-) increased in both pitch and vocal tract dimensions. Data suggested that both pitch and vocal-tract characteristics were important for Japanese macaques to identify individuality.

A comparative study of explicit and implicit modelling of subsegmental speaker-specific excitation source information

In this paper, the explicit and implicit modelling of the subsegmental excitation information are experimentally compared. For explicit modelling, the static and dynamic values of the standard Liljencrants–Fant (LF) parameters that model the glottal flow derivative (GFD) are used. A simplified approximation method is proposed to compute these LF parameters by locating the glottal closing and opening instants. The proposed approach significantly reduces the computation needed to implement the LF model. For implicit modelling, linear prediction (LP) residual samples considered in blocks of 5 ms with shift of 2.5 ms are used. Different speaker recognition studies are performed using NIST-99 and NIST-03 databases. In case of speaker identification, the implicit modelling provides significantly better performance compared to explicit modelling. Alternatively, the explicit modelling seem to be providing better performance in case of speaker verification. This indicates that explicit modelling seem to have relatively less intra and inter-speaker variability. The implicit modelling on the other hand, has more intra and inter-speaker variability. What is desirable is less intra and more inter-speaker variability. Therefore, for speaker verification task explicit modelling may be used and for speaker identification task implicit modelling may be used. Further, for both speaker identification and verification tasks the explicit modelling provides relatively more complimentary information to the state-of-the-art vocal tract features. The contribution of the explicit features is relatively more robust against noise. We suggest that the explicit approach can be used to model the subsegmental excitation information for speaker recognition.

Acoustic discrimination of healthy swallows from upper airway movements

Spontaneous swallowing in dysphagic individuals has been shown to occur at a lower rate compared to healthy controls, and passive swallowing detection may function as a valid screening test to identify dysphagia in at-risk populations. To automate swallow identification, acoustic source and vocal tract features were extracted from two types of swallows and eight upper airway movements from nine healthy subjects. Swallow vs non-swallow classification accuracy was 96.3 ± 1.1%. The results provide useful methods for further development of automated tools for identification of patients with swallowing impairment.

Multi-subject atlas built from structural tongue magnetic resonance images

Magnetic resonance imaging (MRI) is a widely used technology for non-invasive tongue imaging. MRI can detail tongue and muscle shapes and their variability in both healthy and diseased populations. Such detail can aid significantly in the interpretation of muscle interactions in the tongue for both normal and disordered speech production. However, the size or shape of the tongue and muscles may vary from one subject to another. In addition, there exists no comprehensive and systematic framework to assess the difference and variability of tongue and muscles in a normalized space. In the present work, we built a multi-subject atlas from 20 normal subjects that are acquired using structural MRI to offer a normalized space on which all subjects from a target population can be mapped and compared. In order to find accurate one-to-one correspondences, we bounded the tongue so that each volume had the same vocal tract features. For registration, we used symmetric diffeomorphic image registration with a cross-correlation similarity metric, which is widely used in brain image analysis. The atlas facilitates a template-based segmentation in assigning anatomical labels in the images. The tongue atlas is unprecedented and opens new vistas for exploring normal and diseased oral structures and function.

Compensation for vocal tract characteristics across native and non-native languages

Perceptual compensation for speaker vocal tract properties was investigated in four groups of listeners: native speakers of English and native speakers of Dutch, native speakers of Spanish with low proficiency in English, and Spanish–English bilinguals. Listeners categorized targets on a [sofo] to [sufu] continuum. Targets were preceded by sentences that were manipulated to have either a high or a low F1 contour. All listeners performed the categorization task for targets that were preceded by Spanish, English and Dutch precursors. Results show that listeners from each of the four language backgrounds compensate for speaker vocal tract properties regardless of language-specific vowel inventory properties. Listeners also compensate when they listen to stimuli in another language. The results suggest that patterns of compensation are mainly determined by auditory properties of precursor sentences.

Improvement of Electrolaryngeal Speech Quality Using a Supraglottal Voice Source With Compensation of Vocal Tract Characteristics

Electrolarynx (EL) is a medical speech-recovery device designed for patients who have lost their original voice box due to laryngeal cancer. As a substitute for human larynx, the current commercial EL voice source cannot reconstruct natural EL speech under laryngectomy conditions. To eliminate the abnormal acoustic properties of EL speech, a supraglottal voice source with compensation of vocal tract characteristics was proposed and provided through an experimental EL(SGVS-EL) system. The acoustic analyses of simulated EL speech and reconstructed EL speech produced with different voice sources were performed in the normal subject and laryngectomee. The results indicated that the supraglottal voice source was successful in improving the acoustic properties of EL speech by enhancing low- frequency energy, correcting the shifted formants to normal range, and eliminating the visible spectral zeros. Both normal subject and laryngectomee also produced more natural vowels using SGVS-EL than commercial EL, even if the vocal tract parameter was substituted and the supraglottal voice source was biased to a certain degree. Therefore, supraglottal voice source is a feasible and effective approach to improving the acoustic quality of EL speech.

An Investigation of Vocal Tract Characteristics for Acoustic Discrimination of Pathological Voices

This paper investigates the effectiveness of measures related to vocal tract characteristics in classifying normal and pathological speech. Unlike conventional approaches that mainly focus on features related to the vocal source, vocal tract characteristics are examined to determine if interaction effects between vocal folds and the vocal tract can be used to detect pathological speech. Especially, this paper examines features related to formant frequencies to see if vocal tract characteristics are affected by the nature of the vocal fold-related pathology. To test this hypothesis, stationary fragments of vowel /aa/ produced by 223 normal subjects, 472 vocal fold polyp subjects, and 195 unilateral vocal cord paralysis subjects are analyzed. Based on the acoustic-articulatory relationships, phonation for pathological subjects is found to be associated with measures correlated with a raised tongue body or an advanced tongue root. Vocal tract-related features are also found to be statistically significant from the Kruskal-Wallis test in distinguishing normal and pathological speech. Classification results demonstrate that combining the formant measurements with vocal fold-related features results in improved performance in differentiating vocal pathologies including vocal polyps and unilateral vocal cord paralysis, which suggests that measures related to vocal tract characteristics may provide additional information in diagnosing vocal disorders.

The effects of prior access to talker information on vowel identification in single- and mixed-talker contexts

Speech intelligibility is significantly impaired when words are spoken by multiple- compared with single talkers. In the present study, we examined whether providing listeners with information about the vocal tract characteristics of the upcoming speaker would reduce the difference between single- and mixed-talker conditions. All participants were initially trained to identify 6 talkers (3 male and 3 female) from isolated vowels. Participants then completed closed-set vowel identification tests, including a blocked and a mixed condition, and one of two mixed-talker precursor conditions. In one precursor condition, participants saw one of the six talker’s name immediately prior to hearing the target vowel. In the other precursor condition, participants heard a non-target vowel (/i/) spoken immediately before the target stimulus. For both precursor conditions, the name (or vowel) precursor matched the target-vowel speaker on half of the trials. For the other half, the precursor was of the same gender as the target-vowel speaker. Only when a sample vowel precursor was spoken by the same talker as the subsequent target was there a significant improvement in scores relative to the mixed-talker condition. The results suggest that exposure to isolated vowels can provide enough information about that talker’s vocal tract to improve perceptual normalization.

Comparing ANN and GMM in a voice conversion framework

In this paper, we present a comparative analysis of artificial neural networks (ANNs) and Gaussian mixture models (GMMs) for design of voice conversion system using line spectral frequencies (LSFs) as feature vectors. Both the ANN and GMM based models are explored to capture nonlinear mapping functions for modifying the vocal tract characteristics of a source speaker according to a desired target speaker. The LSFs are used to represent the vocal tract transfer function of a particular speaker. Mapping of the intonation patterns (pitch contour) is carried out using a codebook based model at segmental level. The energy profile of the signal is modified using a fixed scaling factor defined between the source and target speakers at the segmental level. Two different methods for residual modification such as residual copying and residual selection methods are used to generate the target residual signal. The performance of ANN and GMM based voice conversion (VC) system are conducted using subjective and objective measures. The results indicate that the proposed ANN-based model using LSFs feature set may be used as an alternative to state-of-the-art GMM-based models used to design a voice conversion system.

A pitch synchronous approach to design voice conversion system using source-filter correlation

We propose a pitch synchronous approach to design the voice conversion system taking into account the correlation between the excitation signal and vocal tract system characteristics of speech production mechanism. The glottal closure instants (GCIs) also known as epochs are used as anchor points for analysis and synthesis of the speech signal. The Gaussian mixture model (GMM) is considered to be the state-of-art method for vocal tract modification in a voice conversion framework. However, the GMM based models generate overly-smooth utterances and need to be tuned according to the amount of available training data. In this paper, we propose the support vector machine multi-regressor (M-SVR) based model that requires less tuning parameters to capture a mapping function between the vocal tract characteristics of the source and the target speaker. The prosodic features are modified using epoch based method and compared with the baseline pitch synchronous overlap and add (PSOLA) based method for pitch and time scale modification. The linear prediction residual (LP residual) signal corresponding to each frame of the converted vocal tract transfer function is selected from the target residual codebook using a modified cost function. The cost function is calculated based on mapped vocal tract transfer function and its dynamics along with minimum residual phase, pitch period and energy differences with the codebook entries. The LP residual signal corresponding to the target speaker is generated by concatenating the selected frame and its previous frame so as to retain the maximum information around the GCIs. The proposed system is also tested using GMM based model for vocal tract modification. The average mean opinion score (MOS) and ABX test results are 3.95 and 85 for GMM based system and 3.98 and 86 for the M-SVR based system respectively. The subjective and objective evaluation results suggest that the proposed M-SVR based model for vocal tract modification combined with modified residual selection and epoch based model for prosody modification can provide a good quality synthesized target output. The results also suggest that the proposed integrated system performs slightly better than the GMM based baseline system designed using either epoch based or PSOLA based model for prosody modification.

Average framing linear prediction coding with wavelet transform for text-independent speaker identification system

In this work, an average framing linear prediction coding (AFLPC) technique for text-independent speaker identification systems is presented. Conventionally, linear prediction coding (LPC) has been applied in speech recognition applications. However, in this study the combination of modified LPC with wavelet transform (WT), termed AFLPC, is proposed for speaker identification. The investigation procedure is based on feature extraction and voice classification. In the phase of feature extraction, the distinguished speaker’s vocal tract characteristics were extracted using the AFLPC technique. The size of a speaker’s feature vector can be optimized in term of an acceptable recognition rate by means of genetic algorithm (GA). Hence, an LPC order of 30 is found to be the best according to the system performance. In the phase of classification, probabilistic neural network (PNN) is applied because of its rapid response and ease in implementation. In the practical investigation, performances of different wavelet transforms in conjunction with AFLPC were compared with one another. In addition, the capability analysis on the proposed system was examined by comparing it with other systems proposed in literature. Consequently, the PNN classifier achieves a better recognition rate (97.36%) with the wavelet packet (WP) and AFLPC termed WPLPCF feature extraction method. It is also suggested to analyze the proposed system in additive white Gaussian noise (AWGN) and real noise environments; 58.56% for 0dB and 70.52% for 5dB. The recognition rates for the whole database of the Gaussian mixture model (GMM) reached the lowest value in case of small number of training samples.

The role of vocal-tract characteristics and pitch patterns in identifying individuality in Coo calls of Japanese macaques

The role of vocal-tract characteristics and pitch patterns in identifying individuality in Coo calls of Japanese macaques

Development of vocal tract and acoustic features in children.

Development of vocal tract and acoustic features in children.

Development of vocal tract and acoustic features in children

Development of vocal tract and acoustic features in children

Listening to different speakers: On the time-course of perceptual compensation for vocal-tract characteristics

This study used an active multiple-deviant oddball design to investigate the time-course of normalization processes that help listeners deal with between-speaker variability. Electroencephalograms were recorded while Dutch listeners heard sequences of non-words (standards and occasional deviants). Deviants were [ipapu] or [ɛpapu], and the standard was [Iɛpapu], where [Iɛ] was a vowel that was ambiguous between [ɛ] and [i]. These sequences were presented in two conditions, which differed with respect to the vocal-tract characteristics (i.e., the average 1st formant frequency) of the [papu] part, but not of the initial vowels [i], [ɛ] or [Iɛ] (these vowels were thus identical across conditions). Listeners more often detected a shift from [Iɛpapu] to [ɛpapu] than from [Iɛpapu] to [ipapu] in the high F1 context condition; the reverse was true in the low F1 context condition. This shows that listeners’ perception of vowels differs depending on the speaker's vocal-tract characteristics, as revealed in the speech surrounding those vowels. Cortical electrophysiological responses reflected this normalization process as early as about 120ms after vowel onset, which suggests that shifts in perception precede influences due to conscious biases or decision strategies. Listeners’ abilities to normalize for speaker-vocal-tract properties are for an important part the result of a process that influences representations of speech sounds early in the speech processing stream.

A test of formant frequency analyzes with simulated child-like vowels.

Speech production by children is typically characterized by a fairly high fundamental frequency of phonation and a short vocal tract length that produces high formant frequencies. Together these two characteristics contribute to the difficulty of making accurate measurements of the formants because the vocal tract transfer function may be undersampled by the voice source harmonics. In addition, the close proximity of the low-numbered harmonics (including the fundamental) to a formant may lead to strong nonlinear interaction of the acoustic pressures in the vocal tract and the glottal airflow. The purpose of this study was to use standard spectrographic and LPC techniques, as well as new pitch-synchronous method, to measure formant frequencies of child-like vowels that have been simulated with a speech production model. Each vowel will be simulated with two different representations of the voice source: a glottal area model that allows for nonlinear source-tract interaction and a glottal flow model in which vocal tract characteristics cannot affect the source. The results of the analyzes will be compared to the actual formant frequencies calculated directly from the known vocal tract area functions used to generate the vowels. [Research supported by NIH R01-DC04789.]

Robust Speaker Recognition Using Denoised Vocal Source and Vocal Tract Features

To alleviate the problem of severe degradation of speaker recognition performance under noisy environments because of inadequate and inaccurate speaker-discriminative information, a method of robust feature estimation that can capture both vocal source- and vocal tract-related characteristics from noisy speech utterances is proposed. Spectral subtraction, a simple yet useful speech enhancement technique, is employed to remove the noise-specific components prior to the feature extraction process. It has been shown through analytical derivation, as well as by simulation results, that the proposed feature estimation method leads to robust recognition performance, especially at low signal-to-noise ratios. In the context of Gaussian mixture model-based speaker recognition with the presence of additive white Gaussian noise, the new approach produces consistent reduction of both identification error rate and equal error rate at signal-to-noise ratios ranging from 0 to 15 dB.

Two stage emotion recognition based on speaking rate

This paper proposes two stage speech emotion recognition approach using speaking rate. The emotions considered in this study are anger, disgust, fear, happy, neutral, sadness, sarcastic and surprise. At the first stage, based on speaking rate, eight emotions are categorized into 3 broad groups namely active (fast), normal and passive (slow). In the second stage, these 3 broad groups are further classified into individual emotions using vocal tract characteristics. Gaussian mixture models (GMM) are used for developing the emotion models. Emotion classification performance at broader level, based on speaking rate is found to be around 99% for speaker and text dependent cases. Performance of overall emotion classification is observed to be improved using the proposed two stage approach. Along with spectral features, the formant features are explored in the second stage, to achieve robust emotion recognition performance in case of speaker, gender and text independent cases.

Linear predictive analysis for ultrasonic speech

Employing low-frequency ultrasonic waves to map vocal tract features is a novel research area that can lead to significant applications in speech rehabilitation for voice-loss patients, and in secure/unvoiced mobile telephony. An analytical formulation is presented to extend the extremely useful tool of linear predictive analysis from the one-dimensional audible speech domain to the three-dimensional ultrasonic speech space. The mathematical and physical analyses, supported by finite-element simulation results, prove the validity of the approach.