Synthetic Speech Signals Research Articles

Processing group delay spectrograms for study of formant and harmonic contours in speech signals.

This paper deals with study of formant and harmonic contours by processing the group delay (GD) spectrograms of speech signals. The GD spectrum is the negative derivative of the phase spectrum with respect to frequency. Recent study shows that the GD spectrogram can be obtained without phase wrapping. Formant frequency contours can be observed in the display of the peaks of the instantaneous wideband equivalent GD spectrogram, derived using the modified single frequency filtering (SFF) analysis of speech signals. Harmonic frequency contours can be observed in the display of the peaks of the instantaneous narrowband equivalent GD spectrogram, derived using the modified SFF analysis of speech signals. For synthetic speech signals, the observed formant contours match the ground truth formant contours from which the signal is derived. For natural speech signals, the observed formant contours match approximately with the given ground truth formant contours mostly in the voiced regions. The results are illustrated for several randomly selected utterances from the TIMIT database. While this study helps to observe the contours of formants in the display, automatic extraction of the formant frequencies needs further processing, requiring logic for eliminating the spurious points, without forcing the number of formants.

Assessment of Tracks of Resonance Frequencies of the Vocal Tract

A new method for estimating formant frequency tracks of the vocal tract for arbitrary speech segments is proposed. The method uses the ratio of two Fourier transforms of a speech signal with special exponential-type windows depending on some parameter. This ratio is used for specific points in time and is considered as a function of frequency and parameter. By analyzing, for several parameter values, the distribution of minimum points (in terms of frequency) for the phase of this ratio and/or a similar distribution of extreme points for its amplitude, it is possible to estimate formant frequencies from the peaks of these distributions. A mathematical study is presented that substantiates this approach. A series of numerical experiments were carried out on the processing of synthetic and real speech signals, which confirmed the performance capabilities of the proposed formant evaluation method. In particular, in experiments with synthesized vowels, it was found that the error in estimating their resonance frequencies is small and stable with respect to additive noise up to a signal-to-noise ratio of 5 dB. For real speech, the method makes it possible to calculate the formant frequency tracks for both sounds with vocal excitation and for voiceless fricatives, aspirated plosives, and whispered speech.

Effects of Natural Versus Synthetic Consonant and Vowel Stimuli on Cortical Auditory-Evoked Potential.

Background and ObjectivesNatural and synthetic speech signals effectively stimulate cortical auditory evoked potential (CAEP). This study aimed to select the speech materials for CAEP and identify CAEP waveforms according to gender of speaker (GS) and gender of listener (GL).Subjects and MethodsTwo experiments including a comparison of natural and synthetic stimuli and CAEP measurement were performed of 21 young announcers and 40 young adults. Plosive /g/ and /b/ and aspirated plosive /k/ and /p/ were combined to /a/. Six bisyllables–/ga/-/ka/, /ga/-/ba/, /ga/-/pa/, /ka/-/ba/, /ka/-/pa/, and /ba/-/pa/–were formulated as tentative forwarding and backwarding orders. In the natural and synthetic stimulation mode (SM) according to GS, /ka/ and /pa/ were selected through the first experiment used for CAEP measurement.ResultsThe correction rate differences were largest (74%) at /ka/-/ pa/ and /pa/-/ka/; thus, they were selected as stimulation materals for CAEP measurement. The SM showed shorter latency with P2 and N1-P2 with natural stimulation and N2 with synthetic stimulation. The P2 amplitude was larger with natural stimulation. The SD showed significantly larger amplitude for P2 and N1-P2 with /pa/. The GS showed shorter latency for P2, N2, and N1-P2 and larger amplitude for N2 with female speakers. The GL showed shorter latency for N2 and N1-P2 and larger amplitude for N2 with female listeners.ConclusionsAlthough several variables showed significance for N2, P2, and N1-P2, P1 and N1 did not show any significance for any variables. N2 and P2 of CAEP seemed affected by endogenous factors.

Time-varying quasi-closed-phase analysis for accurate formant tracking in speech signals

In this paper, we propose a new method for the accurate estimation and tracking of formants in speech signals using time-varying quasi-closed-phase (TVQCP) analysis. Conventional formant tracking methods typically adopt a two-stage estimate-and-track strategy wherein an initial set of formant candidates are estimated using short-time analysis (e.g., 10–50 ms), followed by a tracking stage based on dynamic programming or a linear state-space model. One of the main disadvantages of these approaches is that the tracking stage, however good it may be, cannot improve upon the formant estimation accuracy of the first stage. The proposed TVQCP method provides a single-stage formant tracking that combines the estimation and tracking stages into one. TVQCP analysis combines three approaches to improve formant estimation and tracking: (1) it uses temporally weighted quasi-closed-phase analysis to derive closed-phase estimates of the vocal tract with reduced interference from the excitation source, (2) it increases the residual sparsity by using the $L_1$ optimization and (3) it uses time-varying linear prediction analysis over long time windows (e.g., 100–200 ms) to impose a continuity constraint on the vocal tract model and hence on the formant trajectories. Formant tracking experiments with a wide variety of synthetic and natural speech signals show that the proposed TVQCP method performs better than conventional and popular formant tracking tools, such as Wavesurfer and Praat (based on dynamic programming), the KARMA algorithm (based on Kalman filtering), and DeepFormants (based on deep neural networks trained in a supervised manner). Matlab scripts for the proposed method can be found at: https://github.com/njaygowda/ftrack

Packet loss concealment-based estimation of polynomial interpolation for improving speech quality in VoIP

The main objective of packet loss concealment (PLC) techniques is to improve the speech quality in voice over IP (VoIP). These techniques generate a synthetic speech signal to cover the missing data or the lost packets in a received bit stream. This paper is concerned with performing a new PLC technique using the estimation of polynomial interpolation (EPI) method, which is designed to seek the approximate function as a polynomial. This function can be used to predict the lost packet from previous packets. A two state Markov model, particularly, is used to represent the lost packets. The used test vectors, in the proposed PLC evaluation, are the TIMIT database. Thus, the proposed PLC algorithm, which is provided better speech quality, is evaluated by PESQ and SNR and compared to two techniques; the hidden Markov model (HMM)-based PLC algorithm and the deep neural network (DNN)-based PLC.

Packet loss concealment-based estimation of polynomial interpolation for improving speech quality in VoIP

The main objective of packet loss concealment (PLC) techniques is to improve the speech quality in voice over IP (VoIP). These techniques generate a synthetic speech signal to cover the missing data or the lost packets in a received bit stream. This paper is concerned with performing a new PLC technique using the estimation of polynomial interpolation (EPI) method, which is designed to seek the approximate function as a polynomial. This function can be used to predict the lost packet from previous packets. A two state Markov model, particularly, is used to represent the lost packets. The used test vectors, in the proposed PLC evaluation, are the TIMIT database. Thus, the proposed PLC algorithm, which is provided better speech quality, is evaluated by PESQ and SNR and compared to two techniques; the hidden Markov model (HMM)-based PLC algorithm and the deep neural network (DNN)-based PLC.

Robust Speech Filter and Voice Encoder Parameter Estimation Using the Phase–Phase Correlator

In recent years, linear prediction voice encoders have become very efficient in terms of computing execution time and channel bandwidth usage while providing, in the absence of impulsive noise, natural sounding synthetic speech signals. This good performance has been achieved via the use of a maximum likelihood parameter estimation of an auto-regressive model of order ten that best fits the speech signal under the assumption that the signal and the noise are Gaussian stochastic processes. However, this method breaks down in the presence of impulse noise, which is common in practice, resulting in harsh or non-intelligible audio signals. In this paper, we propose a robust estimator of correlation, the Phase-Phase correlator that is able to cope with impulsive noise. Utilizing this correlator, we develop a Robust Mixed Excitation Linear Prediction encoder that provides improved audio quality for voiced, unvoiced, and transition speech segments. This is achieved by applying a statistical test to robust Mahalanobis distances for identifying the outliers in the corrupted speech signal, which are then replaced with filtered signals. Simulation results reveal that the proposed estimator of correlator outperforms in variance, bias, and breakdown point compared to three other robust approaches based on the arcsin law, the polarity coincidence correlator, and the median-of-ratio estimator without sacrificing the encoder bandwidth efficiency and the compression gain while remaining compatible with real-time applications. Furthermore, in the presence of impulsive noise, the proposed speech encoder speech subjective quality outperforms the state-of-the-art in terms of mean opinion score.

Simultaneous Estimation of Glottal Source Waveforms and Vocal Tract Shapes from Speech Signals Based on ARX-LF Model

Estimating glottal source waveforms and vocal tract shapes is typically done by processing the speech signal using an inverse filter and then fitting the residual signal using the glottal source model. However, due to source-tract interactions, the estimation accuracy is reduced. In this paper, we propose a method to estimate glottal source waveforms and vocal tract shapes simultaneously based on an analysis-by-synthesis approach with a source-filter model constructed of an Auto-Regressive eXogenous (ARX) model and the Liljencrants-Fant (LF) model. Since the optimization of multiple parameters makes simultaneous estimation difficult, we first initialize the glottal source parameters using the inverse filter method, and then simultaneously estimate the accurate parameters of the glottal sources and the vocal tract shapes using an analysis-by-synthesis approach. Experimental results with synthetic and real speech signals showed that the proposed method has higher estimation accuracy than using the inverse filter.

Bayesian Sparse Linear Prediction with Pearson Type VII Distribution

The speech signal can be modelled as AR models with an innovation noise model. The Pearson type VII distribution is used to model the real excitation. Variational Bayesian framework is used to estimate the posteriors of the AR coefficients and noise model parameters. The model is not conjugate, so MCMC is embed into VB framework to estimate the degree of freedom (DOF) parameter of Pearson type VII distribution. The model order selection is carried out by setting ARD priors on the coefficients. Simulation is carried out on synthetic and real data, the results show that the algorithm performs well for linear prediction both for synthetic data and speech signal, and the results are better than using least square method.

Lexical Dependent Emotion Detection Using Synthetic Speech Reference

This paper aims to create neutral reference models from synthetic speech to contrast the emotional content of a speech signal. Modeling emotional behaviors is a challenging task due to the variability in perceiving and describing emotions. Previous studies have indicated that relative assessments are more reliable than absolute assessments. These studies suggest that having a reference signal with known emotional content (e.g., neutral emotion) to compare a target sentence may produce more reliable metrics to identify emotional segments. Ideally, we would like to have an emotionally neutral sentence with the same lexical content as the target sentence where their contents are timely aligned. In this fictitious scenario, we would be able to identify localized emotional cues by contrasting frame-by-frame the acoustic features of the target and reference sentences. This paper explores the idea of building these reference sentences leveraging the advances in speech synthesis. This paper builds a synthetic speech signal that conveys the same lexical information and is timely aligned with the target sentence in the database. Since it is expected that a single synthetic speech will not capture the full range of variability observed in neutral speech, we build multiple synthetic sentences using various voices and text-to-speech approaches. This paper analyzes whether the synthesized signals provide valid template references to describe neutral speech using feature analysis and perceptual evaluation. Finally, we demonstrate how this framework can be used in emotion recognition, achieving improvements over classifiers trained with the state-of-the-art features in detecting low versus high levels of arousal and valence.

Automatic Speech Recognition for VoIP with Packet Loss Concealment

This paper proposes a packet loss concealment (PLC) technique for increase the robustness of automatic speech recognition (ASR) of speech coded with the G729 codec, on the Voice over Internet Protocol (VoIP). Many of the standard ITU-T CELP based speech coders, such as the G.723.1, G.728, and G.729, model speech reproduction in their decoders. These decoders have enough state information to integrate PLC algorithms directly in the decoder, and are specified as part of their standards in particular by PLC based ITU-T G711 Appendix I. Speech is transmitted with source and channel codes optimized, this channel is simulated by two states Markov model to modeled loss packets. The objective of PLC based ITU-T G711 Appendix I is to generate a synthetic speech signal to cover missing data or loss packets in a received bit stream for the ASR application, i.e., to minimize word error rate.

Alternating minimisation for glottal inverse filtering

A new method is proposed for solving the glottal inverse filtering (GIF) problem. The goal of GIF is to separate an acoustical speech signal into two parts: the glottal airflow excitation and the vocal tract filter. To recover such information one has to deal with a blind deconvolution problem. This ill-posed inverse problem is solved under a deterministic setting, considering unknowns on both sides of the underlying operator equation. A stable reconstruction is obtained using a double regularization strategy, alternating between fixing either the glottal source signal or the vocal tract filter. This enables not only splitting the nonlinear and nonconvex problem into two linear and convex problems, but also allows the use of the best parameters and constraints to recover each variable at a time. This new technique, called alternating minimization glottal inverse filtering (AM-GIF), is compared with two other approaches: Markov chain Monte Carlo glottal inverse filtering (MCMC-GIF), and iterative adaptive inverse filtering (IAIF), using synthetic speech signals. The recent MCMC-GIF has good reconstruction quality but high computational cost. The state-of-the-art IAIF method is computationally fast but its accuracy deteriorates, particularly for speech signals of high fundamental frequency (F0). The results show the competitive performance of the new method: With high F0, the reconstruction quality is better than that of IAIF and close to MCMC-GIF while reducing the computational complexity by two orders of magnitude.

Single channel blind source separation on the instantaneous mixed signal of multiple dynamic sources

Abstract Single Channel Blind Source Separation (SCBSS) has had many algorithms for artificial mixed signal, where the number of mixing sources is assumed to be known, and mixed signal used in validation algorithm only contains two signal sources. However, in real-world application, the mixed number of sources is unknown and is usually more than two. This paper presents a new single-channel blind source separation algorithm based on the multi-channel mapping and Independent Component Analysis (ICA), which supposes that mixed signal comes from a dynamic system in which any component depends on the interaction of other components and signals are linear instantaneous mixture. The mathematical model demonstrates the single channel signal of linear instantaneous mixture. In order to map single channel signals into multi-channel signals, Takens theory and C-C method are used to estimate the time delay and the embedding dimension in the time series of the dynamic system. FastICA for multi-channel blind source separation is improved by using FSS-Kernel (Finite Support Samples Kernel), where the nonlinear function of FastICA is replaced by PDF (probability density function) and estimated by FSS-Kernel. The experiments are conducted to evaluate the proposed algorithm of single channel blind source separation, in which the synthetic signals and speech signals are used respectively. The experiment results show that the proposed method is very effective to estimate the number of independent components and is practical to separate two or more mixed signals.

Algorithm for Jitter and Shimmer Measurement in Pathologic Voices

An algorithm to measure the jitter (jitta, jitter, rap and ppq5) and shimmer (ShdB, Shim, apq3 and apq5) parameters was developed. These parameters can be used in an intelligent system to diagnose voice pathologies. The algorithm is sensitive to the fundamental frequency and determines the parameters based on maximum and minimum functions applied to each glottal period of the signal. The algorithm was previously tested with synthesized speech signals with very high accuracy, but several improvements had to be included for the analysis of pathologic voices. A comparison using real, control and pathologic voices was made between the developed algorithm and other software program to evaluate the consistency of measures between different algorithms. Based on this comparison it can be said that the algorithm improved the accuracy with synthetic speech signal and showed consistent measures for pathologic speech signal.

A New Algorithm for Noisy Speech Classification Based on GMM

Speech can be broadly categorized into voiceless, voiced, and mute signal, in which voiced speech can be further classified into vowel and voiced consonant. With the ever increasing demand of the speech synthesis applications, it is urgent to develop an effective classification method to differentiate vowel and voiced consonant signal since they are two distinct components that affect the naturalness of the synthetic speech signal. State-of-the-arts algorithms for speech signal classification are effective in classifying voiceless, voiced and mute speech signal, however, not effective in further classifying the voiced signal. In view of the issue, a new algorithm for speech classification based on Gaussian Mixture Model (GMM) is proposed, which can directly classify a speech into voiceless, voiced consonant, vowel and mute signal. Simulation results demonstrate that the proposed algorithm is effective even under the noisy environments.

Automatic glottal inverse filtering with the Markov chain Monte Carlo method

This paper presents a new glottal inverse filtering (GIF) method that utilizes a Markov chain Monte Carlo (MCMC) algorithm. First, initial estimates of the vocal tract and glottal flow are evaluated by an existing GIF method, iterative adaptive inverse filtering (IAIF). Simultaneously, the initially estimated glottal flow is synthesized using the Rosenberg–Klatt (RK) model and filtered with the estimated vocal tract filter to create a synthetic speech frame. In the MCMC estimation process, the first few poles of the initial vocal tract model and the RK excitation parameter are refined in order to minimize the error between the synthetic and original speech signals in the time and frequency domain. MCMC approximates the posterior distribution of the parameters, and the final estimate of the vocal tract is found by averaging the parameter values of the Markov chain. Experiments with synthetic vowels produced by a physical modeling approach show that the MCMC-based GIF method gives more accurate results compared to two known reference methods.

A New Method for Noisy Speech Classification Based on Gaussian Mixture Models

Speech can be broadly categorized into voiceless, voiced, and mute signal, in which voiced speech can be further classified into vowel and voiced consonant. With the ever increasing demand of the speech synthesis applications, it is urgent to develop an effective classification method to differentiate vowel and voiced consonant signal since they are two distinct components that affect the naturalness of the synthetic speech signal. State-of-the-arts algorithms for speech signal classification are effective in classifying voiceless, voiced and mute speech signal, however, not effective in further classifying the voiced signal. In view of the issue, a new algorithm for speech classification based on Gaussian Mixture Model (GMM) is proposed, which can directly classify a speech into voiceless, voiced consonant, vowel and mute signal. Specifically, a new speech feature is proposed, and the GMM is also modified for speech classification. Simulation results demonstrate that the proposed algorithm is effective even under the noisy environments.

A comparative study of glottal source estimation techniques

Abstract Source-tract decomposition (or glottal flow estimation) is one of the basic problems of speech processing. For this, several techniques have been proposed in the literature. However, studies comparing different approaches are almost nonexistent. Besides, experiments have been systematically performed either on synthetic speech or on sustained vowels. In this study we compare three of the main representative state-of-the-art methods of glottal flow estimation: closed-phase inverse filtering, iterative and adaptive inverse filtering, and mixed-phase decomposition. These techniques are first submitted to an objective assessment test on synthetic speech signals. Their sensitivity to various factors affecting the estimation quality, as well as their robustness to noise are studied. In a second experiment, their ability to label voice quality (tensed, modal, soft) is studied on a large corpus of real connected speech. It is shown that changes of voice quality are reflected by significant modifications in glottal feature distributions. Techniques based on the mixed-phase decomposition and on a closed-phase inverse filtering process turn out to give the best results on both clean synthetic and real speech signals. On the other hand, iterative and adaptive inverse filtering is recommended in noisy environments for its high robustness.

Method and Device for Encoding Speech using Open-Loop Pitch Analysis

The present invention is a synthetic speech encoding device that produces a synthetic speech signal which closely matches an actual speech signal. The actual speech signal is digitized, and excitation pulses are selected by minimizing the error between the actual and synthetic speech signals. The preferred pattern of excitation pulses needed to produce the synthetic speech signal is obtained by using an excitation pattern containing a multiplicity of weighted pulses at timed positions. The selection of the location and amplitude of each excitation pulse is obtained by minimizing an error criterion between the synthetic speech signal and the actual speech signal. The error criterion function incorporates a perceptual weighting filter which shapes the error spectrum.

A Robust ICA-Based Adaptive Filter Algorithm for System Identification

This paper proposes a new adaptive filter algorithm for system identification using independent component analysis. The additive noise is considered as an independent component to be separated from the noisy observation and is simultaneously estimated online. The proposed algorithm is derived by minimizing the mutual information between the estimated additive noise and the input signal. The local convergence conditions are also derived. The proposed algorithm can be directly applied to the acoustic echo canceller without any double-talk detector. Some simulations have been carried out to illustrate its effectiveness for synthetic and real speech signals.