Itakura-Saito Distance Research Articles

Transformed wavelet applied in voice signals for noise reduction

This study presents a technique for noise reduction in speech signals adapting the wavelet transform by comparing it with the power spectral subtraction. To validate the results, segmented signal noise and Itakura Saito distance relationship were used. After analyzing the obtained results, we have verified that the technique based on Wavelet transform showed lower spectral distortion and, in some cases, a better signal to noise ratio.

A Spectral Estimation Framework for Phase Retrieval via Bregman Divergence Minimization

In this paper, we develop a novel framework to optimally design spectral estimators for phase retrieval given measurements realized from an arbitrary model. We begin by deconstructing spectral methods, and identify the fundamental mechanisms that inherently promote the accuracy of estimates. We then propose a general formalism for spectral estimation as approximate Bregman loss minimization in the range of the lifted forward model that is tractable by a search over rank-1, PSD matrices. Essentially, by the Bregman loss approach we transcend the Euclidean sense alignment based similarity measure between phaseless measurements in favor of appropriate divergence metrics over $\mathbb{R}^M_+$. To this end, we derive spectral methods that perform approximate minimization of KL-divergence, and the Itakura-Saito distance over phaseless measurements by using element-wise sample processing functions. As a result, our formulation relates and extends existing results on model dependent design of optimal sample processing functions in the literature to a model independent sense of optimality. Numerical simulations confirm the effectiveness of our approach in problem settings under synthetic and real data sets.

Comparison of affine-projection adaptive filter vs LMS speech enhancement systems

Evaluation of a two systems for speech enhancement is presented. A clasical Widrow dual channel noise canceller adaptive filter is used introducing the implemenation of Affine Projection Algorithm and comparing results versus traditional LMS implementation. Clasical Widrow noise canceller concept is used under reallistic input considerations, where signal of interest and noise are present at both inputs. Simple Root Mean Square Error (RMSE), Itakura-Saito Distance (ISD) and subjective evaluation tests are made in order to ease comparisons.

A method for detection of Mode-Mixing problem

Classical Empirical Mode Decomposition (EMD) is a data-driven method used to analyze non-linear and non-stationary time series data. Besides being an adaptable method by its nature, EMD assumes that every data consists of oscillations of the intrinsic mode functions (IMF). EMD also requires the condition that IMFs which represent the characteristic structures in the data should show only a unique sub-characteristic of the data. However, in some cases, depending on the way the sub-characteristics which make up a sophisticated data coexist, the IMFs are able to be not unique. This is called the mode-mixing problem. Although there are many studies and successful methods (such as EEMD, CEEMDAN) for eliminating the mode-mixing problem, a limited number of studies exist on determining the presence of the aforementioned problem. In this study, a method for the determination of the mode-mixing problem is proposed. In the suggested method, the Itakura–Saito distance, which is a measurement of the similarity of stationary signals and based on Fourier spectrums, is modified by applying Kaiser filter onto short-time signals. The performance of the method is tested via various applications with simulated and real data, and the results show successful detection of the mode-mixing if it exists in time series.

M[formula omitted]-spectral estimation: A relative entropy approach

This paper deals with M2-signals, namely multivariate (or vector-valued) signals defined over a multidimensional domain. In particular, we propose an optimization technique to solve the covariance extension problem for stationary random vector fields. The multidimensional Itakura–Saito distance is employed as an optimization criterion to select the solution among the spectra satisfying a finite number of moment constraints. In order to avoid technicalities that may happen on the boundary of the feasible set, we deal with the discrete version of the problem where the multidimensional integrals are approximated by Riemann sums. The spectrum solution is also discrete, which occurs naturally when the underlying random field is periodic. We show that a solution to the discrete problem exists, is unique and depends smoothly on the problem data. Therefore, we have a well-posed problem whose solution can be tuned in a smooth manner. Finally, we have applied our theory to the target parameter estimation problem in an integrated system of automotive modules. Simulation results show that our spectral estimator has promising performance.

Model averaging estimation for conditional volatility models with an application to stock market volatility forecast

AbstractThis paper is concerned with model averaging estimation for conditional volatility models. Given a set of candidate models with different functional forms, we propose a model averaging estimator and forecast for conditional volatility, and construct the corresponding weight‐choosing criterion. Under some regulatory conditions, we show that the weight selected by the criterion asymptotically minimizes the true Kullback–Leibler divergence, which is the distributional approximation error, as well as the Itakura–Saito distance, which is the distance between the true and estimated or forecast conditional volatility. Monte Carlo experiments support our newly proposed method. As for the empirical applications of our method, we investigate a total of nine major stock market indices and make a 1‐day‐ahead volatility forecast for each data set. Empirical results show that the model averaging forecast achieves the highest accuracy in terms of all types of loss functions in most cases, which captures the movement of the unknown true conditional volatility.

Model Averaging Estimation for Conditional Volatility Models

This paper is concerned with the model averaging estimation for the conditional volatility model family. We propose a model averaging estimator for the conditional volatility under a framework of zero conditional mean and construct the corresponding weight choosing criterion. It is shown that the weight selected by the criterion asymptotically minimizes the true KL divergence as well as the Itakura-Saito distance. The forecast of the conditional volatility is also constructed under the model averaging framework. As a generalization, we extend the method to the framework where both the conditional mean and volatility require estimation. On this condition, we propose the double-weight averaging estimator that assigns separate weights to the conditional mean and volatility. Monte Carlo experiments are in favor of our newly-proposed method. For empirical application, we apply the model averaging method to the forecast of the daily volatility of Shanghai Composite Index and Shenzhen Composite Index between March 10, 2016 and June 1, 2018. We find that compared with other commonly-used methods, the model averaging forecast leads to the highest forecast accuracy even under different loss functions.

Entropic Proximal Operators for Nonnegative Trigonometric Polynomials

Signal processing applications of semidefinite optimization are often rooted in sum-of-squares representations of nonnegative trigonometric polynomials. Interior-point solvers for semidefinite optimization can handle constraints of this form with a per-iteration-complexity that is cubic in the degree of the trigonometric polynomial. The purpose of this paper is to discuss first-order methods with a lower complexity per iteration. The methods are based on generalized proximal operators defined in terms of the Itakura–Saito distance. This is the Bregman distance defined by the negative entropy function. The choice for the Itakura–Saito distance is motivated by the fact that the associated generalized projection on the set of normalized nonnegative trigonometric polynomials can be computed at a cost that is roughly quadratic in the degree of the polynomial. The generalized projection is the key operation in generalized proximal first-order methods that use Bregman distances instead of the squared Euclidean distance. The paper includes numerical results with Auslender and Teboulle's accelerated proximal gradient method for Bregman distances.

Comparative Study between the Discrete-Frequency Kalman Filtering and the Discrete-Time Kalman Filtering with Application in Noise Reduction in Speech Signals

This article aims to carry out a comparative study between discrete-time and discrete-frequency Kalman filters. In order to assess the performance of both methods for speech reconstruction, we measured the output segmental signal-to-noise ratio and the Itakura-Saito distance provided by each algorithm over 25 different voice signals. The results show that although the two algorithms performed very similarly regarding noise reduction, the discrete-time Kalman filter produced smaller spectral distortion on the estimated signals when compared with the discrete-frequency Kalman filter.

Stereophonic Acoustic Echo Suppression for Speech Interfaces for Intelligent TV Applications

In this paper, a unified framework for using speech interfaces with intelligent televisions (TVs) is proposed. The proposed framework is primarily based on suppressing the stereophonic acoustic echo of the TV audio sounds and adaptive tracking of the time varying acoustic transfer function (ATF) of the echo sounds. In order to effectively suppress the acoustic echoes from the TV speakers, an optimal filter with a multichannel subspace approach is imposed on noisy inputs and the second order statistics of the stereophonic echoes are adaptively estimated by tracking the time varying ATF using the normalized least mean square method. Through assessing the proposed method with a database collected in real TV watching environments, noticeable improvements were achieved in the signal to noise ratio gain, Itakura–Saito distance, and speech recognition rates.

Likelihood Analysis of Power Spectra and Generalized Moment Problems

We develop an approach to the spectral estimation that has been advocated by [A. Ferrante et al. , “Time and spectral domain relative entropy: A new approach to multivariate spectral estimation,” IEEE Trans. Autom. Control , vol. 57, no. 10, pp. 2561–2575, Oct. 2012.] and, in the context of the scalar-valued covariance extension problem, by [P. Enqvist and J. Karlsson, “Minimal itakura-saito distance and covariance interpolation,” in Proc. 47th IEEE Conf. Decision Control , 2008, pp. 137–142]. The aim is to determine the power spectrum that is consistent with given moments and minimizes the relative entropy between the probability law of the underlying Gaussian stochastic process to that of a prior. The approach is analogous to the framework of earlier work by Byrnes, Georgiou, and Lindquist and can also be viewed as a generalization of the classical work by Burg and Jaynes on the maximum entropy method. In this paper, we present a new fast algorithm in the general case (i.e., for general Gaussian priors) and show that for priors with a specific structure the solution can be given in closed form.

Itakura-Saito distance based autoencoder for dimensionality reduction of mass spectra

Small signals may contain important information. Mass spectra of chemical compounds are usually given in a format of sparse high-dimensional data of large dynamic range. As peaks at high m/z (mass to charge ratio) region of a mass spectrum contribute to sensory information, they should not be ignored during the dimensionality reduction process even if the peak is small. However, in most of dimensionality reduction techniques, large peaks in a dataset are typically more emphasized than tiny peaks when Euclidean space is assessed. Autoencoders are widely used nonlinear dimensionality reduction technique, which is known as one special form of artificial neural networks to gain a compressed, distributed representation after learning. In this paper, we present an autoencoder which uses IS (Itakura-Saito) distance as its cost function to achieve a high capability of approximation of small target inputs in dimensionality reduction. The result of comparative experiments showed that our new autoencoder achieved the higher performance in approximation of small targets than that of the autoencoders with conventional cost functions such as the mean squared error and the cross-entropy.

A NOVEL APPROACH TO STUTTERED SPEECH CORRECTION

Stuttered speech is a dysfluency rich speech, more prevalent in males than females. It has been associated with insufficient air pressure or poor articulation, even though the root causes are more complex. The primary features include prolonged speech and repetitive speech, while some of its secondary features include, anxiety, fear, and shame. This study used LPC analysis and synthesis algorithms to reconstruct the stuttered speech. The results were evaluated using cepstral distance, Itakura-Saito distance, mean square error, and likelihood ratio. These measures implied perfect speech reconstruction quality. ASR was used for further testing, and the results showed that all the reconstructed speech samples were perfectly recognized while only three samples of the original speech were perfectly recognized.

Binary Classification with a Pseudo Exponential Model and Its Application for Multi-Task Learning

In this paper, we investigate the basic properties of binary classification with a pseudo model based on the Itakura–Saito distance and reveal that the Itakura–Saito distance is a unique appropriate measure for estimation with the pseudo model in the framework of general Bregman divergence. Furthermore, we propose a novelmulti-task learning algorithm based on the pseudo model in the framework of the ensemble learning method. We focus on a specific setting of the multi-task learning for binary classification problems. The set of features is assumed to be common among all tasks, which are our targets of performance improvement. We consider a situation where the shared structures among the dataset are represented by divergence between underlying distributions associated with multiple tasks. We discuss statistical properties of the proposed method and investigate the validity of the proposed method with numerical experiments.

Parametric representation of speech employing multi-component AFM signal model

In this paper, we have proposed parametric representation of speech signals employing a novel multi-component amplitude and frequency modulated (AFM) sinusoidal signal model. The Fourier–Bessel (FB) series expansion is used to separate the multi-component speech signal into a set of mono-component signals. It has been shown that the first component or low-frequency component can be modeled with one set of parameters for the complete signal length. For other components of the speech which is a non-stationary signal, segmentation is required in order to apply the AFM signal model. We have proposed modeling of the second and third (and higher) components based on the AFM model with time-varying parameters. Thus, the signal is to be modeled in segments by selecting suitable length where the AFM signal model is admissible. The Itakura–Saito distance and root mean square log-spectral measure have been applied to determine distortion between the actual and modeled speech signals. Simulation results demonstrate the suitability of the AFM signal model for speech signal representation.

Optimal Selection of Wavelet and Threshold Using Cuckoo Search for Noise Suppression in Speech Signals

Application of threshold based techniques to wavelet transformed speech signals can act as the noise suppression algorithm. There are types of wavelets and thresholding algorithms which exist and selection of optimal wavelet and optimal thresholding technique based on the input speech signal is vital for having better noise suppression. Selection of optimal number of decomposition levels is also has an impact on the resultant signal. In this paper, optimal selection of wavelet, level and thresholding technique for noise suppression in speech signals using Cuckoo search is proposed. After finding these, the signal is wavelet transformed andis applied proposed adaptive coefficient process which is then done thresholding. Subsequently, reconstruction is carried out to have the noise suppressed signal. The implementation is carried out with MATLAB and the evaluation metrics employed are Itakura–Saito distance (IS) and MSE.The results are taken under various noise conditions and compared with existing technique. From the results obtained, IS and MSE values for proposed is far lower than existing technique.Total IS average for proposed was about 0.78×105 compared with 1.3×105 that of existing. Total MSE average for proposed was about 0.22×10-3 compared with 1.25×10-3 that of existing. The results show the effectiveness of the proposed technique.

General H-theorem and Entropies that Violate the Second Law

H-theorem states that the entropy production is nonnegative and, therefore, the entropy of a closed system should monotonically change in time. In information processing, the entropy production is positive for random transformation of signals (the information processing lemma). Originally, the H-theorem and the information processing lemma were proved for the classical Boltzmann-Gibbs-Shannon entropy and for the correspondent divergence (the relative entropy). Many new entropies and divergences have been proposed during last decades and for all of them the H-theorem is needed. This note proposes a simple and general criterion to check whether the H-theorem is valid for a convex divergence H and demonstrates that some of the popular divergences obey no H-theorem. We consider systems with n states Ai that obey first order kinetics (master equation). A convex function H is a Lyapunov function for all master equations with given equilibrium if and only if its conditional minima properly describe the equilibria of pair transitions Ai ⇌ Aj . This theorem does not depend on the principle of detailed balance and is valid for general Markov kinetics. Elementary analysis of pair equilibria demonstrate that the popular Bregman divergences like Euclidian distance or Itakura-Saito distance in the space of distribution cannot be the universal Lyapunov functions for the first-order kinetics and can increase in Markov processes. Therefore, they violate the second law and the information processing lemma. In particular, for these measures of information (divergences) random manipulation with data may add information to data. The main results are extended to nonlinear generalized mass action law kinetic equations.

A New Family of High-Resolution Multivariate Spectral Estimators

In this paper, we extend the Beta divergence family to multivariate power spectral densities. Similarly to the scalar case, we show that it smoothly connects the multivariate Kullback-Leibler divergence with the multivariate Itakura-Saito distance. We successively study a spectrum approximation problem, based on the Beta divergence family, which is related to a multivariate extension of the THREE spectral estimation technique. It is then possible to characterize a family of solutions to the problem. An upper bound on the complexity of these solutions will also be provided. Finally, we will show that the most suitable solution of this family depends on the specific features required from the estimation problem.

Enhancement of Speech Signals Using Weighted Mask and Neuro-Fuzzy Classifier

In this paper, we present an effective noise suppression technique for enhancement of speech signals using weighted mask and Neuro-fuzzy. Initially, the noisy speech signal is broken down into various time-frequency (TF) units and the features are extracted by finding out the Modified Amplitude Magnitude Spectrogram (MAMS). The signal is then classified into respective classes based on the ratio value between the estimated spectrum value and the original spectrum value using Neuro-Fuzzy Classifier. Subsequently, in the enhancement stage, filtered waveforms are windowed and then weighted by the mask value and summed up to get the enhanced signal. We have used evaluation metrics parameters of PESQ (Perceptual Evaluation of Speech Quality), IS (Itakura–Saito distance) and MARS based composite measures in-order to evaluate the proposed technique. We have taken sound samples under various conditions from two databases. We have compared our proposed technique having different Neuro-fuzzy classifiers with the previous technique (having Bayesian classifier). We have found out that the proposed technique achieves good results. Average values obtained for proposed technique considering all noise categories at -5dB had PESQ score of 0.9738, IS score of 17.558, Mars based composite measure of 3.728. The highest values obtained by the technique for PESQ was 1.76, for IS was 96.83 and for MARS based composite measures was 4.147.

Evaluation of a model for voice enhancement system of two channels

A dual channel system for speech enhancement is proposed and evaluated under babbling noise. A two stage dual channel structure of adaptive filters is used with different microphones, one directional for receiving the desired signal and the other omni directional for receiving the ambient noise which for this model is mainly babbling. Simple Root Mean Square (RMS), Itakura-Saito Distance (ISD) and subjective evaluation tests are made in order to ease comparisons. Using a simple adaptive filter, it is not enough to reduce background noise while maintaining the intelligibility of enhanced speech. Therefore, a noise emphasizing channel is used in order to achieve a better noise canceling. This work is done as precedent for the implementation of a system capable of diminish the noise from voices different than the target by using adaptive filter structures offering good results but small enough to be embedded in hearing aid devices.