Heavy-tailed Noise Research Articles

Alternative Measures of Dependence for Cyclic Behaviour Identification in the Signal with Impulsive Noise—Application to the Local Damage Detection

The local damage detection procedures in rotating machinery are based on the analysis of the impulsiveness and/or the periodicity of disturbances corresponding to the failure. Recent findings related to non-Gaussian vibration signals showed some drawbacks of the classical methods. If the signal is noisy and it is strongly non-Gaussian (heavy-tailed), searching for impulsive behvaior is pointless as both informative and non-informative components are transients. The classical dependence measure (autocorrelation) is not suitable for non-Gaussian signals. Thus, there is a need for new methods for hidden periodicity detection. In this paper, an attempt will be made to use alternative measures of dependence used in time series analysis that are less known in the condition monitoring (CM) community. They are proposed as alternatives for the classical autocovariance function used in the cyclostationary analysis. The methodology of the auto-similarity map calculation is presented as well as a procedure for a “quality” or “informativeness” assessment of the map is proposed. In the most complex case, the most resistant to heavy-tailed noise turned out the proposed techniques based on Kendall, Spearman and Quadrant autocorrelations. Whereas in the case of the local fault disturbed by the Gaussian noise, the most efficient proved to be a commonly-known approach based on Pearson autocorrelation. The ideas proposed in the paper are supported by simulation signals and real vibrations from heavy-duty machines.

Exact Frequency Estimation in the i.i.d. Noise via KL Divergence of Accumulated Power

To solve the frequency estimation deterioration caused by independent and identically distributed (i.i.d.) noise, this letter proposes an exact and robust frequency estimation algorithm based on the accumulated phase-difference power on Kullback-Leibler divergence (APP-KLD). First, the frequency estimation problem is modeled by the accumulated power of phase-difference between the receiving signal and Fourier basis function. Furthermore, the true frequency is estimated iteratively by maximizing the Kullback–Leibler divergence, which measures the distance between the accumulated power and a constant reference vector. Simulations demonstrate the robust performance of the APP-KLD algorithm at various levels of heavy-tailed noise and sample numbers.

A Novel Robust Nonlinear Kalman Filter Based on Multivariate Laplace Distribution

In this brief, we investigate the robust state estimation of nonlinear systems with heavy-tailed noise. Based on the fact that the multivariate Laplace (ML) distribution has heavier tails than Gaussian distribution but is only determined by its mean and covariance, we utilize the ML distribution to model the heavy-tailed measurement noise. Based on the Gaussian mixture form of ML distribution, the system state and the covariance of ML distribution are inferred by variational Bayesian method, and then a closed recursive robust nonlinear Kalman filter is obtained with the nonlinear integrations calculated by numerical integration methods. Simulation results demonstrate that the proposed algorithm is insensitive to its free parameters and can obtain better estimation accuracy than related robust algorithms.

Robust adaptive Kalman filter for strapdown inertial navigation system dynamic alignment

The measurement noise covariance R plays a vital role in the Kalman filter (KF) algorithm. Traditionally, a constant R is obtained by experience or experiments. However, the KF cannot achieve optimal estimation using constant R under non-Gaussian conditions. A robust strategy for adaptive estimation of R is proposed to suppress the influence of non-Gaussian noise on the in-motion alignment performance of the Doppler velocity log (DVL) velocity-aided strapdown inertial navigation system (SINS). Furthermore, an improved Sage–Husa robust adaptive KF algorithm (SHRAKF) based on the Mahalanobis distance (MD) algorithm is proposed to handle the outliers that frequently occur within the complicated underwater environment. The contributions of this work are twofold—the SHRAKF (1) designs a robust strategy to adaptively estimate R in real time and (2) further improves filtering robustness and adaptability with the MD algorithm, conditional on the DVL outputs being contaminated by non-Gaussian noise. A semi-physical simulation experiment for SINS/DVL in-motion alignment based on the test data is carried out, and the experimental results show that the SHRAKF adaptively estimates R in real time and effectively suppresses observational outliers. For non-Gaussian noise pollution, including outliers and heavy-tailed noise, the SHRAKF performs better than traditional methods.

Centered error entropy Kalman filter with application to satellite attitude determination

Due to unavoidable factors, heavy-tailed noise appears in satellite attitude estimation. Traditional Kalman filter is prone to performance degradation and even filtering divergence when facing non-Gaussian noise. The existing robust algorithms have limited accuracy. To improve the attitude determination accuracy under non-Gaussian noise, we use the centered error entropy (CEE) criterion to derive a new filter named centered error entropy Kalman filter (CEEKF). CEEKF is formed by maximizing the CEE cost function. In the CEEKF algorithm, the prior state values are transmitted the same as the classical Kalman filter, and the posterior states are calculated by the fixed-point iteration method. The CEE EKF (CEE-EKF) algorithm is also derived to improve filtering accuracy in the case of the nonlinear system. We also give the convergence conditions of the iteration algorithm and the computational complexity analysis of CEEKF. The results of the two simulation examples validate the robustness of the algorithm we presented.

Online support vector quantile regression for the dynamic time series with heavy-tailed noise

In this paper, we propose an online support vector quantile regression approach with an ɛ-insensitive pinball loss function, called Online-SVQR, for dynamic time series with heavy-tailed noise. Online-SVQR is robust to heavy-tailed noise, as it can control the negative influence of heavy-tailed noise by using a quantile parameter. By using an incremental learning algorithm to update the new samples, the coefficients of Online-SVQR reflect the dynamic information in the examined time series. During each incremental training process, the nonsupport vector is ignored while the support vector continues training with new updated samples. Online-SVQR can select useful training samples and discard irrelevant samples. As a result, the training speed of Online-SVQR is accelerated. Experimental results on one artificial dataset and three real-world datasets indicate that Online-SVQR outperforms ɛ-support vector quantile regression in terms of both sample selection ability and training speed.

Privacy protection with heavy-tailed noise for linear dynamical systems

Privacy protection in linear dynamical systems is investigated in this paper. A standard mechanism employed in systems and control literature is to mask private data by adding Gaussian noise. A shortcoming of this mechanism is that the occurrence of outliers is vulnerable. The goal of this paper is to present a novel mechanism that can hide outliers. The key idea is to utilize stably distributed noise, which has the following two preferable properties for this purpose; One is its heavy-tailed distribution that is beneficial to hide extreme values including scale-free data. The other is its closedness with respect to addition similarly to Gaussian, which enables us to design filter/controller under privacy requirements based on linear control theory. From a theoretical point of view, we quantify the privacy level of the proposed mechanism in terms of differential privacy. The derivation is nontrivial because the density function of the stable distribution has no analytic expression.

Sparse Modal Additive Model.

Sparse additive models have been successfully applied to high-dimensional data analysis due to the flexibility and interpretability of their representation. However, the existing methods are often formulated using the least-squares loss with learning the conditional mean, which is sensitive to data with the non-Gaussian noises, e.g., skewed noise, heavy-tailed noise, and outliers. To tackle this problem, we propose a new robust regression method, called as sparse modal additive model (SpMAM), by integrating the modal regression metric, the data-dependent hypothesis space, and the weighted lq,1 -norm regularizer (q ≥ 1) into the additive models. Specifically, the modal regression metric assures the model robustness to complex noises via learning the conditional mode, the data-dependent hypothesis space offers the model adaptivity via sample-based presentation, and the lq,1 -norm regularizer addresses the algorithmic interpretability via sparse variable selection. In theory, the proposed SpMAM enjoys statistical guarantees on asymptotic consistency for regression estimation and variable selection simultaneously. Experimental results on both synthetic and real-world benchmark data sets validate the effectiveness and robustness of the proposed model.

Evaluating Gaussian process metamodels and sequential designs for noisy level set estimation

We consider the problem of learning the level set for which a noisy black-box function exceeds a given threshold. To efficiently reconstruct the level set, we investigate Gaussian process (GP) metamodels. Our focus is on strongly stochastic simulators, in particular with heavy-tailed simulation noise and low signal-to-noise ratio. To guard against noise misspecification, we assess the performance of three variants: (i) GPs with Student-t observations; (ii) Student-t processes (TPs); and (iii) classification GPs modeling the sign of the response. In conjunction with these metamodels, we analyze several acquisition functions for guiding the sequential experimental designs, extending existing stepwise uncertainty reduction criteria to the stochastic contour-finding context. This also motivates our development of (approximate) updating formulas to efficiently compute such acquisition functions. Our schemes are benchmarked by using a variety of synthetic experiments in 1–6 dimensions. We also consider an application of level set estimation for determining the optimal exercise policy of Bermudan options in finance.

Robust estimation and variable selection for the accelerated failure time model.

This article concerns robust modeling of the survival time for cancer patients. Accurate prediction of patient survival time is crucial to the development of effective therapeutic strategies. To this goal, we propose a unified Expectation-Maximization approach combined with the L1 -norm penalty to perform variable selection and parameter estimation simultaneously in the accelerated failure time model with right-censored survival data of moderate sizes. Our approach accommodates general loss functions, and reduces to the well-known Buckley-James method when the squared-error loss is used without regularization. To mitigate the effects of outliers and heavy-tailed noise in real applications, we recommend the use of robust loss functions under the general framework. Furthermore, our approach can be extended to incorporate group structure among covariates. We conduct extensive simulation studies to assess the performance of the proposed methods with different loss functions and apply them to an ovarian carcinoma study as an illustration.

A Framework of Learning Through Empirical Gain Maximization.

We develop in this letter a framework of empirical gain maximization (EGM) to address the robust regression problem where heavy-tailed noise or outliers may be present in the response variable. The idea of EGM is to approximate the density function of the noise distribution instead of approximating the truth function directly as usual. Unlike the classical maximum likelihood estimation that encourages equal importance of all observations and could be problematic in the presence of abnormal observations, EGM schemes can be interpreted from a minimum distance estimation viewpoint and allow the ignorance of those observations. Furthermore, we show that several well-known robust nonconvex regression paradigms, such as Tukey regression and truncated least square regression, can be reformulated into this new framework. We then develop a learning theory for EGM by means of which a unified analysis can be conducted for these well-established but not fully understood regression approaches. This new framework leads to a novel interpretation of existing bounded nonconvex loss functions. Within this new framework, the two seemingly irrelevant terminologies, the well-known Tukey's biweight loss for robust regression and the triweight kernel for nonparametric smoothing, are closely related. More precisely, we show that Tukey's biweight loss can be derived from the triweight kernel. Other frequently employed bounded nonconvex loss functions in machine learning, such as the truncated square loss, the Geman-McClure loss, and the exponential squared loss, can also be reformulated from certain smoothing kernels in statistics. In addition, the new framework enables us to devise new bounded nonconvex loss functions for robust learning.

Sparsity-aware robust community detection (SPARCODE)

Community detection refers to finding densely connected groups of nodes in graphs. In important applications, such as cluster analysis and network modelling, the graph is sparse but outliers and heavy-tailed noise may obscure its structure. We propose a new method for Sparsity-aware Robust Community Detection (SPARCODE). Starting from a densely connected and outlier-corrupted graph, we first extract a preliminary sparsity-improved graph model where we optimize the level of sparsity by mapping the feature vectors from different communities such that the distance of their embedding is maximal. Then, undesired edges are removed and the graph is constructed robustly by detecting the outliers using the connectivity of nodes in the improved graph model. Finally, fast spectral partitioning is performed on the resulting robust sparse graph model. The number of communities is estimated using modularity optimization on the partitioning results. We compare the performance to popular graph and cluster-based community detection approaches on a variety of benchmark network and cluster analysis data sets. Comprehensive experiments demonstrate that our method consistently finds the correct number of communities and outperforms existing methods in terms of detection performance, robustness and modularity score while requiring a reasonable computation time.

Quantile spectral analysis of long-memory processes

This study examines the problem of robust spectral analysis of long-memory processes. We investigate the possibility of using Laplace and quantile periodograms for a non-Gaussian distribution structure. The Laplace periodogram, derived by the least absolute deviations in the harmonic regression procedure, demonstrates its superiority in handling heavy-tailed noise and nonlinear distortion. In this study, we discuss an asymptotic distribution of the Laplace periodogram for long-memory processes. We also derive an asymptotic distribution of the quantile periodogram. Through numerical experiments, we demonstrate the robustness of the Laplace periodogram and the usefulness of the quantile periodogram in detecting the hidden frequency for the spectral analysis of the long-memory process under non-Gaussian distribution. Moreover, as an application of robust periodograms under the long-memory process, we discuss the long-memory parameter estimation based on a log periodogram regression approach.

Hierarchical Dynamical Model for Multiple Cortical Neural Decoding.

Motor brain machine interfaces (BMIs) interpret neural activities from motor-related cortical areas in the brain into movement commands to control a prosthesis. As the subject adapts to control the neural prosthesis, the medial prefrontal cortex (mPFC), upstream of the primary motor cortex (M1), is heavily involved in reward-guided motor learning. Thus, considering mPFC and M1 functionality within a hierarchical structure could potentially improve the effectiveness of BMI decoding while subjects are learning. The commonly used Kalman decoding method with only one simple state model may not be able to represent the multiple brain states that evolve over time as well as along the neural pathway. In addition, the performance of Kalman decoders degenerates in heavy-tailed nongaussian noise, which is usually generated due to the nonlinear neural system or influences of movement-related noise in online neural recording. In this letter, we propose a hierarchical model to represent the brain states from multiple cortical areas that evolve along the neural pathway. We then introduce correntropy theory into the hierarchical structure to address the heavy-tailed noise existing in neural recordings. We test the proposed algorithm on in vivo recordings collected from the mPFC and M1 of two rats when the subjects were learning to perform a lever-pressing task. Compared with the classic Kalman filter, our results demonstrate better movement decoding performance due to the hierarchical structure that integrates the past failed trial information over multisite recording and the combination with correntropy criterion to deal with noisy heavy-tailed neural recordings.

Limit Theory for Stationary Autoregression with Heavy-Tailed Augmented GARCH Innovations

This paper considers stationary autoregressive (AR) models with heavy-tailed, general GARCH (G-GARCH) or augmented GARCH noises. Limit theory for the least squares estimator (LSE) of autoregression coefficient ρ=ρn is derived uniformly over stationary values in [0,1), focusing on ρn→1 as sample size n tends to infinity. For tail index α∈(0,4) of G-GARCH innovations, asymptotic distributions of the LSEs are established, which are involved with the stable distribution. The convergence rate of the LSE depends on 1−ρn2, but no condition on the rate of ρn is required. It is shown that, for the tail index α∈(0,2), the LSE is inconsistent, for α=2, logn/(1−ρn2)-consistent, and for α∈(2,4), n1−2/α/(1−ρn2)-consistent. Proofs are based on the point process and the asymptotic properties in AR models with G-GARCH errors. However, this present work provides a bridge between pure stationary and unit-root processes. This paper extends the existing uniform limit theory with three issues: the errors have conditional heteroscedastic variance; the errors are heavy-tailed with tail index α∈(0,4); and no restriction on the rate of ρn is necessary.

Chaotic systems with asymmetric heavy-tailed noise: Application to 3D attractors

Abstract Yilmaz et al. (Fluct. Noise Lett. 17, 1830002, 2018) investigated the stochastic phenomenological bifurcations of a generalized Chua circuit driven by Skew-Gaussian distributed noise. They proved it is possible to decrease the number of scrolls by properly choosing the stochastic excitation, manipulating the skewness and noise intensity parameters. Based on the latter, this paper proposes an extension of skew-gaussian noise based on the family of Scale Mixtures of Skew-normal (SMSN) distributions, which includes the skew- t , the skew-gaussian, and the gaussian noises as particular cases. The Lorenz, Generalized Lorenz, Proto–Lorenzand Rossler attractors driven by skew- t distributed noise are considered. Results show that the chaotic regime’s behavior is influenced by the freedom parameter degrees of skew- t noise, increasing the noise variance. This paper concludes that noise intensity increases by rescaling the skew- t distribution at zero mean, rather than by increasing the asymmetry parameter.

Dependency measures for the diagnosis of local faults in application to the heavy-tailed vibration signal

The extraction of the informative frequency band from the signal with heavy-tailed noise is introduced to detect local damage. The algorithm for the vibration signal from a bearing installed in the ore crusher is proposed. From a signal processing point of view, this is related to the detection and recognition of cyclic and non-cyclic impulsive components in the signal. It is assumed that the non-periodic impulses have amplitudes significantly higher than the cyclic impulses – usually fully hidden in the non-Gaussian noise - which makes the diagnosis difficult. As both components are non-Gaussian, a periodicity detector can be used as an alternative to the impulsiveness criteria. In this paper, the proposed informative frequency band selectors utilize the dependency measures, i.e., the Pearson, Spearman and Kendall correlations. These dependency measures are applied to the time–frequency domain representation of the signal to find similarities between sub-signals associated with frequency bands. Three types of frequency bands are expected – the first one – with just Gaussian noise, the second one – with cyclic impulsive behavior, and the last one - with non-cyclic impulsive behavior. For simplicity, it is assumed that frequency bands occupied by cyclic and non-cyclic impulses are different. Finally, one may obtain a map of similarities between all frequency bands, and based on this map the 1D selector for informative frequency band identification can be estimated. The main contribution of the paper is the introduction of the holistic procedure of the fault diagnosis containing the time–frequency representation of vibration signal, including the estimation of the dependency map (D-map) using three different dependency measures, D-map de-noising, and the integration of D-map to one-dimensional (1D) selector. The introduced procedure is applied to the complex simulated data and then to real vibrations from the heavy-duty machine.

A novel robust Kalman filter with unknown non-stationary heavy-tailed noise

In this article, the state estimation with unknown non-stationary heavy-tailed process and measurement noises (HPMN) is considered. The measurement likelihood and the one-step state prediction are modelled as the mixture of two Gaussian (M2G) distributions, respectively. In the proposed M2G, one is a high probability Gaussian distribution with a nominal covariance and the other is a low probability Gaussian distribution with an adaptive large covariance. The adaptive large covariance is formulated as the nominal covariance multiplied by a time varying scale parameter. An indicator is introduced to identify whether it is generated by the nominal covariance or the larger covariance. The unknown nominal covariance is modelled as an inverse Wishart distribution. Using hierarchical priors on the indicator and scale parameter, a robust model is formulated and a variational Bayesian algorithm is developed. Experiments with synthetic data and real intelligent vehicle data show the effectiveness of the proposed filter under unknown non-stationary HPMN.

DOA estimation based on fractional low-order multi-sensor time-frequency analysis in heavy tailed noise

In order to improve the performance of Direction-of-arrival (DOA) estimation in heavy-tailed noise, a novel DOA estimation method based on fractional low-order multi-sensor time-frequency distribution is proposed. We first use the high-resolution of the fractional low-order multi-sensor time-frequency analysis (FLOM-MTFA) to obtain the time-frequency matrices of the array signal. Then, the high-energy points of the fractional low-order multi-sensor time-frequency matrices is selected to improve the MSNR. We finally compute the averaged FLOM-MTFA matrix and exploit estimation signal parameter via rotational invariance techniques to estimate DOA. Theoretical analysis and simulation results show that the proposed method can effectively estimate DOA in heavy tailed noise.

Robust M-Estimation Based Bayesian Cluster Enumeration for Real Elliptically Symmetric Distributions

Robustly determining the optimal number of clusters in a data set is an essential factor in a wide range of applications. Cluster enumeration becomes challenging when the true underlying structure in the observed data is corrupted by heavy-tailed noise and outliers. Recently, Bayesian cluster enumeration criteria have been derived by formulating cluster enumeration as maximization of the posterior probability of candidate models. This article generalizes robust Bayesian cluster enumeration so that it can be used with any arbitrary Real Elliptically Symmetric (RES) distributed mixture model. Our framework also covers the case of M-estimators that allow for mixture models, which are decoupled from a specific probability distribution. Examples of Huber's and Tukey's M-estimators are discussed. We derive a robust criterion for data sets with finite sample size, and also provide an asymptotic approximation to reduce the computational cost at large sample sizes. The algorithms are applied to simulated and real-world data sets, including radar-based person identification, and show a significant robustness improvement in comparison to existing methods.