Heavy-tailed Noise Research Articles

Stochastic PDEs with heavy-tailed noise

We analyze the nonlinear stochastic heat equation driven by heavy-tailed noise on unbounded domains and in arbitrary dimension. The existence of a solution is proved even if the noise only has moments up to an order strictly smaller than its Blumenthal–Getoor index. In particular, this includes all stable noises with index α<1+2/d. Although we cannot show uniqueness, the constructed solution is natural in the sense that it is the limit of the solutions to approximative equations obtained by truncating the big jumps of the noise or by restricting its support to a compact set in space. Under growth conditions on the nonlinear term we can further derive moment estimates of the solution, uniformly in space. Finally, the techniques are shown to apply to Volterra equations with kernels bounded by generalized Gaussian densities. This includes, for instance, a large class of uniformly parabolic stochastic PDEs.

Fitting Generalized Multivariate Huber Loss Functions

In this letter, we consider a class of generalized multivariate Huber (GMH) loss functions. Our goal is parameter estimation in linear models contaminated by non-Gaussian noise. We assume access to a secondary dataset of independent noise realizations, and we use these data to fit a convex GMH function that will then lead to efficient parameter estimation. Our framework includes the classical weighted least squares and Huber's function as special cases. We demonstrate its advantages in heavy-tailed noise distributions.

PDF approximations to estimation and detection in time-correlated alpha-stable channels

This paper considers the estimation and detection problems for statistically dependent heavy-tailed signals with no closed-form probability density function (PDF). We propose two parametric PDF approximations for symmetric α-stable (SαS) distribution to be utilized in approaches based on the Maximum likelihood (ML) criterion. The nonlinear least square (LS) and curve fitting are used to compute parameters of the new formulations which are functions of the characteristic exponent. Moreover, we study binary signal detection in channels with time-dependent heavy-tailed noise modeled by SαS distribution and first order autoregressive (AR(1)) process. Using the novel PDF approximations in the ML estimator, an algorithm for model parameters estimation of the noise is initially developed. Then, new suboptimal receivers are designed through the use of the new PDF formulations and parameter estimates. Numerical results demonstrate the superiority of the proposed approximations over the existing formulations, and also good accuracy for the estimation algorithm. Additionally, it is shown that the proposed detectors operate near optimal receiver and also outperform the other suboptimal detectors, especially when α is small.

Inexact Alternating Optimization for Phase Retrieval in the Presence of Outliers

Phase retrieval has been mainly considered in the presence of Gaussian noise. However, the performance of the algorithms proposed under the Gaussian noise model severely degrades when grossly corrupted data, i.e., outliers, exist. This paper investigates techniques for phase retrieval in the presence of heavy-tailed noise -- which is considered a better model for situations where outliers exist. An $\ell_p$-norm ($0<p<2$) based estimator is proposed for fending against such noise, and two-block inexact alternating optimization is proposed as the algorithmic framework to tackle the resulting optimization problem. Two specific algorithms are devised by exploring different local approximations within this framework. Interestingly, the core conditional minimization steps can be interpreted as iteratively reweighted least squares and gradient descent. Convergence properties of the algorithms are discussed, and the Cram\'er-Rao bound (CRB) is derived. Simulations demonstrate that the proposed algorithms approach the CRB and outperform state-of-the-art algorithms in heavy-tailed noise.

Comparison of Bistable Systems and Matched Filters in Non-Gaussian Noise

In this paper, we report that for a weak signal buried in the heavy-tailed noise, the bistable system can outperform the matched filter, yielding a higher output signal-to-noise ratio (SNR) or a lower probability of error. Moreover, by adding mutually independent internal noise components to an array of bistable systems, the output SNR or the probability of error can be further improved via the mechanism of stochastic resonance (SR). These comparison results demonstrate the potential capability of bistable systems for detecting weak signals in non-Gaussian noise environments.

Non-Parametric Estimation of Intraday Spot Volatility: Disentangling Instantaneous Trend and Seasonality

We provide a new framework for modeling trends and periodic patterns in high-frequency financial data. Seeking adaptivity to ever-changing market conditions, we enlarge the Fourier flexible form into a richer functional class: both our smooth trend and the seasonality are non-parametrically time-varying and evolve in real time. We provide the associated estimators and use simulations to show that they behave adequately in the presence of jumps and heteroskedastic and heavy-tailed noise. A study of exchange rate returns sampled from 2010 to 2013 suggests that failing to factor in the seasonality’s dynamic properties may lead to misestimation of the intraday spot volatility.

Extending multivariate- t linear mixed models for multiple longitudinal data with censored responses and heavy tails.

The analysis of complex longitudinal data is challenging due to several inherent features: (i) more than one series of responses are repeatedly collected on each subject at irregularly occasions over a period of time; (ii) censorship due to limits of quantification of responses arises left- and/or right- censoring effects; (iii) outliers or heavy-tailed noises are possibly embodied within multiple response variables. This article formulates the multivariate- t linear mixed model with censored responses (MtLMMC), which allows the analysts to model such data in the presence of the above described features simultaneously. An efficient expectation conditional maximization either (ECME) algorithm is developed to carry out maximum likelihood estimation of model parameters. The implementation of the E-step relies on the mean and covariance matrix of truncated multivariate- t distributions. To enhance the computational efficiency, two auxiliary permutation matrices are incorporated into the procedure to determine the observed and censored parts of each subject. The proposed methodology is demonstrated via a simulation study and a real application on HIV/AIDS data.

An Analog–Digital Hardware for L-Estimate Space-Varying Image Filtering

An analog---digital hardware solution for implementation of the L-estimate space-varying filtering has been proposed. The considered filter form is based on the robust space/spatial-frequency representation and provides efficient denoising of two-dimensional signals/images corrupted by heavy-tailed noise. Moreover, for images with fast-varying details and textures, the L-estimate filtering outperforms the commonly used filters. However, it requires significant processing time, since the space/spatial-frequency representation is calculated for each pixel, on a window by window basis. Therefore, in order to make it feasible for practical applications, a fast implementation of L-estimate space-varying filtering is proposed using a combined analog---digital approach. It provides efficient real-time processing of images corrupted by strong mixed Gaussian and impulse noise.

Robust Gradient Learning With Applications.

This paper addresses the robust gradient learning (RGL) problem. Gradient learning models aim at learning the gradient vector of some target functions in supervised learning problems, which can be further used to applications, such as variable selection, coordinate covariance estimation, and supervised dimension reduction. However, existing GL models are not robust to outliers or heavy-tailed noise. This paper provides an RGL framework to address this problem in both regression and classification. This is achieved by introducing a robust regression loss function and proposing a robust classification loss. Moreover, our RGL algorithm works in an instance-based kernelized dictionary instead of some fixed reproducing kernel Hilbert space, which may provide more flexibility. To solve the proposed nonconvex model, a simple computational algorithm based on gradient descent is provided and the convergence of the proposed method is also analyzed. We then apply the proposed RGL model to applications, such as nonlinear variable selection and coordinate covariance estimation. The efficiency of our proposed model is verified on both synthetic and real data sets.

LADE-Based Inference for ARMA Models With Unspecified and Heavy-Tailed Heteroscedastic Noises

This article develops a systematic procedure of statistical inference for the auto-regressive moving average (ARMA) model with unspecified and heavy-tailed heteroscedastic noises. We first investigate the least absolute deviation estimator (LADE) and the self-weighted LADE for the model. Both estimators are shown to be strongly consistent and asymptotically normal when the noise has a finite variance and infinite variance, respectively. The rates of convergence of the LADE and the self-weighted LADE are n− 1/2, which is faster than those of least-square estimator (LSE) for the ARMA model when the tail index of generalized auto-regressive conditional heteroskedasticity (GARCH) noises is in (0, 4], and thus they are more efficient in this case. Since their asymptotic covariance matrices cannot be estimated directly from the sample, we develop the random weighting approach for statistical inference under this nonstandard case. We further propose a novel sign-based portmanteau test for model adequacy. Simulation study is carried out to assess the performance of our procedure and one real illustrating example is given. Supplementary materials for this article are available online.

Robust Multitask Multiview Tracking in Videos.

Various sparse-representation-based methods have been proposed to solve tracking problems, and most of them employ least squares (LSs) criteria to learn the sparse representation. In many tracking scenarios, traditional LS-based methods may not perform well owing to the presence of heavy-tailed noise. In this paper, we present a tracking approach using an approximate least absolute deviation (LAD)-based multitask multiview sparse learning method to enjoy robustness of LAD and take advantage of multiple types of visual features, such as intensity, color, and texture. The proposed method is integrated in a particle filter framework, where learning the sparse representation for each view of the single particle is regarded as an individual task. The underlying relationship between tasks across different views and different particles is jointly exploited in a unified robust multitask formulation based on LAD. In addition, to capture the frequently emerging outlier tasks, we decompose the representation matrix to two collaborative components that enable a more robust and accurate approximation. We show that the proposed formulation can be effectively approximated by Nesterov's smoothing method and efficiently solved using the accelerated proximal gradient method. The presented tracker is implemented using four types of features and is tested on numerous synthetic sequences and real-world video sequences, including the CVPR2013 tracking benchmark and ALOV++ data set. Both the qualitative and quantitative results demonstrate the superior performance of the proposed approach compared with several state-of-the-art trackers.

Robust sequential learning of feedforward neural networks in the presence of heavy-tailed noise

Feedforward neural networks (FFNN) are among the most used neural networks for modeling of various nonlinear problems in engineering. In sequential and especially real time processing all neural networks models fail when faced with outliers. Outliers are found across a wide range of engineering problems. Recent research results in the field have shown that to avoid overfitting or divergence of the model, new approach is needed especially if FFNN is to run sequentially or in real time. To accommodate limitations of FFNN when training data contains a certain number of outliers, this paper presents new learning algorithm based on improvement of conventional extended Kalman filter (EKF). Extended Kalman filter robust to outliers (EKF-OR) is probabilistic generative model in which measurement noise covariance is not constant; the sequence of noise measurement covariance is modeled as stochastic process over the set of symmetric positive-definite matrices in which prior is modeled as inverse Wishart distribution. In each iteration EKF-OR simultaneously estimates noise estimates and current best estimate of FFNN parameters. Bayesian framework enables one to mathematically derive expressions, while analytical intractability of the Bayes’ update step is solved by using structured variational approximation. All mathematical expressions in the paper are derived using the first principles. Extensive experimental study shows that FFNN trained with developed learning algorithm, achieves low prediction error and good generalization quality regardless of outliers’ presence in training data.

Robust low-rank tensor factorization by cyclic weighted median

Low-rank tensor factorization (LRTF) provides a useful mathematical tool to reveal and analyze multi-factor structures underlying data in a wide range of practical applications. One challenging issue in LRTF is how to recover a low-rank higher-order representation of the given high dimensional data in the presence of outliers and missing entries, i.e., the so-called robust LRTF problem. The L 1-norm LRTF is a popular strategy for robust LRTF due to its intrinsic robustness to heavy-tailed noises and outliers. However, few L 1-norm LRTF algorithms have been developed due to its non-convexity and non-smoothness, as well as the high order structure of data. In this paper we propose a novel cyclic weighted median (CWM) method to solve the L 1-norm LRTF problem. The main idea is to recursively optimize each coordinate involved in the L 1-norm LRTF problem with all the others fixed. Each of these single-scalar-parameter sub-problems is convex and can be easily solved by weighted median filter, and thus an effective algorithm can be readily constructed to tackle the original complex problem. Our extensive experiments on synthetic data and real face data demonstrate that the proposed method performs more robust than previous methods in the presence of outliers and/or missing entries.

ℓ1 Major Component Detection and Analysis (ℓ1 MCDA) in Three and Higher Dimensional Spaces

Based on the recent development of two dimensional ℓ1 major component detection and analysis (ℓ1 MCDA), we develop a scalable ℓ1 MCDA in the n-dimensional space to identify the major directions of star-shaped heavy-tailed statistical distributions with irregularly positioned “spokes” and “clutters”. In order to achieve robustness and efficiency, the proposed ℓ1 MCDA in n-dimensional space adopts a two-level median fit process in a local neighbor of a given direction in each iteration. Computational results indicate that in terms of accuracy ℓ1 MCDA is competitive with two well-known PCAs when there is only one major direction in the data, and ℓ1 MCDA can further determine multiple major directions of the n-dimensional data from superimposed Gaussians or heavy-tailed distributions without and with patterned artificial outliers. With the ability to recover complex spoke structures with heavy-tailed noise and clutter in the data, ℓ1 MCDA has potential to generate better semantics than other methods.

On Hilbert transform methods for low frequency oscillations detection

This study tackles the issue of electromechanical modes identification through a measurement-based methodology employing a novel signal decomposition theorem based upon the Hilbert transform. The methodology aims to answer in a simpler and more pragmatic manner to the main weaknesses of the Hilbert-Huang transform with respect to the major refinements in the relevant literature. These weak points are discussed with sufficient detailed degree in the study. The main contribution of this study consists in combining a recent signal decomposition theorem for separating an assigned signal into elemental ones, each of them characterised by a single frequency component and a robust preliminary non-linear spectral analyser, named L p periodogram. This procedure's results are very appropriate for analysing some critical cases of electromechanical oscillations, because of the L p periodogram robustness against heavy-tailed noise and also its intrinsic ability in estimating closely spaced frequency components. The proposed approach is found to be inherently simple, reliable and consistent in performance as well as characterised by low computational burden. Some numerical applications validate the methodology and assess its own performance on synthetic signals, near real-life signals acquired by IEEE test networks and on a real measured signal from a wide-area monitoring system currently in operation.

Logical stochastic resonance in bistable system under α-stable noise

In the presence of α-stable noise, the logical stochastic resonance (LSR) phenomenon in a class of double well nonlinear system is investigated in this paper. LSR effect is obtained under α-stable noise. The probability of getting correct logic outputs is used to evaluate LSR behavior. Four main results are presented. Firstly, in the optimal band of noise intensity, Gaussian white noise is considered a better choice than heavy tailed noise to obtain clean logic operation. But at weak noise background, the success probability of getting the right logic outputs is higher when the system is subjected to heavy tailed noise. Secondly, it is shown that over the entire range of noise variance, the asymmetric noise induced LSR performs better than that induced by the symmetric noise. Furthermore, we find which side the tail skews also affects the correct probability of LSR. At last, the fractional Fokker-Planck equation is presented to show when the characteristic exponent of α-stable noise is less than 1, LSR behavior will not be obtained irrespective of the setting for other parameters.

Spatial Sign and Rank Cyclic Detectors

This letter proposes robust nonparametric cyclic detectors based on the spatial sign and magnitude rank concepts. The proposed detectors are single-cycle detectors exploiting the known (conjugate) cyclic autocorrelation of the signal of interest. Cyclostationarity allows distinguishing among different signals and emitters that exhibit distinct cyclostationary properties. The asymptotic distributions of the proposed cyclic detectors are established under the null hypothesis when only noise is present. Simulation results demonstrate the robust performance of the proposed detection algorithms in both Gaussian and heavy-tailed non-Gaussian noise environments.

Non-Parametric Estimation of Intraday Spot Volatility: Disentangling Instantaneous Trend and Seasonality

We provide a new framework for modeling trends and periodic patterns in high-frequency financial data. Seeking adaptivity to ever-changing market conditions, we enlarge the Fourier flexible form into a richer functional class: both our smooth trend and the seasonality are non-parametrically time-varying and evolve in real time. We provide the associated estimators and use simulations to show that they behave adequately in the presence of jumps and heteroskedastic and heavy-tailed noise. A study of exchange rate returns sampled from 2010 to 2013 suggests that failing to factor in the seasonality's dynamic properties may lead to misestimation of the intraday spot volatility.

M-estimator-based robust estimation of the number of components of a superimposed sinusoidal signal model

In this paper, we consider the problem of estimating the number of components of a superimposed nonlinear sinusoids model of a signal in the presence of additive noise. We propose and provide a detailed empirical comparison of robust methods for estimation of the number of components. The proposed methods, which are robust modifications of the commonly used information theoretic criteria, are based on various M-estimator approaches and are robust with respect to outliers present in the data and heavy-tailed noise. The proposed methods are compared with the usual non-robust methods through extensive simulations under varied model scenarios. We also present real signal analysis of two speech signals to show the usefulness of the proposed methodology.

Signal Detection in Generalized Gaussian Noise by Nonlinear Wavelet Denoising

In this paper, a nonlinear suboptimal detector whose performance in heavy-tailed noise is significantly better than that of the matched filter is proposed. The detector consists of a nonlinear wavelet denoising filter to enhance the signal-to-noise ratio, followed by a replica correlator. Performance of the detector is investigated through an asymptotic theoretical analysis as well as Monte Carlo simulations. The proposed detector offers the following advantages over the optimal (in the Neyman-Pearson sense) detector: it is easier to implement, and it is more robust with respect to error in modeling the probability distribution of noise.