Sparse Outliers Research Articles

Robust low-rank matrix completion via sparsity-inducing regularizer

This paper proposes a sparsity-inducing regularizer associated with the Welsch function. We theoretically show that the regularizer is quasiconvex and the corresponding Moreau envelope is convex. Moreover, the closed-form solution to its Moreau envelope, namely, the proximity operator, is derived. Unlike conventional nonconvex regularizers like the ℓp-norm with 0<p<1 that generally needs iterations to obtain the corresponding proximity operator, the developed regularizer has a closed-form proximity operator. We utilize our regularizer to penalize the singular values as well as sparse outliers of the distorted data, and develop an efficient algorithm for robust matrix completion. Convergence of the suggested method is analyzed and we prove that any accumulation point is a stationary point. Finally, experimental results demonstrate that our algorithm is superior to the competing techniques in terms of restoration performance. MATALB codes are available at https://github.com/bestzywang/RMC-NNSR.

Robust sparse representation based on fitting error decomposition

Sparse representation (SR) of a signal aims at finding the minimum number of atoms for its representation. In several practical scenarios, the signal is vulnerable to outliers and thus robustness is required for SR based algorithms. However, most existing robust schemes are designed on the assumption that the anomalies are independently distributed, which may not perform well encountering complex interferences, especially the correlated ones. To deal with this drawback, a robust SR model is proposed, where both independent and correlated outliers are considered. Specifically, the fitting error is decomposed as the combination of a low-rank component and a sparse part, corresponding to the correlated gross error and independent outlier, respectively. Then, the group sparsity of the representation coefficient is utilized. Moreover, ℓ0-norm and ℓ2,0-norm are adopted as the sparsity regularization for the sparse outlier and representation coefficients, respectively. The solutions to ℓ0/ℓ2,0-norm minimization are generated by the hard-thresholding strategy, where the decision threshold is adaptively determined using median absolute deviation operator. As for the low-rank regularization, due to the NP-hardness of rank minimization, we employ matrix logarithmic norm as the rank surrogate to lessen the approximation gap. Finally, we apply the proposed model to face recognition task, and the excellent performance demonstrates its effectiveness.

Joint magnetic resonance imaging artifacts and noise reduction on discrete shape space of images

Magnetic resonance (MR) images can be corrupted by artifacts and noise, potentially leading to misinterpretation of the images. In this paper, we propose a novel approach based on the discrete shape space of images (DSSI) to jointly reduce artifacts and noise in MR images. The proposed method restores MR images in multiple domains based on the distinct generation mechanisms of noise and artifacts. The images in multiple domains are analyzed in a non-Euclidean space. The DSSI is constructed as a Riemannian manifold to measure the intrinsic properties of images. Images are considered shapes from a geometric perspective, and the impact of similarity transformations (e.g., rotation, scaling, and translation) on image analysis is eliminated. The patch-based rank-ordered difference (PROD) detector is defined in k-space within the framework of DSSI to detect and remove sparse outliers that cause artifacts. In addition, a novel similarity function for images is defined using the DSSI and be used to design the improved filter. Finally, the convergence of the improved filter is theoretically analyzed, indicating that our method offers an effective estimator of the ideal image. The experimental results of various MR images demonstrate that the proposed approach outperforms classical and state-of-the-art methods for artifact correction and noise removal, both qualitatively and quantitatively.

ERPCA: Robust Principal Component Analysis for Exponential Family Distributions

AbstractRobust principal component analysis (RPCA) is a widely used method for recovering low‐rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious tampering, or other causes for anomalies, and the joint identification of such corruptions with low‐rank background is critical for process monitoring and diagnosis. However, existing RPCA methods and their extensions largely do not account for the underlying probabilistic distribution for the data matrices, which in many applications are known and can be highly non‐Gaussian. We thus propose a new method called RPCA for exponential family distributions (), which can perform the desired decomposition into low‐rank and sparse matrices when such a distribution falls within the exponential family. We present a novel alternating direction method of multiplier optimization algorithm for efficient decomposition, under either its natural or canonical parametrization. The effectiveness of is then demonstrated in two applications: the first for steel sheet defect detection and the second for crime activity monitoring in the Atlanta metropolitan area.

Towards Multi-Mode Outlier Robust Tensor Ring Decomposition

Conventional Outlier Robust Tensor Decomposition (ORTD) approaches generally represent sparse outlier corruption within a specific mode. However, such an assumption, which may hold for matrices, proves inadequate when applied to high-order tensors. In the tensor domain, the outliers are prone to be corrupted in multiple modes simultaneously. Addressing this limitation, this study proposes a novel ORTD approach by recovering low-rank tensors contaminated by outliers spanning multiple modes. In particular, we conceptualize outliers within high-order tensors as latent tensor group sparsity by decomposing the corrupted tensor into a sum of multiple latent components, where each latent component is exclusive to outliers within a particular direction. Thus, it can effectively mitigate the outlier corruptions prevalent in high-order tensors across multiple modes. To theoretically guarantee recovery performance, we rigorously analyze a non-asymptotic upper bound of the estimation error for the proposed ORTD approach. In the optimization process, we develop an efficient alternate direction method of multipliers (ADMM) algorithm. Empirical validation of the approach's efficacy is undertaken through comprehensive experimentation.

Probability-weighted tensor robust PCA with CP decomposition for hyperspectral image restoration

This paper presents a novel probability-weighted tensor robust principal component analysis (TRPCA) method based on CANDECOMP/PARAFAC decomposition (CPD) for hyperspectral image (HSI) restoration with low computational complexity from the coexistence of dense noise and sparse outliers. In the proposed method, via the CPD property, the tensor nuclear norm (TNN) optimization object is replaced as the CPD factor matrix of HSI with lower dimensions without losing low-rank property, so as to the computational complexity could be reduced during optimizing TNN in TRPCA model. To demonstrate the different effects of noise and outliers on HSI restoration, two weighted sets are defined with probability by the prior knowledge of the occurrence of outliers, which are used to analyze the probability whether the HSI is corrupted by dense noise or outliers. Via the probability theory, information entropy is employed to measure the uncertainty of noise and outlier occurrence in received data, so that to demonstrate their difference impacts on HSI restoration to improve the recovery accuracy. Theoretical analysis and simulation results show that the proposed method has lower computational complexity and better restoration performance compared with other related methods.

Online Robust Subspace Clustering With Application to Power Grid Monitoring

In this work, a robust subspace clustering algorithm is developed to exploit the inherent union-of-subspaces structure in the data for reconstructing missing measurements and detecting anomalies. Our focus is on processing an incessant stream of large-scale data such as synchronized phasor measurements in the power grid, which is challenging due to computational complexity, memory requirement, and missing and corrupt observations. In order to mitigate these issues, a low-rank representation (LRR) model-based subspace clustering problem is formulated that can handle missing measurements and sparse outliers in the data. Then, an efficient online algorithm is derived based on stochastic approximation. The convergence property of the algorithm is established. Strategies to maintain a representative yet compact dictionary for capturing the subspace structure are also proposed. The developed method is tested on both simulated and real phasor measurement unit (PMU) data to verify the effectiveness and is shown to significantly outperform existing algorithms based on simple low-rank structure of data.

Tensor-Based Sparse Bayesian Learning With Intra-Dimension Correlation

Unlike the flat-view matrix, tensor provides an elegant representation to preserve the structure of data in the multidimensional nature. The wide use of tensor-based approaches highlights the limitation of matrix-based methods and has generated increasing needs in multidimensional data analytics. Driven by such demands, tensor decomposition with automatic rank determination has recently emerged as a promising tool for image processing, wireless communications and neural network compression. However, unlike the success in matrix-based schemes of exploring the correlation for performance improvement, existing tensor-based methods fail to consider the correlation structure in the tensor. In this paper, we propose to exploit the tensor intra-dimension correlation within the framework of sparse Bayesian learning. We provide the sparsity-inducing probabilistic models under both canonical polyadic decomposition and Tucker decomposition to capture the correlation structure effectively. We derive an efficient model inference method with a fast convergence rate based on the expectation-maximization algorithm. Furthermore, a complexity reduction method is proposed to speed up the computation by exploiting several properties of the Kronecker product. We further extend the proposed framework for the incomplete tensor corrupted by sparse outliers to achieve effective tensor completion and outlier mitigation. Simulation results demonstrate the superior performance of the proposed framework over those ignoring the intra-dimension correlation in terms of accuracy and sparsity.

Sparse Affine Sampling: Ambiguity-Free and Efficient Sparse Phase Retrieval

Conventional sparse phase retrieval schemes can recover sparse signals from the magnitude of linear measurements only up to a global phase ambiguity. This work proposes a novel approach that instead utilizes the magnitude of affine measurements to achieve ambiguity-free signal reconstruction. The proposed method relies on two-stage approach that consists of support identification followed by the exact recovery of nonzero signal entries. In the noise-free case, perfect support identification using a simple counting rule is guaranteed subject to a mild condition on the signal sparsity, and subsequent exact recovery of the nonzero signal entries can be obtained in closed-form. The proposed approach is then extended to two noisy scenarios, namely, sparse noise (or outliers) and non-sparse bounded noise. For both cases, perfect support identification is still ensured under mild conditions on the noise model, namely, the support size for sparse outliers and the power of the bounded noise. Under perfect support identification, exact signal recovery can be achieved using a simple majority rule for the sparse noise scenario, and reconstruction up to a bounded error can be achieved using linear least-squares (LS) estimation for the non-sparse bounded noise scenario. The obtained analytic performance guarantee for the latter case also sheds light on the construction of the sensing matrix and bias vector. In fact, we show that a near optimal performance can be achieved with high probability by the random generation of the nonzero entries of the sparse sensing matrix and bias vector according to the uniform distribution over a circle. Computer simulations using both synthetic and real-world data sets are provided to demonstrate the effectiveness of the proposed scheme.

Abnormal data cleaning of wind turbine power curve using Bayesian change point-quartile combined algorithm

During the operation of wind turbines, there are a lot of abnormal data collected from wind power data, which cannot be used to truly evaluate the performance and operation state of wind turbines. According to the distribution characteristics of these abnormal data, an outlier delete method using Bayesian change point-quartile combined algorithm is proposed. Firstly, to identify and delete the abnormal data below wind power curve, based on the Bayesian theorem, change point position is identified. Finally, the quartile method is utilized to delete the upper sparse outliers. The proposed method is applied to 10-min field monitoring wind power data, root mean square error (RMSE), program running time and the rejection rate of field wind turbine data are used to measure model accuracy, the results show that compared with the least squares (LS) change point method, the method proposed in this paper is feasible and effective.

A general theory for subspace-sparse recovery

High-dimensional data that often lie in low-dimensional subspaces are ubiquitous in many fields of signal and image processing, pattern recognition, machine learning, etc. Finding sparse representations of data points in a dictionary built by using the collection of data helps to study low-dimensional subspaces. In this paper, we consider the problem of subspace-sparse representation for data composed of two distinct features. Applying different measures to the coefficients of the two distinct features under different dictionaries makes the model used in this paper more general. We present theoretical guarantee for subspace-sparse representation under conditions on the subspaces and data. The program used in this paper can handle data with structured corruption, missing entries, sparse outliers and random bounded noise well.

Outlier Detection for Process Monitoring in Industrial Cyber-Physical Systems

The development of industrial cyber-physical system (ICPS) provides a tight connection between the digital model and the industrial physical plant, which enables to use the data-driven methods to reflect the state of process running in the real world. However, due to the noisy and harsh industrial environment, the collected data are often corrupted to some extent. If the corrupted data are not detected in time, the data-driven model will inevitably degenerate and induce a poor process monitoring performance. In addition, the nonlinear characteristics between process variables due to the high complexities in physical plant bring challenges to the data-driven methods. In this article, a robust kernel dictionary learning method, which can overcome the negative influence of outliers and simultaneously extracts the nonlinear characteristics of industrial process, is proposed to address the above problems in ICPS. Our extensive experiments demonstrate that the proposed method has achieved significantly better and stable performance to deal with outlier detection and process monitoring in ICPS. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —In order to mitigate the impacts due to process noise and outliers, a robust kernel dictionary learning method is proposed to improve the accuracy and stability of the process monitoring of industrial cyber-physical systems. This method considers the process noise, sparse outlier, as well as the nonlinear characteristic of industrial systems for improving the accuracy and stability of monitoring. Compared with many state-of-the-art methods, the proposed method can detect the outliers in the training dataset adaptively, which is more applicable to the real industrial system.

DEMO: A Flexible Deartifacting Module for Compressed Sensing MRI

Compressed sensing (CS) has been a novel technique for fast reconstruction of magnetic resonance (MR) images from their partial <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-space measurements. However, the quality of the reconstructed images could be severely affected by artifacts from various sources. In this paper, we propose a deartifacting module (DEMO) that can effectively remove the artifacts by eliminating sparse outliers in the <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-space. Specifically, DEMO augments the measurements in the original loss function to approximate a new loss that is robust to outliers. Since DEMO is developed independently of any backbone algorithm to perform with, it can be flexibly incorporated into a broad range of CS-MRI methods, including both model-based methods and unrolling deep neural networks. Extensive experiments under various settings demonstrate the effectiveness and robustness of DEMO.

Low-Rank Plus Sparse Decomposition of fMRI Data With Application to Alzheimer's Disease.

Studying functional brain connectivity plays an important role in understanding how human brain functions and neuropsychological diseases such as autism, attention-deficit hyperactivity disorder, and Alzheimer's disease (AD). Functional magnetic resonance imaging (fMRI) is one of the most popularly used tool to construct functional brain connectivity. However, the presence of noises and outliers in fMRI blood oxygen level dependent (BOLD) signals might lead to unreliable and unstable results in the construction of connectivity matrix. In this paper, we propose a pipeline that enables us to estimate robust and stable connectivity matrix, which increases the detectability of group differences. In particular, a low-rank plus sparse (L + S) matrix decomposition technique is adopted to decompose the original signals, where the low-rank matrix L recovers the essential common features from regions of interest, and the sparse matrix S catches the sparse individual variability and potential outliers. On the basis of decomposed signals, we construct connectivity matrix using the proposed novel concentration inequality-based sparse estimator. In order to facilitate the comparisons, we also consider correlation, partial correlation, and graphical Lasso-based methods. Hypothesis testing is then conducted to detect group differences. The proposed pipeline is applied to rs-fMRI data in Alzheimer's disease neuroimaging initiative to detect AD-related biomarkers, and we show that the proposed pipeline provides accurate yet more stable results than using the original BOLD signals.

Stable Recovery and Separation of Structured Signals by Convex Geometry

We consider the ill-posed problem of identifying the signal that is structured in some general dictionary (i.e., possibly redundant or over-complete matrix) from corrupted measurements, where the corruption is structured in another general dictionary. We formulate the problem by applying appropriate convex constraints to the signal and corruption according to their structures and provide conditions for exact recovery from structured corruption and stable recovery from structured corruption with added stochastic bounded noise. In addition, this paper provides estimates of the number of the measurements needed for recovery. These estimates are based on computing the Gaussian complexity of a tangent cone and the Gaussian distance to a subdifferential. Applications covered by the proposed programs include the recovery of signals that is disturbed by impulse noise, missing entries, sparse outliers, random bounded noise, and signal separation. Numerical simulation results are presented to verify and complement the theoretical results.

Robust Outlier Bound Condition to Phase Retrieval with Adversarial Sparse Outliers

Robust Outlier Bound Condition to Phase Retrieval with Adversarial Sparse Outliers

Robust Tensor Tracking With Missing Data and Outliers: Novel Adaptive CP Decomposition and Convergence Analysis

Canonical Polyadic (CP) decomposition is a powerful multilinear algebra tool for analyzing multiway (a.k.a. tensor) data and has been used for various signal processing and machine learning applications. When the underlying tensor is derived from data streams, adaptive CP decomposition is required. In this paper, we propose a novel method called robust adaptive CP decomposition (RACP) for dealing with high-order incomplete streaming tensors that are corrupted by outliers. At each time instant, RACP first performs online outlier rejection to accurately detect and remove sparse outliers, and then performs tensor factor tracking to efficiently update the tensor basis. A unified convergence analysis of RACP is also established in that the sequence of generated solutions converges asymptotically to a stationary point of the objective function. Extensive experiments were conducted on both synthetic and real data to demonstrate the effectiveness of RACP in comparison with state-of-the-art adaptive CP algorithms.

BRIDGING CONVEX AND NONCONVEX OPTIMIZATION IN ROBUST PCA: NOISE, OUTLIERS, AND MISSING DATA.

This paper delivers improved theoretical guarantees for the convex programming approach in low-rank matrix estimation, in the presence of (1) random noise, (2) gross sparse outliers, and (3) missing data. This problem, often dubbed as robust principal component analysis (robust PCA), finds applications in various domains. Despite the wide applicability of convex relaxation, the available statistical support (particularly the stability analysis vis-à-vis random noise) remains highly suboptimal, which we strengthen in this paper. When the unknown matrix is well-conditioned, incoherent, and of constant rank, we demonstrate that a principled convex program achieves near-optimal statistical accuracy, in terms of both the Euclidean loss and the ℓ ∞ loss. All of this happens even when nearly a constant fraction of observations are corrupted by outliers with arbitrary magnitudes. The key analysis idea lies in bridging the convex program in use and an auxiliary nonconvex optimization algorithm, and hence the title of this paper.

Guaranteed Robust Tensor Completion via ∗L-SVD with Applications to Remote Sensing Data

This paper conducts a rigorous analysis for the problem of robust tensor completion, which aims at recovering an unknown three-way tensor from incomplete observations corrupted by gross sparse outliers and small dense noises simultaneously due to various reasons such as sensor dead pixels, communication loss, electromagnetic interferences, cloud shadows, etc. To estimate the underlying tensor, a new penalized least squares estimator is first formulated by exploiting the low rankness of the signal tensor within the framework of tensor ∗L-Singular Value Decomposition (∗L-SVD) and leveraging the sparse structure of the outlier tensor. Then, an algorithm based on the Alternating Direction Method of Multipliers (ADMM) is designed to compute the estimator in an efficient way. Statistically, the non-asymptotic upper bound on the estimation error is established and further proved to be optimal (up to a log factor) in a minimax sense. Simulation studies on synthetic data demonstrate that the proposed error bound can predict the scaling behavior of the estimation error with problem parameters (i.e., tubal rank of the underlying tensor, sparsity of the outliers, and the number of uncorrupted observations). Both the effectiveness and efficiency of the proposed algorithm are evaluated through experiments for robust completion on seven different types of remote sensing data.

Traffic Estimation and Prediction via Online Variational Bayesian Subspace Filtering

With the increased proliferation of smart devices, the transit passengers of today expect a higher quality of service in the form of real-time traffic updates, accurate expected time-of-arrival (ETA) predictions. Providing these services requires public transit agencies and private transportation players to maintain full situational awareness of the city-wide traffic. However, most such agencies and companies are resource-constrained and do not have access to city-wide traffic data. The availability of sparsely sampled and outlier-corrupted traffic data renders the resulting traffic maps patchy and unreliable and necessitates the use of sophisticated real-time traffic interpolation and prediction algorithms. Moreover, since the traffic data is measured and collected in a sequential manner, the estimations must also be generated online. Thankfully, the traffic matrices are spatially and temporally structured, allowing the use of time-series and matrix/tensor completion algorithms. This work puts forth a generative model for the traffic density and subsequently uses a variational Bayesian formalism to learn the parameters of the model. Specifically, we consider low-rank traffic matrices whose subspace evolves according to a state-space model with possible sparse outliers. Unlike most matrix/tensor completion algorithms, the proposed model is equipped with automatic relevance determination priors that allow it to learn the parameters in an entirely data-driven manner. A forward-backward algorithm is proposed that enables the updates to be carried out at low-complexity. Simulations carried out on real traffic speed data demonstrate that the proposed algorithm better predicts the future traffic densities as compared to the state-of-the-art matrix/tensor completion algorithms.