Robust Principal Component Analysis Problem Research Articles

Outlier Suppression via Non-Convex Robust PCA for Efficient Localization in Wireless Sensor Networks

Due to the fact that outliers can degrade localization accuracy significantly in wireless sensor networks, we propose an outlier suppression approach via non-convex robust principal component analysis (Robust PCA). By introducing several non-convex penalty functions to approximate both the rank function and the sparse penalty function in the original Robust PCA problem, we establish a non-convex objective function and solve it efficiently by the augmented Lagrangian multiplier method and difference of convex programming technique. This framework creates the opportunity to safely perform localization with dimension reduction by standard PCA, which has well-known fragile characteristics in the presence of outliers. Thus, the localization algorithms obtained by sparse coding provide the benefits of computational efficiency in low-dimension. The device-free localization (DFL) experiment on a real-world data set shows the improvement in localization accuracy after standard PCA in the presence of outliers. A statistical distance, called the Wasserstein distance, is introduced to evaluate the processed result of outlier suppression, illustrating that the proposed approach can identify and eliminate outliers in DFL problems.

Dictionary-Based Principal Component Analysis for Ground Moving Target Indication by Synthetic Aperture Radar

This letter addresses an efficient algorithm for ground moving target detection and estimation of motion parameters by synthetic aperture radar (SAR). The proposed method outperforms the conventional robust principal component analysis (RPCA)–based ground moving target indication (GMTI) methods that were proposed in the literature. The ability to estimate the radial and along-track velocities of ground moving targets is provided. The rank constraint in the conventional RPCA problem will be automatically relaxed by employing a dictionary matrix for clutter representation. Thus, the new optimization problem will be solved easier with lower degrees of freedom. Furthermore, this dictionary helps to suppress the clutter of higher Doppler frequencies in case of wind blowing scenarios and intrinsic clutter motion modeling. By employing another dictionary matrix for all possible moving targets with different location and velocity, each solution of the optimization problem will be reasonable as it corresponds to a moving target. Although, the two dictionary matrices impose extra computational burden, this load will be prepared prior to other GMTI processing by the information of the SAR system and scenario parameters. Moreover, the algorithm is proposed for the single-channel SAR configurations and has lower computational load than the conventional RPCA-GMTI methods that have to process the recorded data of multichannel systems. Numerical and experimental results are used to evaluate the performance of the proposed method and validate the theoretical discussions.

Non-convex online robust PCA: Enhance sparsity via ℓp-norm minimization

Compressive sensing, matrix rank optimization and Robust PCA-based matrix decomposition have an increasing number of non-convex approaches for optimizing the ideal ℓ0-norm sparsity. This paper presents a novel online non-convex ℓp-norm based Robust PCA (OLP-RPCA) approach, where 0 < p < 1. OLP-RPCA is developed from the offline version LP-RPCA. Our LP-RPCA method uses a new objective function in the Alternating Direction Method of Multipliers (ADMM) framework to efficiently solve the Robust PCA problem. More importantly, our OLP-RPCA method can achieve real-time performance on large-scale data without parallelizing or implementing on a graphics processing unit. We mathematically and empirically show that the computational complexity of our OLP-RPCA algorithm is linear in both the sample dimension and the number of samples. The proposed approaches are evaluated in various applications including Gaussian/non-Gaussian image denoising, face modeling, real-time background subtraction and video inpainting and compared against numerous state-of-the-art low-rank minimization methods to demonstrate the robustness of our algorithms.

A proximal partially parallel splitting method for separable convex programs

In this paper, we propose a proximal partially parallel splitting method for solving convex minimization problems, where the objective function is separable into m individual operators without any coupled variables, and the structural constraint set comprises only linear functions. At each iteration of this algorithm, one selected subproblem is solved, and subsequently the remaining subproblems are solved in parallel, utilizing the new iterate information. Hence, the proposed method is a hybrid mechanism that combines the nice features of parallel decomposition methods and alternating direction methods, while simultaneously adopting the predictor–corrector strategy to ensure convergence of the algorithm. Our algorithmic framework is also amenable to admitting linearized versions of the subproblems, which frequently have closed-form solutions, thereby making the proposed method more implementable in practice. Furthermore, the worst-case convergence rate of the proposed method is obtained under both ergodic and nonergodic conditions. The efficiency of the proposed algorithm is also demonstrated by solving several instances of the robust PCA problem.

A NSGA-II with Alternating Direction Method of Multipliers Mutation for Solving Multiobjective Robust Principal Component Analysis Problem

A NSGA-II with Alternating Direction Method of Multipliers Mutation for Solving Multiobjective Robust Principal Component Analysis Problem

Double-noise-dual-problem approach to the augmented Lagrange multiplier method for robust principal component analysis

Robust principal component analysis (RPCA) is one of the most useful tools to recover a low-rank data component from the superposition of a sparse component. The augmented Lagrange multiplier (ALM) method enjoys the highest accuracy among all the approaches to the RPCA. However, it still suffers from two problems, namely, a brutal force initialization phase resulting in low convergence speed and ignorance of other types of noise resulting in low accuracy. To this end, this paper proposes a double-noise, dual-problem approach to the augmented Lagrange multiplier method, referred to as DNDP-ALM, for robust principal component analysis. Firstly, the original ALM method considers sparse noise only, ignoring Gaussian noise, which generally exists in real-world data. In our proposed DNDP-ALM, the data consist of low-rank component, sparse component and Gaussian noise component, with RPCA problem converted to convex optimization. Secondly, the original ALM uses a rough initialization of multipliers, leading to more work of iterative calculation and lower calculation accuracy. In our proposed DNDP-ALM, the initialization is carried out by solving a dual problem to obtain the optimal multiplier. The experimental results show that the proposed approach super-performs in solving robust principal component analysis problems in terms of speed and accuracy, compared to the state-of-the-art techniques.

Sparse Malicious False Data Injection Attacks and Defense Mechanisms in Smart Grids

This paper discusses malicious false data injection attacks on the wide area measurement and monitoring system in smart grids. First, methods of constructing sparse stealth attacks are developed for two typical scenarios: 1) random attacks in which arbitrary measurements can be compromised; and 2) targeted attacks in which specified state variables are modified. It is already demonstrated that stealth attacks can always exist if the number of compromised measurements exceeds a certain value. In this paper, it is found that random undetectable attacks can be accomplished by modifying only a much smaller number of measurements than this value. It is well known that protecting the system from malicious attacks can be achieved by making a certain subset of measurements immune to attacks. An efficient greedy search algorithm is then proposed to quickly find this subset of measurements to be protected to defend against stealth attacks. It is shown that this greedy algorithm has almost the same performance as the brute-force method, but without the combinatorial complexity. Third, a robust attack detection method is discussed. The detection method is designed based on the robust principal component analysis problem by introducing element-wise constraints. This method is shown to be able to identify the real measurements, as well as attacks even when only partial observations are collected. The simulations are conducted based on IEEE test systems.

Robust PCA With Partial Subspace Knowledge

In recent work, robust Principal Components Analysis (PCA) has been posed as a problem of recovering a low-rank matrix L and a sparse matrix S from their sum, M: = L + S and a provably exact convex optimization solution called PCP has been proposed. This work studies the following problem. Suppose that we have partial knowledge about the column space of the low rank matrix L. Can we use this information to improve the PCP solution, i.e., allow recovery under weaker assumptions? We propose here a simple but useful modification of the PCP idea, called modified-PCP, that allows us to use this knowledge. We derive its correctness result which shows that, when the available subspace knowledge is accurate, modified-PCP indeed requires significantly weaker incoherence assumptions than PCP. Extensive simulations are also used to illustrate this. Comparisons with PCP and other existing work are shown for a stylized real application as well. Finally, we explain how this problem naturally occurs in many applications involving time series data, i.e., in what is called the online or recursive robust PCA problem. A corollary for this case is also given.

Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise

This paper studies the recursive robust principal components analysis problem. If the outlier is the signal-of-interest, this problem can be interpreted as one of recursively recovering a time sequence of sparse vectors, St, in the presence of large but structured noise, Lt. The structure that we assume on Lt is that Lt is dense and lies in a low-dimensional subspace that is either fixed or changes slowly enough. A key application where this problem occurs is in video surveillance where the goal is to separate a slowly changing background (Lt) from moving foreground objects (St) on-the-fly. To solve the above problem, in recent work, we introduced a novel solution called recursive projected CS (ReProCS). In this paper, we develop a simple modification of the original ReProCS idea and analyze it. This modification assumes knowledge of a subspace change model on the Lt's. Under mild assumptions and a denseness assumption on the unestimated part of the subspace of Lt at various times, we show that, with high probability, the proposed approach can exactly recover the support set of St at all times, and the reconstruction errors of both St and Lt are upper bounded by a time-invariant and small value. In simulation experiments, we observe that the last assumption holds as long as there is some support change of St every few frames.

Speech Denoising via Low-Rank and Sparse Matrix Decomposition

Speech Denoising via Low-Rank and Sparse Matrix Decomposition

Proximity point algorithm for low-rank matrix recovery from sparse noise corrupted data

The method of recovering a low-rank matrix with an unknown fraction whose entries are arbitrarily corrupted is known as the robust principal component analysis (RPCA). This RPCA problem, under some conditions, can be exactly solved via convex optimization by minimizing a combination of the nuclear norm and the l1 norm. In this paper, an algorithm based on the Douglas-Rachford splitting method is proposed for solving the RPCA problem. First, the convex optimization problem is solved by canceling the constraint of the variables, and then the proximity operators of the objective function are computed alternately. The new algorithm can exactly recover the low-rank and sparse components simultaneously, and it is proved to be convergent. Numerical simulations demonstrate the practical utility of the proposed algorithm.

Efficient algorithms for robust and stable principal component pursuit problems

The problem of recovering a low-rank matrix from a set of observations corrupted with gross sparse error is known as the robust principal component analysis (RPCA) and has many applications in computer vision, image processing and web data ranking. It has been shown that under certain conditions, the solution to the NP-hard RPCA problem can be obtained by solving a convex optimization problem, namely the robust principal component pursuit (RPCP). Moreover, if the observed data matrix has also been corrupted by a dense noise matrix in addition to gross sparse error, then the stable principal component pursuit (SPCP) problem is solved to recover the low-rank matrix. In this paper, we develop efficient algorithms with provable iteration complexity bounds for solving RPCP and SPCP. Numerical results on problems with millions of variables and constraints such as foreground extraction from surveillance video, shadow and specularity removal from face images and video denoising from heavily corrupted data show that our algorithms are competitive to current state-of-the-art solvers for RPCP and SPCP in terms of accuracy and speed.

A fast tri-factorization method for low-rank matrix recovery and completion

In recent years, matrix rank minimization problems have received a significant amount of attention in machine learning, data mining and computer vision communities. And these problems can be solved by a convex relaxation of the rank minimization problem which minimizes the nuclear norm instead of the rank of the matrix, and has to be solved iteratively and involves singular value decomposition (SVD) at each iteration. Therefore, those algorithms for nuclear norm minimization problems suffer from high computation cost of multiple SVDs. In this paper, we propose a Fast Tri-Factorization (FTF) method to approximate the nuclear norm minimization problem and mitigate the computation cost of performing SVDs. The proposed FTF method can be used to reliably solve a wide range of low-rank matrix recovery and completion problems such as robust principal component analysis (RPCA), low-rank representation (LRR) and low-rank matrix completion (MC). We also present three specific models for RPCA, LRR and MC problems, respectively. Moreover, we develop two alternating direction method (ADM) based iterative algorithms for solving the above three problems. Experimental results on a variety of synthetic and real-world data sets validate the efficiency, robustness and effectiveness of our FTF method comparing with the state-of-the-art nuclear norm minimization algorithms.