Principal Component Pursuit Research Articles

Robust PCA via <inline-formula> <tex-math notation="LaTeX">$\ell _{0}$</tex-math> </inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\ell _{1}$</tex-math> </inline-formula> Regularization

We study the problem of low-rank and sparse decomposition from possibly noisy observations, known as Robust PCA, where the sparse component can be seen as outliers. We first propose a modified objective function where the nuclear norm captures the low-rank term, $\ell _0$ -“norm” addresses the sparse outlier term, and an $\ell _1$ -norm to deal with the additive noise term. The associated algorithm, termed sparsity regularized principal component pursuit (SRPCP), is shown to converge. Under certain model and algorithm parameter settings, it is shown that SRPCP can recover the low-rank component and sparse component exactly in the noiseless case. In the noisy case, we first prove that the widely used principal component pursuit (PCP) method, which was designed for the noiseless case, is actually stable to dense noise. Then, we show that SRPCP has smaller estimation error bound, and can identify outlier entries without any false alarm. Another important byproduct of our analysis is the result that PCP with missing entries is also stable to dense noise. We further propose another objective function which replaces the above nuclear norm by the log-determinant. The proposed algorithm, termed iterative reweighted sparsity regularized principal component pursuit, is also shown to converge. In each iteration, it solves a weighted nuclear norm regularized robust matrix completion problem. We propose an alternating direction method of multipliers algorithm to solve this nonconvex subproblem, which also converges. Empirical studies demonstrate the efficacy of the proposed $\ell _{0}$ - $\ell _{1}$ regularization framework to deal with the outliers as well as its advantage over the existing state-of-the-art methods.

Fast Multilevel Algorithms for Compressive Principal Component Pursuit

Recovering a low-rank matrix from highly corrupted measurements arises in compressed sensing of structured high-dimensional signals (e.g., videos and hyperspectral images among others). Robust principal component analysis, solved via principal component pursuit (PCP), recovers a low-rank matrix from sparse corruptions that are of unknown value and support by decomposing the observation matrix into two terms, a low-rank matrix and a sparse one, accounting for sparse noise and outliers. In the more general setting, where only a fraction of the data matrix has been observed, low-rank matrix recovery is achieved by solving the compressive principal component pursuit (CPCP). Both PCP and CPCP are well-studied convex programs, and numerous iterative algorithms have been proposed for their optimization. Nevertheless, these algorithms involve singular value decomposition (SVD) at each iteration, which renders their applicability challenging in the case of massive data. In this paper, we propose a multilevel approach for the solution of PCP and CPCP problems. The core principle behind our algorithm is to apply SVD in models of lower dimensionality than the original one and then lift its solution to the original problem dimension. Hence, our methods rely on the assumption that the low-rank component can be represented in a lower dimensional space. We show that the proposed algorithms are easy to implement and converge at the same rate but with much lower iteration cost. Numerical experiments on numerous synthetic and real problems indicate that the proposed multilevel algorithms are several times faster than their original counterparts, namely, PCP and CPCP.

Moving Object Detection Through Robust Matrix Completion Augmented With Objectness

We present a novel approach for unsupervised detection of moving objects with nonsalient movements (e.g., rodents in their home cage). The proposed approach starts with separating the moving object from its background by modeling the background in a computationally efficient way. The background modeling is based on the assumption that background in natural videos lies on a low-dimensional subspace. We formulated and solved this problem using a low-rank matrix completion framework. To achieve computational efficiency, we proposed the fast robust matrix completion (fRMC) algorithm, which benefits from the in-face extended Frank–Wolfe approach as its optimization solver. We then augmented our fRMC-based moving object detection by incorporating the spatial information of the object as its objectness into the detection algorithm. With this augmentation we tackle the problem of nonsalient motion. The proposed fRMC algorithm is evaluated on background models challenge and Stuttgart artificial background subtraction datasets. Its detection results are then compared with the popular methods of background subtraction based on the robust principle component analysis and low-rank robust matrix completion methods, solved by inexact augmented Lagrangian multiplier and fast principal component pursuit via alternating minimization (FPCP). The outcomes showed faster computation, at least twice as when other methods are applied, while having a comparable detection accuracy. Moreover, fRMC observed to outperform the FPCP algorithm in background/foreground separation with minor computational overhead. Beyond that, we verified the performance improvement of the augmented fRMC with objectness on detecting the nonsalient motion of in-cage mice using the Caltech resident-intruder mice dataset. The evaluation showed 10% improvement in the detection performance, while significantly dropping the computational time.

A Study of Automatic and Real-Time Table Tennis Fault Serve Detection System.

Calling a table tennis fault serve has never been easy for umpires, since they can only rely on their intuition. This study presents an algorithm that is able to automatically find the positions of the ball and racket in the images captured by high-speed camera. The trajectory of ball toss is analyzed and the result can be used as the objective basis for the umpire to decide if the serve is legal. This algorithm mainly consists of YCbCr color space processing, morphological processing method, circle Hough transform application, separation of moving and static components in an image sequence using the stable principal component pursuit method. The experiment results show that YCbCr color space provides better performance than HSV color space in recognizing the ball color close to skin tone. It is also demonstrated that the positions of the ball and racket can be successfully located by using the methods of color segmentation and stable principal component pursuit. Lastly, it is hoped that this study will provide more useful information regarding how to identify illegal ball toss in tennis ball game using image processing techniques to other researchers.

Dual principal component pursuit

We consider the problem of learning a linear subspace from data corrupted by outliers. Classical approaches are typically designed for the case in which the subspace dimension is small relative to the ambient dimension. Our approach works with a dual representation of the subspace and hence aims to find its orthogonal complement; as such, it is particularly suitable for subspaces whose dimension is close to the ambient dimension (subspaces of high relative dimension). We pose the problem of computing normal vectors to the inlier subspace as a non-convex l1 minimization problem on the sphere, which we call Dual Principal Component Pursuit (DPCP) problem. We provide theoretical guarantees under which every global solution to DPCP is a vector in the orthogonal complement of the inlier subspace. Moreover, we relax the non-convex DPCP problem to a recursion of linear programs whose solutions are shown to converge in a finite number of steps to a vector orthogonal to the subspace. In particular, when the inlier subspace is a hyperplane, the solutions to the recursion of linear programs converge to the global minimum of the nonconvex DPCP problem in a finite number of steps. We also propose algorithms based on alternating minimization and iteratively re-weighted least squares, which are suitable for dealing with large-scale data. Experiments on synthetic data show that the proposed methods are able to handle more outliers and higher relative dimensions than current state-of-the-art methods, while experiments in the context of the three-view geometry problem in computer vision suggest that the proposed methods can be a useful or even superior alternative to traditional RANSAC-based approaches for computer vision and other applications.

Robust Principal Component Pursuit for Fault Detection in a Blast Furnace Process

Since blast furnaces are generally controlled by operators, the minor faults regarded as disturbances might be contained in the collected data matrix. This can severely affect sample distributions, which leads to arbitrary fault detection results using traditional data-driven methods. In this paper, a novel fault detection method named robust principal component pursuit (PCP) to handle minor faults is proposed. The minor faults are separated from columns and rows, respectively, in the training matrix via two matrix norms. By applying the proposed robust PCP method, a low rank matrix containing important process information, as well as explicit variable relationships, and a block sparse matrix containing minor faults are derived. Moreover, the convergence of the proposed method is discussed. Hotelling’s T2 statistic is potentially useful for online process monitoring in the low rank matrix. Finally, to evaluate the decomposition capacity of the proposed method for a matrix containing minor faults, a compar...

MARO: matrix rank optimization for the detection of small-size moving objects from aerial camera platforms

Moving objects detection from aerial camera platforms is a very challenging problem due to the small-size of the moving objects and the false motion of the static background elements. Although many methods have been proposed in this domain, they always have a trade-off between true detections and false detections. This paper proposes a novel solution called matrix rank optimization method (MARO) to achieve high true detections with low false detections. In MARO, the detection problem is formulated as a principal component pursuit with a transformation domain. The novelty of MARO is that it solves this problem by using the inexact Newton method and a backtracking behaviour in inexact augmented Lagrange multiplier. MARO has been extensively evaluated using DARPA VIVID, UCF aerial action, and VIRAT aerial datasets. The results show that MARO outperforms current-state-of-the-art methods, as well as lowers the execution time without sacrificing the accuracy.

A generalized inexact Uzawa method for stable principal component pursuit problem with nonnegative constraints

The problem of recovering the low-rank and sparse components of a matrix is known as the stable principal component pursuit (SPCP) problem. It has found many applications in compressed sensing, image processing, and web data ranking. This paper proposes a generalized inexact Uzawa method for SPCP with nonnegative constraints. The main advantage of our method is that the resulting subproblems all have closed-form solutions and can be executed in distributed manners. Global convergence of the method is proved from variational inequalities perspective. Numerical experiments show that our algorithm converges to the optimal solution as other distributed methods, with better performances.

Infrared dim target detection based on total variation regularization and principal component pursuit

Robust detection of infrared dim and small target contributes significantly to the infrared systems in many applications. Due to the diversity of background scene and unique characteristic of target, the detection of infrared targets remains a challenging problem. In this paper, a novel approach based on total variation regularization and principal component pursuit (TV-PCP) is presented to deal with this problem. The principal component pursuit model only considers the low-rank feature of background images, which will result in poor detection ability in non-uniform and non-smooth scenes. We take into account the total variation regularization term to thoroughly describe background feature, which can achieve good detection result as well as good background estimation result. Firstly, the input infrared image is transformed to a patch image model. Secondly, the TV-PCP model is presented on the patch image. An effective optimization algorithm is proposed to solve this model. Experiments on six real datasets show that the proposed method has superior detection ability under various backgrounds, especially with good background suppression performance and low false alarm rate.

Reinforced Robust Principal Component Pursuit.

High-dimensional data present in the real world is often corrupted by noise and gross outliers. Principal component analysis (PCA) fails to learn the true low-dimensional subspace in such cases. This is the reason why robust versions of PCA, which put a penalty on arbitrarily large outlying entries, are preferred to perform dimension reduction. In this paper, we argue that it is necessary to study the presence of outliers not only in the observed data matrix but also in the orthogonal complement subspace of the authentic principal subspace. In fact, the latter can seriously skew the estimation of the principal components. A reinforced robustification of principal component pursuit is designed in order to cater to the problem of finding out both types of outliers and eliminate their influence on the final subspace estimation. Simulation results under different design situations clearly show the superiority of our proposed method as compared with other popular implementations of robust PCA. This paper also showcases possible applications of our method in critically tough scenarios of face recognition and video background subtraction. Along with approximating a usable low-dimensional subspace from real-world data sets, the technique can capture semantically meaningful outliers.

Stable Analysis of Compressive Principal Component Pursuit

Compressive principal component pursuit (CPCP) recovers a target matrix that is a superposition of low-complexity structures from a small set of linear measurements. Pervious works mainly focus on the analysis of the existence and uniqueness. In this paper, we address its stability. We prove that the solution to the related convex programming of CPCP gives an estimate that is stable to small entry-wise noise. We also provide numerical simulation results to support our result. Numerical results show that the solution to the related convex program is stable to small entry-wise noise under board condition.

Two proximal splitting methods for multi-block separable programming with applications to stable principal component pursuit

Recent years have witnessed the rapid development of the first order methods for multi-block separable programming involving large-scale date-set. The purpose of this paper is to introduce two new first order methods, the proximal splitting methods (PSMs), for the model under consideration. The first PSM fully utilizes the desired property of such problems and adopts fully Jacobian updating rule, which often results in easy subproblems in practice. The global convergence and the worst-case \(\mathcal {O}(1/t)\) convergence rate in an ergodic sense of the first PSM are proved under the condition that the involved functions are assumed to be strongly convex. Applying the hybrid Jacobian and Gauss–Seidel updating rule to the first PSM, we derive the second PSM, whose global convergence can be guaranteed only under the condition that the involved functions are convex. Furthermore, its worst-case \(\mathcal {O}(1/t)\) convergence rate in both the ergodic and non-ergodic senses is also established. Finally, numerical results on stable principal component pursuit are reported to testify the accuracy and speed of the second PSM, and some numerical comparisons are also reported.

Exact Solution Analysis of Strongly Convex Programming for Principal Component Pursuit

In this paper, we address strongly convex programming for principal component analysis, which recovers a target matrix that is a superposition of low-complexity structures from a small set of linear measurements. In this paper, we firstly provide sufficient conditions under which the strongly convex models lead to the exact low-rank matrix recovery. Secondly, we also give suggestions that will guide us how to choose suitable parameters in practical algorithms. Finally, the proposed result is extended to the principal component pursuit with reduced linear measurements and we provide numerical experiments.

A Proximal Fully Parallel Splitting Method for Stable Principal Component Pursuit

As a special three-block separable convex programming, the stable principal component pursuit (SPCP) arises in many different disciplines, such as statistical learning, signal processing, and web data ranking. In this paper, we propose a proximal fully parallel splitting method (PFPSM) for solving SPCP, in which the resulting subproblems all admit closed-form solutions and can be solved in distributed manners. Compared with other similar algorithms in the literature, PFPSM attaches a Glowinski relaxation factor η∈3/2,2/3 to the updating formula for its Lagrange multiplier, which can be used to accelerate the convergence of the generated sequence. Under mild conditions, the global convergence of PFPSM is proved. Preliminary computational results show that the proposed algorithm works very well in practice.

Polar $n$-Complex and $n$-Bicomplex Singular Value Decomposition and Principal Component Pursuit

Informed by recent work on tensor singular value decomposition and circulant algebra matrices, this paper presents a new theoretical bridge that unifies the hypercomplex and tensor-based approaches to singular value decomposition and robust principal component analysis. We begin our work by extending the principal component pursuit to Olariu's polar n-complex numbers as well as their bicomplex counterparts. In doing so, we have derived the polar n-complex and n-bicomplex proximity operators for both the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> - and trace-norm regularizers, which can be used by proximal optimization methods such as the alternating direction method of multipliers. Experimental results on two sets of audio data show that our algebraically informed formulation outperforms tensor robust principal component analysis. We conclude with the message that an informed definition of the trace norm can bridge the gap between the hypercomplex and tensor-based approaches. Our approach can be seen as a general methodology for generating other principal component pursuit algorithms with proper algebraic structures.

Stable principal component pursuit-based thermographic data analysis for defect detection in polymer composites

Defects such as inclusions and voids are commonly observed in fiber reinforced polymer (FRP) composites. In order to ensure the quality of FRP products, it is desirable to have reliable and non-destructive testing techniques for detecting defects. Among existing techniques, pulsed thermography (PT) has the advantages of a wide scanning range and simple operation. However, thermal images generated using PT are often noisy, and can contain non-uniform backgrounds resulting from uneven heating. As a result, post-processing is necessary to improve the detection capability of PT. In this study, stable principal component pursuit (SPCP) is integrated with a moving-window strategy to decompose thermographic data into three parts: a low-rank matrix to approximately extract background information, a dense noise term containing most of the measurement noise, and a sparse matrix reflecting the defects in the tested specimen. In this manner, improved detection results can be obtained from the reconstructed thermal images based on the sparse matrix. The effectiveness of the proposed method is illustrated through experiments.

Fault detection with improved principal component pursuit method

In modern industries, principal component analysis (PCA) is one of the most popular data-driven methods. Since the directions of loading vectors can be severely affected by samples distribution, PCA is known for its sensitivity to outliers. In this paper, a robust principal component analysis method, improved principal component pursuit (IPCP), is proposed. Based on this IPCP method, a process model and an online monitoring statistic is developed. The traditional low rank representation (LRR) idea is introduced into PCP method. By applying this IPCP method, a low-rank coefficient matrix is constructed, and it represents explicit relationships between the variables as well as contains other useful information of the processes. Moreover, the obtained coefficient matrix is potentially useful to derive a powerful fault detection statistic. In order to test and evaluate the effectiveness of coefficient matrix constructed by IPCP and the power of proposed monitoring statistic, all algorithms are first tested in numerical simulation. Then they are illustrated in the TE process, which simulates the practical field condition. Finally the proposed methods are implemented in a blast furnace process.

Robust Multivariate Statistical Process Monitoring via Stable Principal Component Pursuit

In process industries, multivariate statistical process monitoring (MSPM) has become an important technique for enhancing product quality and operation safety. Among the family of MSPM methods, principal component analysis (PCA) may be the most commonly used one, due to its capabilities of dimensionality reduction and highlighting variation in a data set. However, the performance of a PCA model often degrades significantly when there are gross sparse errors, i.e. outliers, contained in the training data, since PCA assumes that the training data matrix only contains an underlying low-rank structure corrupted by dense noise. In this paper, a robust matrix recovery method named stable principal component pursuit (SPCP) is adopted to solve this problem, based on which a process modeling and online monitoring procedure is developed. Such a method inherits the benefits of PCA while being robust to outliers. The effectiveness of the SPCP-based monitoring method is illustrated using the benchmark Tennessee Eastma...

Complex and Quaternionic Principal Component Pursuit and Its Application to Audio Separation

Recently, the principal component pursuit has received increasing attention in signal processing research ranging from source separation to video surveillance. So far, all existing formulations are real-valued and lack the concept of phase, which is inherent in inputs such as complex spectrograms or color images. Thus, in this letter, we extend principal component pursuit to the complex and quaternionic cases to account for the missing phase information. Specifically, we present both complex and quaternionic proximity operators for the $\ell_1$- and trace-norm regularizers. These operators can be used in conjunction with proximal minimization methods such as the inexact augmented Lagrange multiplier algorithm. The new algorithms are then applied to the singing voice separation problem, which aims to separate the singing voice from the instrumental accompaniment. Results on the iKala and MSD100 datasets confirmed the usefulness of phase information in principal component pursuit.

Linear Convergence of Descent Methods for the Unconstrained Minimization of Restricted Strongly Convex Functions

Linear convergence rates of descent methods for unconstrained minimization are usually proved under the assumption that the objective function is strongly convex. Recently it was shown that the weaker assumption of restricted strong convexity suffices for linear convergence of the ordinary gradient descent method. A decisive difference to strong convexity is that the set of minimizers of a restricted strongly convex function need be neither a singleton nor bounded. In this paper we extend the linear convergence results under this weaker assumption to a larger class of descent methods including restarted nonlinear CG, BFGS, and its damped limited memory variants, L-D-BFGS. For twice continuously differentiable objective functions we even obtain r-step superlinear convergence for the CG_DESCENT conjugate gradient method of Hager and Zhang, where r is greater than or equal to the rank of the Hessian at a minimizer. This is remarkable since the Hessian of a restricted strongly convex function need not have full rank. Furthermore we show that convex quadratic splines and objective functions of the unconstrained duals to some linearly constrained optimization problems are restricted strongly convex. In particular this holds for the regularized basis pursuit problem and its analogues for nuclear norm minimization and principal component pursuit.