L1-norm Principal Component Analysis Research Articles

FFT calculation of the L1-norm principal component of a data matrix

This paper presents a fast approximate rank-1 L1-norm Principal Component Analysis (L1-PCA) estimator implemented in the Fourier domain. Specifically, we first rephrase the problem of rank-1 L1-PCA estimation as a cyclic shift parameter estimation and then we present an algorithm for estimating the first L1-norm Principal Component (L1-PC) in the Fourier domain, practically using FFT. The proposed method is shown to be asymptotically efficient and our numerical studies corroborate its performance merits.

L1-norm unsupervised Fukunaga-Koontz transform

The Fukunaga-Koontz transform (FKT) is a powerful supervised feature extraction method used in two-class recognition problems, particularly when the classes have equal mean vectors but different covariance matrices. The present work proves that it is also possible to perform the FKT in an unsupervised manner, sparing the need for labeled data, by using a variant of L1-norm Principal Component Analysis (L1-PCA) that minimizes the L1-norm in the feature space. Rigorous proof is given in the case of data drawn from a mixture of Gaussians. A working iterative algorithm based on gradient-descent in the Stiefel manifold is put forward to perform L1-norm minimization with orthogonal constraints. A number of numerical experiments on synthetic and real data confirm the theoretical findings and the good convergence characteristics of the proposed algorithm.

Reduced-Rank L1-Norm Principal-Component Analysis With Performance Guarantees

Standard Principal-Component Analysis (PCA) is known to be sensitive to outliers among the processed data. On the other hand, L1-norm-based PCA (L1-PCA) exhibits sturdy resistance against outliers, while it performs similar to standard PCA when applied to nominal or smoothly corrupted data [1]. Exact calculation of the K L1-norm Principal Components (L1-PCs) of a rank-r datamatrix X ∈ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D×N</sup> costs O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(r-1)K+1</sup> ) [1], [2]. In this work, we present reduced-rank L1-PCA (RR L1-PCA): a hybrid approach that approximates the K L1-PCs of X by the L1-PCs of its L2-norm-based rank-d approximation (d ≤ r), calculable exactly with reduced complexity O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(d-1)K+1</sup> ). The proposed method combines the denoising capabilities and low computation cost of standard PCA with the outlier-resistance of L1-PCA. RR L1-PCA is accompanied by formal performance guarantees as well as thorough numerical studies that corroborate its computational and corruption resistance merits.

A Method of L1-Norm Principal Component Analysis for Functional Data

Recently, with the popularization of intelligent terminals, research on intelligent big data has been paid more attention. Among these data, a kind of intelligent big data with functional characteristics, which is called functional data, has attracted attention. Functional data principal component analysis (FPCA), as an unsupervised machine learning method, plays a vital role in the analysis of functional data. FPCA is the primary step for functional data exploration, and the reliability of FPCA plays an important role in subsequent analysis. However, classical L2-norm functional data principal component analysis (L2-norm FPCA) is sensitive to outliers. Inspired by the multivariate data L1-norm principal component analysis methods, we propose an L1-norm functional data principal component analysis method (L1-norm FPCA). Because the proposed method utilizes L1-norm, the L1-norm FPCs are less sensitive to the outliers than L2-norm FPCs which are the characteristic functions of symmetric covariance operator. A corresponding algorithm for solving the L1-norm maximized optimization model is extended to functional data based on the idea of the multivariate data L1-norm principal component analysis method. Numerical experiments show that L1-norm FPCA proposed in this paper has a better robustness than L2-norm FPCA, and the reconstruction ability of the L1-norm principal component analysis to the original uncontaminated functional data is as good as that of the L2-norm principal component analysis.

On the rotational invariant L1-norm PCA

Principal component analysis (PCA) is a powerful tool for dimensionality reduction. Unfortunately, it is sensitive to outliers, so that various robust PCA variants were proposed in the literature. One of the most frequently applied methods for high dimensional data reduction is the rotational invariant L1-norm PCA of Ding and coworkers. So far no convergence proof for this algorithm was available. The main topic of this paper is to fill this gap. We reinterpret this robust approach as a conditional gradient algorithm and show moreover that it coincides with a gradient descent algorithm on Grassmann manifolds. Based on the latter point of view, we prove global convergence of the whole series of iterates to a critical point using the Kurdyka-Łojasiewicz property of the objective function, where we have to pay special attention to so-called anchor points, where the function is not differentiable.

Robust direction finding in the fluctuating oceans by complex L1-norm principal component analysis

We consider the problem of robust direction-of-arrival (DoA) estimation in dynamic ocean environments. Robust direction finding of underwater acoustic signals transmitted from known locations may enable accurate underwater localization and enhance link communication rate by instructing a receiver to listen for transmissions from a specific direction. We propose to estimate the DoA of underwater acoustic signals via subspace methods, executed at a very large receiver array, that involve performing what is known as principal-component analysis (PCA) for finding the L2-norm principal vector subspace of the recorded signal snapshots. However, in practice coherence loss, which typically arises from dynamic wavefront fluctuations due to internal waves, scattering from the sea surface and/or bottom and other unknown environmental parameters, may result in recorded signal snapshots that are corrupted by faulty measurements, leading to an inaccurate estimation of the DoA and source position. We propose to model the loss of coherence as multiplicative random noise applied to the measured acoustic signal. In such cases, L2-norm PCA methods suffer from significant performance degradation. Motivated by the resistance of novel L1-nrom-derived subspaces against the impact of irregular, highly deviating points in reduced-dimensionality data approximations, we propose to employ L1-norm (absolute error) maximum projection PCA of the antenna array measurements and evaluate the performance of a novel, outlier-resistant DoA estimation algorithm.

L1-Subspace Tracking for Streaming Data

High-dimensional data usually exhibit intrinsic low-rank structures. With tremendous amount of streaming data generated by ubiquitous sensors in the world of Internet-of-Things, fast detection of such low-rank pattern is of utmost importance to a wide range of applications. In this work, we present an L1-subspace tracking method to capture the low-rank structure of streaming data. The method is based on the L1-norm principal-component analysis (L1-PCA) theory that offers outlier resistance in subspace calculation. The proposed method updates the L1-subspace as new data are acquired by sensors. In each time slot, the conformity of each datum is measured by the L1-subspace calculated in the previous time slot and used to weigh the datum. Iterative weighted L1-PCA is then executed through a refining function. The superiority of the proposed L1-subspace tracking method compared to existing approaches is demonstrated through experimental studies in various application fields.

Realified L1-PCA for direction-of-arrival estimation: theory and algorithms

Subspace-based direction-of-arrival (DoA) estimation commonly relies on the Principal-Component Analysis (PCA) of the sensor-array recorded snapshots. Therefore, it naturally inherits the sensitivity of PCA against outliers that may exist among the collected snapshots (e.g., due to unexpected directional jamming). In this work, we present DoA-estimation based on outlier-resistant L1-norm principal component analysis (L1-PCA) of the realified snapshots and a complete algorithmic/theoretical framework for L1-PCA of complex data through realification. Our numerical studies illustrate that the proposed DoA estimation method exhibits (i) similar performance to the conventional L2-PCA-based method, when the processed snapshots are nominal/clean, and (ii) significantly superior performance when the snapshots are faulty/corrupted.

Adaptive L1-Norm Principal-Component Analysis With Online Outlier Rejection

L1-norm principal-component analysis (L1-PCA) is known to attain sturdy resistance against faulty points (outliers) among the processed data. However, computing the L1-PCA of large datasets, with high number of measurements and/or dimensions, may be computationally impractical; in such cases, incremental solutions could be preferred. At the same time, in many applications it is desired to track the signal subspace via principal component adaptation. In this paper, we present new methods for both incremental and adaptive L1-PCA. Our first algorithm computes L1-PCA incrementally, processing one measurement at a time and performing online rejection of possible outliers; due to its low computational and storage cost, this algorithm is appropriate for application to both large and streaming datasets. Our second algorithm combines the merits of the first one with the additional ability to track changes in the nominal signal subspace. The proposed algorithms are evaluated with experimental studies on subspace estimation/tracking, video surveillance, image conditioning, and direction-of-arrival estimation/tracking.

L1-Norm Principal-Component Analysis of Complex Data

L1-norm Principal-Component Analysis (L1-PCA) of real-valued data has attracted significant research interest over the past decade. However, L1-PCA of complex-valued data remains to date unexplored despite the many possible applications (e.g., in communication systems). In this work, we establish theoretical and algorithmic foundations of L1-PCA of complex-valued data matrices. Specifically, we first show that, in contrast to the real-valued case for which an optimal polynomial-cost algorithm was recently reported by Markopoulos et al., complex L1-PCA is formally NP-hard in the number of data points. Then, casting complex L1-PCA as a unimodular optimization problem, we present the first two suboptimal algorithms in the literature for its solution. Our experimental studies illustrate the sturdy resistance of complex L1-PCA against faulty measurements/outliers in the processed data.

On the Link Between L1-PCA and ICA.

Principal component analysis (PCA) based on L1-norm maximization is an emerging technique that has drawn growing interest in the signal processing and machine learning research communities, especially due to its robustness to outliers. The present work proves that L1-norm PCA can perform independent component analysis (ICA) under the whitening assumption. However, when the source probability distributions fulfil certain conditions, the L1-norm criterion needs to be minimized rather than maximized, which can be accomplished by simple modifications on existing optimal algorithms for L1-PCA. If the sources have symmetric distributions, we show in addition that L1-PCA is linked to kurtosis optimization. A number of numerical experiments illustrate the theoretical results and analyze the comparative performance of different algorithms for ICA via L1-PCA. Although our analysis is asymptotic in the sample size, this equivalence opens interesting new perspectives for performing ICA using optimal algorithms for L1-PCA with guaranteed global convergence while inheriting the increased robustness to outliers of the L1-norm criterion.

A pure [formula omitted]-norm principal component analysis

The L1 norm has been applied in numerous variations of principal component analysis (PCA). An L1-norm PCA is an attractive alternative to traditional L2-based PCA because it can impart robustness in the presence of outliers and is indicated for models where standard Gaussian assumptions about the noise may not apply. Of all the previously-proposed PCA schemes that recast PCA as an optimization problem involving the L1 norm, none provide globally optimal solutions in polynomial time. This paper proposes an L1-norm PCA procedure based on the efficient calculation of the optimal solution of the L1-norm best-fit hyperplane problem. We present a procedure called L1-PCA∗ based on the application of this idea that fits data to subspaces of successively smaller dimension. The procedure is implemented and tested on a diverse problem suite. Our tests show that L1-PCA∗ is the indicated procedure in the presence of unbalanced outlier contamination.