Robust Kernel Principal Component Analysis Research Articles

Robust kernel principal component analysis and its application in blockage detection at the turn of conveyor belt

Industrial data is usually nonlinear and corrupted by noise and outliers, which brings great challenges to fault detection and modeling in industrial data. To this end, L21-norm-based kernel principal component analysis is incorporated into the self-paced learning framework (L21-KPCA-SPL) in this study. It is innovative in sense that: (1) L21-KPCA is proposed, which can solve the nonlinear problem of data and increase the robustness of the algorithm; (2) self-paced learning (SPL) framework can avoid the local optimal solution problem caused by non-convex optimization; (3) based on the process monitoring of L21-KPCA-SPL, the pixel cumulative contribution of monitoring statistics is proposed. Compared with traditional PCA-like methods, the proposed algorithm is more robust. Compared with other robust methods, the proposed algorithm is more suitable for dealing with nonlinear data. Extensive experiments have been conducted on image classification datasets to demonstrate that the proposed method is more effective than other state-of-the-art methods. Furthermore, the proposed algorithm is used to detect ore blockage fault at the turn of conveyor belt. The experimental results further verify the effectiveness of the proposed method, which can replace the traditional manual detection and meet the requirements of real-time detection of ore blockage.

Robust kernel principal component analysis with optimal mean

The kernel principal component analysis (KPCA) serves as an efficient approach for dimensionality reduction. However, the KPCA method is sensitive to the outliers since the large square errors tend to dominate the loss of KPCA. To strengthen the robustness of KPCA method, we propose a novel robust kernel principal component analysis with optimal mean (RKPCA-OM) method. RKPCA-OM not only possesses stronger robustness for outliers than the conventional KPCA method, but also can eliminate the optimal mean automatically. What is more, the theoretical proof proves the convergence of the algorithm to guarantee that the optimal subspaces and means are obtained. Lastly, exhaustive experimental results verify the superiority of our method.

Fault Detection of Wind Turbines by Subspace Reconstruction-Based Robust Kernel Principal Component Analysis

Data collected from the supervisory control and data acquisition (SCADA) system are widely used in wind farms to detect wind turbines’ (WTs) faults. However, it is challenging to extract wind turbines’ performance trends and provide accurate alarms based on SCADA data with large operating conditions fluctuations. This article presents a fault detection frame of subspace reconstruction-based robust kernel principal component analysis (SR-RKPCA) model for wind turbines SCADA data to extract nonlinear features under discontinuous interference. First, to improve the stability of the fault detection model of wind turbines, an RKPCA method is developed to decompose the original signal, decomposed into principal component subspace and residual subspace. Second, the permutation entropy is adopted to measure the residual matrix components’ disturbance level and remove trendless noise components. Then, a combined index is developed for fault detection with the aid of the residual space matrix with trend information. Ultimately, the proposed SR-RKPCA method for wind turbine detection is verified by investigating wind turbine faults cases. The proposed method has outstanding robustness and a more vital ability to extract nonlinear features for fault detection than traditional principal component analysis (PCA)- or KPCA-based methods.

Robust Kernel Principal Component Analysis With ℓ2,1-Regularized Loss Minimization

Principal component analysis (PCA) is a widely used unsupervised method for dimensionality reduction. The kernelized version is called kernel principal component analysis (KPCA), which can capture the nonlinear data structure. KPCA is derived from the Gram matrix, which is not robust when outliers exist in the data. This may yield the principal axis in the feature space deviated by outliers, leading to misinterpretation of the principal components. In this paper, we propose a robust method for KPCA with a reformulation in Euclidean space to construct a robust KPCA method, where an error measurement is introduced into the loss function, and l 2,1 -regularization is added to the loss function. The idea of l 2,1 -regularization of the proposed method is motivated by sparse PCA via variable projection. However, because orthogonality is not satisfied in the proposed method, orthonormal bases are obtained by using the Gram-Schmidt orthonormalization process. In the experiments, a toy example and real data are used for outlier detection to verify the method's performance and effectiveness. In the toy example, the proposed method reduces the influence of outliers and detects more outliers than KPCA. For the real data, the proposed method improves detection in comparison to other existing methods.

A novel robust kernel principal component analysis for nonlinear statistical shape modeling from erroneous data

Statistical Shape Models (SSMs) have achieved considerable success in medical image segmentation. A high quality SSM is able to approximate the main plausible variances of a given anatomical structure to guide segmentation. However, it is technically challenging to derive such a quality model because: (1) the distribution of shape variance is often nonlinear or multi-modal which cannot be modeled by standard approaches assuming Gaussian distribution; (2) as the quality of annotations in training data usually varies, heavy corruption will degrade the quality of the model as a whole. In this work, these challenges are addressed by introducing a generic SSM that is able to model nonlinear distribution and is robust to outliers in training data. Without losing generality and assuming a sparsity in nonlinear distribution, a novel Robust Kernel Principal Component Analysis (RKPCA) for statistical shape modeling is proposed with the aim of constructing a low-rank nonlinear subspace where outliers are discarded. The proposed approach is validated on two different datasets: a set of 30 public CT kidney pairs and a set of 49 MRI ankle bones volumes. Experimental results demonstrate a significantly better performance on outlier recovery and a higher quality of the proposed model as well as lower segmentation errors compared to the state-of-the-art techniques.

Exactly Robust Kernel Principal Component Analysis.

Robust principal component analysis (RPCA) can recover low-rank matrices when they are corrupted by sparse noises. In practice, many matrices are, however, of high rank and, hence, cannot be recovered by RPCA. We propose a novel method called robust kernel principal component analysis (RKPCA) to decompose a partially corrupted matrix as a sparse matrix plus a high- or full-rank matrix with low latent dimensionality. RKPCA can be applied to many problems such as noise removal and subspace clustering and is still the only unsupervised nonlinear method robust to sparse noises. Our theoretical analysis shows that, with high probability, RKPCA can provide high recovery accuracy. The optimization of RKPCA involves nonconvex and indifferentiable problems. We propose two nonconvex optimization algorithms for RKPCA. They are alternating direction method of multipliers with backtracking line search and proximal linearized minimization with adaptive step size (AdSS). Comparative studies in noise removal and robust subspace clustering corroborate the effectiveness and the superiority of RKPCA.

Density-sensitive Robust Fuzzy Kernel Principal Component Analysis technique

In order to deal with the sensitivity of traditional kernel principal component analysis (KPCA) to the outliers and high computational complexity of the other existing robust KPCAs, a novel density-sensitive Robust Fuzzy Kernel Principal Component Analysis (DRF-KPCA) is proposed in this paper. First, the initial membership degree of the sample is determined by introducing the relative density. Second, the membership degree is formulated based on reconstruction error and updated by the optimal gradient descent approach, which can effectively solve the problem of the principal component skewing caused by the sensitivity of KPCA to the outliers. Ultimately, the computational complexity and running time of DRF-KPCA are significantly reduced by simplifying the calculation formula of the reconstruction error. In the experiments, as compared with KPCA and other modified approaches on both the datasets with outliers and the datasets without outliers, DRF-KPCA is evaluated to effectively eliminate the impacts of the outliers on the performance with low computational complexity. In addition, the influence of parameters on the performance of DRF-KPCA is also analyzed in detail and the suggestions of determination on the optimal coefficients are given. Eventually, the comparison results with the-state-of-art techniques on UCI benchmark datasets and other high-dimensional classification datasets demonstrate that the performance of DRF-KPCA is significantly improved.

Remaining useful life prediction for power-shift steering transmission based on fusion of multiple oil spectra

Remaining useful life prediction is a critical issue to fault diagnosis and health management of power-shift steering transmission. Power-shift steering transmission wear, which leads to the increase of wear particles and severe wear afterwards, is a slow degradation process, which can be monitored by oil spectral analysis, but the actual degree of the power-shift steering transmission degradation is often difficult to evaluate. The main purpose of this article is to provide a more accurate remaining useful life prediction methodology for power-shift steering transmission compared to relying solely on an individual spectral oil data. Our methodology includes multiple degradation data fusion, degradation index construction, degradation modelling and remaining useful life estimation procedures. First, the robust kernel principal component analysis is used to reduce the data dimension, and the state space model is utilized to construct the wear degradation index. Then, the Wiener process–based degradation model is established based on the constructed degradation index, and the explicit formulas for several important quantities for remaining useful life estimation such as the probability density function and cumulative distribution function are derived. Finally, a case study is presented to demonstrate the applicability of the proposed methodology. The results show that the proposed remaining useful life prediction methodology can objectively describe the power-shift steering transmission degradation law, and the predicted remaining useful life has been extended as 65 Mh (38.2%) compared with specified maintenance interval. This will reduce the maintenance times of power-shift steering transmission life cycle and finally save the maintenance costs.

Object registration for remote sensing images using robust kernel pattern vectors

The registration of multitemperal remote sensing images before and after a disaster is a challenging problem. The main reason is that there is a dearth of homologous features that can be used as tie points for registration of variform objects. This paper proposes a new registration method based on robust kernel pattern vector (KPV) for water objects in the remote sensing images, which have complex deformation before and after a disaster. We show how the variform objects can be precisely registered using their robust kernel pattern vectors (RKPVs). The contribution can be divided into three parts. First, a novel shape descriptor, named as kernel pattern vector (KPV), is defined. Second, a robust kernel principal component analysis (RKPCA) method is proposed, which can not only capture the common pattern of the variform objects but can also act as the Criterion for outlier detection. Finally, a registration approach is derived based on the implicit RKPVs. We demonstrate the power of the proposed approach by comparing it with other existing methods using two real cases: one for lake monitoring in the Jiayu region, and the other for damage mapping of earthquake-induced barrier lake at Tangjiashan (2008 Wenchuan Earthquake). The results show that the method is effective in capturing the common structural pattern of the variform objects before and after a disaster.

An iterative algorithm for robust kernel principal component analysis

We introduce a technique to improve iterative kernel principal component analysis (KPCA) robust to outliers due to undesirable artifacts such as noises, alignment errors, or occlusion. The proposed iterative robust KPCA (rKPCA) links the iterative updating and robust estimation of principal directions. It inherits good properties from these two ideas for reducing the time complexity, space complexity, and the influence of these outliers on estimating the principal directions. In the asymptotic stability analysis, we also show that our iterative rKPCA converges to the weighted kernel principal kernel components from the batch rKPCA. Experimental results are presented to confirm that our iterative rKPCA achieves the robustness as well as time saving better than batch KPCA.

Registration Using Robust Kernel Principal Component for Object-Based Change Detection

Postclassification comparison provides a feasible approach to detect the changes of remote sensing images which have strongly inhomogeneous scenes. For pre- and postevent scenarios, registration is a challenging task because variform classifications may result in a dearth of homologous points to be used as tie points. In this letter, we show how the variform objects can be precisely registered using their robust kernel principal components (RKPCs). The contribution can be divided into two parts. First, a robust kernel principal component analysis (RKPCA) method is proposed to capture the common pattern of the variform objects. Second, a registration approach based on the implicit RKPCs is derived. We demonstrate the power of the proposed approach using two real cases: one for lake monitoring in the Jiayu region, and the other for damage mapping of earthquake-induced barrier lake at Tangjiashan. The results show that the method is effective in capturing structural pattern and generalizes well for registration.

Robust kernel principal component analysis and classification

Kernel principal component analysis (KPCA) extends linear PCA from a real vector space to any high dimensional kernel feature space. The sensitivity of linear PCA to outliers is well-known and various robust alternatives have been proposed in the literature. For KPCA such robust versions received considerably less attention. In this article we present kernel versions of three robust PCA algorithms: spherical PCA, projection pursuit and ROBPCA. These robust KPCA algorithms are analyzed in a classification context applying discriminant analysis on the KPCA scores. The performances of the different robust KPCA algorithms are studied in a simulation study comparing misclassification percentages, both on clean and contaminated data. An outlier map is constructed to visualize outliers in such classification problems. A real life example from protein classification illustrates the usefulness of robust KPCA and its corresponding outlier map.

Generalized KPCA by adaptive rules in feature space

Principal component analysis (PCA) is well recognized in dimensionality reduction, and kernel PCA (KPCA) has also been proposed in statistical data analysis. However, KPCA fails to detect the nonlinear structure of data well when outliers exist. To reduce this problem, this paper presents a novel algorithm, named iterative robust KPCA (IRKPCA). IRKPCA works well in dealing with outliers, and can be carried out in an iterative manner, which makes it suitable to process incremental input data. As in the traditional robust PCA (RPCA), a binary field is employed for characterizing the outlier process, and the optimization problem is formulated as maximizing marginal distribution of a Gibbs distribution. In this paper, this optimization problem is solved by stochastic gradient descent techniques. In IRKPCA, the outlier process is in a high-dimensional feature space, and therefore kernel trick is used. IRKPCA can be regarded as a kernelized version of RPCA and a robust form of kernel Hebbian algorithm. Experimental results on synthetic data demonstrate the effectiveness of IRKPCA.