Kernel Principal Component Analysis Method Research Articles

Comparing the performance of linear and nonlinear principal components in the context of high-dimensional genomic data integration.

Linear principal component analysis (PCA) is a widely used approach to reduce the dimension of gene or miRNA expression data sets. This method relies on the linearity assumption, which often fails to capture the patterns and relationships inherent in the data. Thus, a nonlinear approach such as kernel PCA might be optimal. We develop a copula-based simulation algorithm that takes into account the degree of dependence and nonlinearity observed in these data sets. Using this algorithm, we conduct an extensive simulation to compare the performance of linear and kernel principal component analysis methods towards data integration and death classification. We also compare these methods using a real data set with gene and miRNA expression of lung cancer patients. First few kernel principal components show poor performance compared to the linear principal components in this occasion. Reducing dimensions using linear PCA and a logistic regression model for classification seems to be adequate for this purpose. Integrating information from multiple data sets using either of these two approaches leads to an improved classification accuracy for the outcome.

Fault discriminant enhanced kernel principal component analysis incorporating prior fault information for monitoring nonlinear processes

Kernel principal component analysis (KPCA) based fault detection method, whose statistical model only utilizes normal operating data and ignores available prior fault information, may not provide the best fault detection performance for nonlinear process monitoring. In order to exploit available prior fault data to enhance fault detection performance, a fault discriminant enhanced KPCA (FDKPCA) method is proposed, which simultaneously monitors two types of data features, nonlinear kernel principal components (KPCs) and fault discriminant components (FDCs). More specifically, based on the normal operating data, KPCs are extracted by usual KPCA modeling, while with the normal operating data and prior fault data, FDCs are obtained by the kernel local-nonlocal preserving discriminant analysis (KLNPDA). Monitoring statistics are constructed for both the KPCA and KLNPDA sub-models. Moreover, Bayesian inference is employed to transform the corresponding monitoring statistics into fault probabilities, and the overall probability-based monitoring statistics are constructed by weighting the results of the two sub-models, which provides more effective on-line fault detection capability. Two case studies involving a simulated nonlinear system and a continuous stirred tank reactor demonstrate the superior fault detection performance of the proposed KLNPDA scheme over the traditional KPCA method.

Nonlinear Multimode Industrial Process Fault Detection Using Modified Kernel Principal Component Analysis

Kernel principal component analysis (KPCA) has been a state-of-the-art nonlinear process monitoring method. However, KPCA assumes the single operation mode while the real industrial processes often run under multiple operation conditions. In order to monitor the nonlinear multimode processes effectively, this paper proposes a modified KPCA method assisted by the local statistical analysis, referred to as local statistics KPCA (LSKPCA). In the proposed method, two kinds of strategies, including local probability density estimation and statistics pattern analysis, are integrated to improve the traditional KPCA method. To handle the multimode characteristic of industrial processes, local probability density estimation is developed to transform the monitored variables into their probability density values, which follow the unimodal data distribution. For further extracting the statistical information among the process data, statistics pattern analysis technique is applied to capture various orders of statistics, including one-order, second-order, and high-order ones, which constitute the statistics pattern matrix of the monitored data. Furthermore, KPCA modeling is performed on the statistics pattern matrix. The simulations on one numerical example and the continuous stirred tank reactor system demonstrate that the proposed LSKPCA method has the superior fault detection performance compared with the conventional KPCA method.

Anomaly Detection for Aviation Safety Based on an Improved KPCA Algorithm

Thousands of flights datasets should be analyzed per day for a moderate sized fleet; therefore, flight datasets are very large. In this paper, an improved kernel principal component analysis (KPCA) method is proposed to search for signatures of anomalies in flight datasets through the squared prediction error statistics, in which the number of principal components and the confidence for the confidence limit are automatically determined by OpenMP-based K-fold cross-validation algorithm and the parameter in the radial basis function (RBF) is optimized by GPU-based kernel learning method. Performed on Nvidia GeForce GTX 660, the computation of the proposed GPU-based RBF parameter is 112.9 times (average 82.6 times) faster than that of sequential CPU task execution. The OpenMP-based K-fold cross-validation process for training KPCA anomaly detection model becomes 2.4 times (average 1.5 times) faster than that of sequential CPU task execution. Experiments show that the proposed approach can effectively detect the anomalies with the accuracy of 93.57% and false positive alarm rate of 1.11%.

Fault detection and isolation in nonlinear systems with partial Reduced Kernel Principal Component Analysis method

In this article, we suggest an extension of our proposed method in fault detection called Reduced Kernel Principal Component Analysis (RKPCA) (Taouali et al., 2015) to fault isolation. To this end, a set of structured residues is generated by using a partial RKPCA model. Furthermore, each partial RKPCA model was performed on a subset of variables to generate structured residues according to a properly designed incidence matrix. The relevance of the proposed algorithm is revealed on Continuous Stirred Tank Reactor.

3D MR image denoising using rough set and kernel PCA method

In this paper, we have presented a two stage method, using kernel principal component analysis (KPCA) and rough set theory (RST), for denoising volumetric MRI data. A rough set theory (RST) based clustering technique has been used for voxel based processing. The method groups similar voxels (3D cubes) using class and edge information derived from noisy input. Each clusters thus formed now represented via basis vector. These vectors now projected into kernel space and PCA is performed in the feature space. This work is motivated by idea that under Rician noise MRI data may be non-linear and kernel mapping will help to define linear separator between these clusters/basis vectors thus used for image denoising. We have further investigated various kernels for Rician noise for different noise levels. The best kernel is then selected on the performance basis over PSNR and structure similarity (SSIM) measures. The work has been compared with state-of-the-art methods under various measures for synthetic and real databases.

Fault detection for turbine engine disk based on an adaptive kernel principal component analysis algorithm

For commercial aircraft, real-time fault detection is essential for condition monitoring of rotating engine components, which can improve aviation safety and reduce maintenance cost for airline companies. In this article, based on the adaptive kernel principal component analysis method, a real-time fault detection algorithm is proposed for turbine engine disk condition monitoring. A sample reduction strategy based on the k-nearest neighbors method is presented to speed up the kernel principal component analysis approach while still guaranteeing correct results. To efficiently detect fault, the fault detection model is updated timely to suit the working process of turbine engine disk. Sample clusters are obtained through the k-mean method, and the parameter of the kernel function is adaptively adjusted by minimizing the within-cluster distance and maximizing the between-cluster distance in the feature space. Experiments have demonstrated the superiority of the proposed approach in fault detection for turbine engine disk.

A kernel principal component analysis–based degradation model and remaining useful life estimation for the turbofan engine

Remaining useful life estimation of the prognostics and health management technique is a complicated and difficult research question for maintenance. In this article, we consider the problem of prognostics modeling and estimation of the turbofan engine under complicated circumstances and propose a kernel principal component analysis–based degradation model and remaining useful life estimation method for such aircraft engine. We first analyze the output data created by the turbofan engine thermodynamic simulation that is based on the kernel principal component analysis method and then distinguish the qualitative and quantitative relationships between the key factors. Next, we build a degradation model for the engine fault based on the following assumptions: the engine has only had constant failure (i.e. no sudden failure is included), and the engine has a Wiener process, which is a covariate stand for the engine system drift. To predict the remaining useful life of the turbofan engine, we built a health index based on the degradation model and used the method of maximum likelihood and the data from the thermodynamic simulation model to estimate the parameters of this degradation model. Through the data analysis, we obtained a trend model of the regression curve line that fits with the actual statistical data. Based on the predicted health index model and the data trend model, we estimate the remaining useful life of the aircraft engine as the index reaches zero. At last, a case study involving engine simulation data demonstrates the precision and performance advantages of this prediction method that we propose. At last, a case study involving engine simulation data demonstrates the precision and performance advantages of this proposed method, the precision of the method can reach to 98.9% and the average precision is 95.8%.

Online identification using radial basis function neural network coupled with KPCA

In this paper, we are going to propose an online radial basis function (RBF) neural network algorithm without any preprocessing step. Then a kernel principal component analysis (KPCA) is coupled with the proposed online RBF neural network algorithm. Indeed, the KPCA method is used as a preprocessing step to reduce the feature dimension which fed to the RBF neural network. Reducing memory requirements of the models makes RBF neural network training efficient and fast. These two proposed algorithms are applied, with success, for identification of a mobile robot position. The simulation results present that the used sigmoid function as a kernel, compared to other kernel functions, which gives an excellent model and a minimum mean square error.

A new fault detection method for nonlinear process monitoring

Kernel Principal Component Analysis (KPCA) is a nonlinear extension of Principal Component Analysis (PCA). Recently, it is the most popular technique for monitoring nonlinear processes. However, the time-varying property of the industrial processes requires the adaptive ability of the KPCA. Therefore, in this paper, a Variable Moving Window Kernel PCA (VMWKPCA) method is proposed to update the KPCA model. The concept of this method consists of varying the size of the moving window according to the change of the normal process. To evaluate the performance of the proposed method, the VMWKPCA is applied for monitoring a Continuous Stirred Tank Reactor (CSTR) and a Tennessee Eastman process (TE). The results are satisfactory compared to the conventional Moving Window Kernel PCA (MWKPCA) and the Adaptive Kernel PCA (AKPCA).

Multivariate Statistical Kernel PCA for Nonlinear Process Fault Diagnosis in Military Barracks

Because of the nonlinear characteristics of monitoring system in military barracks, the traditional KPCA method either have low sensitivity or unable to detect the fault quickly and accurately. In order to make use of higher-order statistics to get more useful information and meet the requirements of real-time fault diagnosis and sensitivity, a new method of fault detection and diagnosis is proposed based on multivariate statistical kernel principal component analysis (MSKPCA), which combines statistic pattern analysis framework (SPA) and kernel principal component analysis (KPCA). First, the transformation of multivariate statistics and kernel function are conducted in which technology of moving time window is used. Then, PCA is executed to analysis the kernel function obtained from the first step. Moreover, the statistics of T^2 and SPE and the control limits of them are calculated. Finally, simulations on a typical nonlinear numerical example show that the proposed MSKPCA method is more effective than PCA and KPCA in terms of fault detection and diagnosis.

Overview of principal component analysis algorithm

Principal component analysis (PCA) algorithm has been extensively employed in face recognition. However, existing PCA algorithms have some limitations in face recognition. In order to overcome these limitations, researchers proposed some extended-PCA algorithms. In this paper, we present algorithm implementation of the original PCA algorithm and main extended-PCA algorithms including two-dimensional PCA (2DPCA), 2DPCA-based feature fusion approach, the kernel PCA (KPCA), the modular PCA, improved KPCA (IKPCA), efficient sparse KPCA (ESKPCA) and incremental PCA (IPCA). 2DPCA directly computes the projection of a matrix onto a transforming axis. 2DPCA-based feature fusion approach combines the features generated from the two schemes of 2DPCA. KPCA can perform well in extracting features from samples whose components have nonlinear relations. The modular PCA approach divides the original face image into sub-images and applies the original PCA approach to each of these sub-images. The IKPCA algorithm improves KPCA for more efficient feature extraction. The efficient sparse KPCA (ESKPCA) improves the computational efficiency of the previous sparse KPCA methods on large-scale training sample sets. Incremental PCA (IPCA) overcomes the limitation which is hard to scale up the developed systems. In order to compare the recognition rate of these algorithms in face recognition, a series of experiments are performed on three face image databases: ORL, Yale and NIR face databases.

Related and independent variable fault detection based on KPCA and SVDD

This paper proposes a new independent and related variable monitoring based on kernel principal component analysis (KPCA) and support vector data description (SVDD) algorithm. Some process variables are considered independent from other variables and the monitoring of independent and related variables should be performed separately. First, an independent variable division strategy based on mutual information is presented. Second, SVDD and KPCA methods are adopted to monitor independent variable space and related variable space, respectively. Finally, a general statistic is built according to the monitoring results of SVDD and KPCA. The proposed KPCA–SVDD method considers the related and independent characters of variables. This method combines the advantages of KPCA in managing nonlinear related variables and those of SVDD in handling independent variables. A numerical system and the Tennessee Eastman process are used to examine the efficiency of the proposed method. Simulation results have proved the superiority of KPCA–SVDD method.

Kernel PCA-based GLRT for nonlinear fault detection of chemical processes

Fault detection is often utilized for proper operation of chemical processes. In this paper, a nonlinear statistical fault detection using kernel principal component analysis (KPCA)-based generalized likelihood ratio test (GLRT) is proposed. The objective of this work is to extend our previous work (Harrou et al. (2013) to achieve further improvements and widen the applicability of the developed method in practice by using the KPCA method. The KPCA presented here is derived from the nonlinear case of principal component analysis (PCA) algorithm and it is investigated here as modeling algorithm in the task of fault detection. The fault detection problem is addressed so that the data are first modeled using the KPCA algorithm and then the faults are detected using GLRT. The detection stage is related to the evaluation of detection indices, which are signals that reveal the fault presence. Those indices are obtained from the analysis of the difference between the process measurements and their estimations using the KPCA technique. The fault detection performance is illustrated through two simulated examples, one using synthetic data and the other using simulated continuously stirred tank reactor (CSTR) data. The results demonstrate the effectiveness of the KPCA-based GLRT method over the conventional KPCA method through its two charts T2 and Q for detection of single as well as multiple sensor faults.

KPCA method based on within-class auxiliary training samples and its application to pattern classification

Principal component analysis (PCA) and kernel principal component analysis (KPCA) are classical feature extraction methods. However, PCA and KPCA are unsupervised learning methods which always maximize the overall variance and ignore the information of within-class and between-class. In this paper, we propose a simple yet effective strategy to improve the performance of PCA and then this strategy is generalized to KPCA. The proposed methods utilize within-class auxiliary training samples, which are constructed through linear interpolation method. These within-class auxiliary training samples are used to train and get the principal components. In contrast with conventional PCA and KPCA, our proposed methods have more discriminant information. Several experiments are respectively conducted on XM2VTS face database, United States Postal Service (USPS) handwritten digits database and three UCI repository of machine learning databases, experimental results illustrate the effectiveness of the proposed method.

Fault Diagnosis Method on Polyvinyl Chloride Polymerization Process Based on Dynamic Kernel Principal Component and Fisher Discriminant Analysis Method

In view of the fact that the production process of Polyvinyl chloride (PVC) polymerization has more fault types and its type is complex, a fault diagnosis algorithm based on the hybrid Dynamic Kernel Principal Component Analysis-Fisher Discriminant Analysis (DKPCA-FDA) method is proposed in this paper. Kernel principal component analysis and Dynamic Kernel Principal Component Analysis are used for fault diagnosis of Polyvinyl chloride (PVC) polymerization process, while Fisher Discriminant Analysis (FDA) method was adopted to make failure data for further separation. The simulation results show that the Dynamic Kernel Principal Component Analyses to fault diagnosis of Polyvinyl chloride (PVC) polymerization process have better diagnostic accuracy, the Fisher Discriminant Analysis (FDA) can further realize the fault isolation, and the actual fault in the process of Polyvinyl chloride (PVC) polymerization production can be monitored by Dynamic Kernel Principal Component Analysis.

Monitoring of Distillation Column Based on Indiscernibility Dynamic Kernel PCA

Aiming at complicated faults detection of distillation column industrial process, it has faced a grave challenge. In this paper, a new indiscernibility dynamic kernel principal component analysis (I-DKPCA) method is presented and applied to distillation column. Compared with traditional statistical techniques, I-DKPCA not only can capture nonlinear property and dynamic characteristic of processes but also can extract relevant variables from all the variables. Applying this new method to distillation column process (a hardware-in-the-loop simulation system), the results prove the proposed method has great advantages, that is, lower missing rate and higher detection rate for the faults, compared with KPCA and DPCA.

In-process cutting tool remaining useful life evaluation based on operational reliability assessment

In this paper, a method for evaluating the remaining useful life of an individual cutting tool while the tool is in process is proposed. The method is based on the operational reliability of a cutting tool which is used to assess its ability to complete a machining operation. Sensitive features extracted from force, vibration and acoustic emission signals are used to form characteristic matrices. Based on the kernel principal component analysis method, subspace matrices can be developed by reducing redundant information. The principal angle between the matrices of the normal state and the running state in the subspace is calculated. The cosine value of the minimum principal angle is used to assess the tool operational reliability. The remaining useful life of a cutting tool can be evaluated when the operational reliability assessment result is one of the back propagation neural network model’s input parameters together with some machining parameters. A chaotic genetic algorithm is used to optimize the initial weights and thresholds of the model with improved ergodicity and recurrence properties. The chaotic variables are introduced to improve the global searching ability and convergence speed. A case study is presented to validate the performance of the proposed method. The remaining useful life of an individual cutting tool can be evaluated quantitatively without the need of large samples and probability or statistic techniques.

Nonlinear process monitoring based on kernel global–local preserving projections

A new nonlinear dimensionality reduction method called kernel global–local preserving projections (KGLPP) is developed and applied for fault detection. KGLPP has the advantage of preserving global and local data structures simultaneously. The kernel principal component analysis (KPCA), which only preserves the global Euclidean structure of data, and the kernel locality preserving projections (KLPP), which only preserves the local neighborhood structure of data, are unified in the KGLPP framework. KPCA and KLPP can be easily derived from KGLPP by choosing some particular values of parameters. As a result, KGLPP is more powerful than KPCA and KLPP in capturing useful data characteristics. A KGLPP-based monitoring method is proposed for nonlinear processes. T2 and SPE statistics are constructed in the feature space for fault detection. Case studies in a nonlinear system and in the Tennessee Eastman process demonstrate that the KGLPP-based method significantly outperforms KPCA, KLPP and GLPP-based methods, in terms of higher fault detection rates and better fault sensitivity.

Iterated Robust kernel Fuzzy Principal Component Analysis and application to fault detection

In this paper, we propose an Iterated Robust kernel Fuzzy Principal Component Analysis (IRkFPCA), which is the method that attempts to combine the advantages of the state of art methods and use a more accurate multi-objective function for jointly reducing the modeling errors, optimizing the robustness to outliers and improving the time complexity since it does not require the storage and inversion of the covariance matrix to obtain a memory-efficient approximation of kernel PCA. This proposed technique computes iteratively the principal components, which are used for modeling and fault detection. The detection stage is related to the evaluation of residuals, also known as detection indices, which are signals that reveal the fault presence. Those indices are obtained from the analysis of the difference between the process measurements and their estimations using the IRkFPCA technique. The performance of the proposed method is illustrated and compared to Iterated kernel Principal Component Analysis (IkPCA) and Iterated Principal Component Analysis (IPCA) methods through two simulated examples, one using synthetic data and the other using simulated continuously stirred tank reactor (CSTR) data. The results of the comparative studies reveal that the developed IRkFPCA method provides a better performance in terms of modeling and fault detection accuracies than the Iterated Robust Fuzzy Principal Component Analysis (IRFPCA) and Iterated kernel Principal Component Analysis (IkPCA) methods; while both methods provide improved accuracy over the Iterated Principal Component Analysis (IPCA) method.