Kernel Principal Component Analysis Model Research Articles

Bayesian optimization of multiscale kernel principal component analysis and its application to model Gas-to-liquid (GTL) process data

Kernel methods map the data from original space into a higher-dimensional space in which linear methods are applied. In many applications, the inverse mapping is also important, and the pre-image of a feature vector must be found in the original space. Kernel principal component analysis (KPCA) based kernel density estimation (KDE) has been developed to solve this problem. However, the performance of the KPCA technique greatly depends on the choice of some parameters which can lead to poor modeling performance when these parameters are not well identified. Thus, fully Bayesian optimization KPCA (BOKPCA) is proposed to enhance the performance of the KPCA model. BOKPCA method aims to automatically select the best parameters of the KPCA model. Generally, kernel methods struggle to handle nonlinear data contaminated with high levels of noise. This is because the noise affects every principal component, making it challenging to mitigate its influence during the reconstruction step. Consequently, to further enhance the ability of KPCA and BOKPCA models, we propose to integrate multi-scale filtering with these two models. The efficiency of the proposed methods are evaluated using a simulated nonlinear process and real data generated from a bench-scale Fischer–Tropsch (FT) process.

Improving kernel PCA-based algorithm for fault detection in nonlinear industrial process through fractal dimension

Principal Component Analysis (PCA) is a widely used technique for fault detection and diagnosis. PCA works well when the data set has linear characteristics. However, most industrial processes have nonlinear characteristics in their data. Kernel PCA (KPCA) is an alternative solution for such types of data sets. This solution doesn’t come without a cost since one of KPCA’s disadvantages is a large number of observations which results in more occupied storage space and more execution time than the PCA technique. Furthermore, if the data is too large it may minimize the monitoring performance of the KPCA model. Reduced KPCA (RKPCA) is a solution for the conventional KPCA limitations. Firstly, RKPCA can deal with nonlinear characteristics without crucial problems because it is based on the KPCA algorithm with a data reduction part where it keeps most of the data’s information. Thus, by reducing the number of observations RKPCA reduces the occupied storage space and execution time while preserving tolerable monitoring performance. The proposed RKPCA algorithm consists of two parts. First, the large-sized training data set is reduced using the fractal dimension technique (correlation dimension). Afterward, the KPCA model is developed through the obtained reduced training data set. The proposed scheme is applied to the Tennessee Eastman Process and the Cement Plant Rotary Kiln data sets to evaluate its performance in comparison with other algorithms.

Multi-view kernel PCA for time series forecasting

In this paper, we propose a kernel principal component analysis model for multi-variate time series forecasting, where the training and prediction schemes are derived from the multi-view formulation of Restricted Kernel Machines. The training problem is simply an eigenvalue decomposition of the summation of two kernel matrices corresponding to the views of the input and output data. When a linear kernel is used for the output view, it is shown that the forecasting equation takes the form of kernel ridge regression. When that kernel is non-linear, a pre-image problem has to be solved to forecast a point in the input space. We evaluate the model on several standard time series datasets, perform ablation studies, benchmark with closely related models and discuss its results.

Semi-Supervised KPCA-Based Monitoring Techniques for Detecting COVID-19 Infection through Blood Tests.

This study introduces a new method for identifying COVID-19 infections using blood test data as part of an anomaly detection problem by combining the kernel principal component analysis (KPCA) and one-class support vector machine (OCSVM). This approach aims to differentiate healthy individuals from those infected with COVID-19 using blood test samples. The KPCA model is used to identify nonlinear patterns in the data, and the OCSVM is used to detect abnormal features. This approach is semi-supervised as it uses unlabeled data during training and only requires data from healthy cases. The method's performance was tested using two sets of blood test samples from hospitals in Brazil and Italy. Compared to other semi-supervised models, such as KPCA-based isolation forest (iForest), local outlier factor (LOF), elliptical envelope (EE) schemes, independent component analysis (ICA), and PCA-based OCSVM, the proposed KPCA-OSVM approach achieved enhanced discrimination performance for detecting potential COVID-19 infections. For the two COVID-19 blood test datasets that were considered, the proposed approach attained an AUC (area under the receiver operating characteristic curve) of 0.99, indicating a high accuracy level in distinguishing between positive and negative samples based on the test results. The study suggests that this approach is a promising solution for detecting COVID-19 infections without labeled data.

A deep kernel method for lithofacies identification using conventional well logs

How to fit a properly nonlinear classification model from conventional well logs to lithofacies is a key problem for machine learning methods. Kernel methods (e.g., KFD, SVM, MSVM) are effective attempts to solve this issue due to abilities of handling nonlinear features by kernel functions. Deep mining of log features indicating lithofacies still needs to be improved for kernel methods. Hence, this work employs deep neural networks to enhance the kernel principal component analysis (KPCA) method and proposes a deep kernel method (DKM) for lithofacies identification using well logs. DKM includes a feature extractor and a classifier. The feature extractor consists of a series of KPCA models arranged according to residual network structure. A gradient-free optimization method is introduced to automatically optimize parameters and structure in DKM, which can avoid complex tuning of parameters in models. To test the validation of the proposed DKM for lithofacies identification, an open-sourced dataset with seven conventional logs (GR, CAL, AC, DEN, CNL, LLD, and LLS) and lithofacies labels from the Daniudi Gas Field in China is used. There are eight lithofacies, namely clastic rocks (pebbly, coarse, medium, and fine sandstone, siltstone, mudstone), coal, and carbonate rocks. The comparisons between DKM and three commonly used kernel methods (KFD, SVM, MSVM) show that (1) DKM (85.7%) outperforms SVM (77%), KFD (79.5%), and MSVM (82.8%) in accuracy of lithofacies identification; (2) DKM is about twice faster than the multi-kernel method (MSVM) with good accuracy. The blind well test in Well D13 indicates that compared with the other three methods DKM improves about 24% in accuracy, 35% in precision, 41% in recall, and 40% in F1 score, respectively. In general, DKM is an effective method for complex lithofacies identification. This work also discussed the optimal structure and classifier for DKM. Experimental results show that (m1,m2,0) is the optimal model structure and linear SVM is the optimal classifier. (m1,m2,0) means there are m1 KPCAs, and then m2 residual units. A workflow to determine an optimal classifier in DKM for lithofacies identification is proposed, too.

Effective Fault Detection and Diagnosis for Power Converters in Wind Turbine Systems Using KPCA-Based BiLSTM

The current work presents an effective fault detection and diagnosis (FDD) technique in wind energy converter (WEC) systems. The proposed FDD framework merges the benefits of kernel principal component analysis (KPCA) model and the bidirectional long short-term memory (BiLSTM) classifier. In the developed FDD approach, the KPCA model is applied to extract and select the most effective features, while the BiLSTM is utilized for classification purposes. The developed KPCA-based BiLSTM approach involves two main steps: feature extraction and selection, and fault classification. The KPCA model is developed in order to select and extract the most efficient features and the final features are fed to the BiLSTM to distinguish between different working modes. Different simulation scenarios are considered in this study in order to show the robustness and performance of the developed technique when compared to the conventional FDD methods. To evaluate the effectiveness of the proposed KPCA-based BiLSTM approach, we utilize data obtained from a healthy WTC, which are then injected with several fault scenarios: simple fault generator-side, simple fault grid-side, multiple fault generator-side, multiple fault grid-side, and mixed fault on both sides. The diagnosis performance is analyzed in terms of accuracy, recall, precision, and computation time. Furthermore, the efficiency of fault diagnosis is shown by the classification accuracy parameter. The experimental results show the efficiency of the developed KPCA-based BiLSTM technique compared to the classical FDD techniques (an accuracy of 97.30%).

Automatic Detection of Unbalanced Sitting Postures in Wheelchairs Using Unlabeled Sensor Data

This letter presents an effective data-driven anomaly detection scheme for automatically recognizing unbalanced sitting posture in a wheelchair using data from pressure sensors embedded in the wheelchair. Essentially, the designed approach merges the desirable features of the kernel principal component analysis (KPCA) as a feature extractor with the Kantorovich distance (KD)-driven monitoring chart to detect abnormal sitting posture in a wheelchair. It is worth noting that this approach does not require labeled data but only employs normal event data to train the detection model, which makes it more appealing in practice. Specifically, we used the KPCA model to exploit its capacity to reduce the dimensionality of nonlinear data to obtain good detection. At the same time, the KD monitoring scheme is an efficient distribution-driven anomaly detection approach in multivariate data. Furthermore, a nonparametric decision threshold using kernel density estimation is adopted to extend the flexibility of the proposed approach. Publicly available data have been used to verify the detection capacity of the proposed approach. The overall detection system proved promising, outperforming some commonly used monitoring methods.

Dynamic nonlinear process monitoring based on dynamic correlation variable selection and kernel principal component regression

In actual industrial processes, data are usually time series. Each process variable may have strong autocorrelation and cross-correlation with other variables with different delays. In addition, there are usually complex nonlinearities among variables. To further improve the monitoring performance for dynamic nonlinear processes, establishing a nonlinear filtering model for each variable is necessary. Therefore, a novel dynamic nonlinear process monitoring method based on dynamic nonlinear feature selection and kernel principal component regression (KPCR) is proposed in this study. First, dynamic nonlinear related variables are selected for each variable through mutual information by considering variables with different time delays. Second, process variables are divided into response and independent variable sets. Third, corresponding KPCR models are established to describe the dynamic relationships with the selected dynamic related variables as input variables and with response variables as output variables. To monitor the dynamic processes, kernel principal component analysis model is constructed on the basis of the residuals, where the residuals are obtained by comparing measured values of instruments with the predicted values of KPCR models. A support vector data description model is established to monitor the independent variables. Three cases are used to verify the performance of the novel approach. Results show that the proposed method is superior to and more effective than other advanced dynamic process monitoring methods.

RKPCA-based approach for fault detection in large scale systems using variogram method

Principal Component Analysis (PCA)-based approach for fault detection is a simple and accurate data-driven technique for feature extraction and selection. However, PCA performs poorly if the data used has nonlinear characteristics where this type of data is widely present in most industrial processes. To overcome this drawback, Kernel PCA (KPCA) is an alternative technique used to work on this type of data but it requires more computation time and memory storage space for large-sized data sets. Many size reduction techniques have been developed to select the most relevant observations that will be employed by KPCA. This, known as Reduced KPCA (RKPCA), consequently requires less computation time and memory storage space than KPCA. Besides, it possesses the advantages of both KPCA and standard PCA. In this paper, a reduction in the size of a data set based on a multivariate variogram is proposed. According to its conventional formalism, the uncorrelated observations are selected and kept to form a reduced training data set. Afterward, the KPCA model is built through this data set for faults detection purposes. The proposed RKPCA scheme is tested using an actual involuntary process fault and various simulated sensor faults in a cement plant. Compared to other RKPCA techniques, the developed one yields better results.

Rapid Speaker Adaptation Based on Combination of KPCA and Latent Variable Model

Speaker adaptation is implemented in order to shift the speaker-independent model closer to the new speaker speech characteristics to improve the speech recognition performance. The kernel eigenspace-based speaker adaptation methods provide satisfactory performance using only a small amount of adaptation data. In such adaptation methods, kernel principal component analysis (KPCA) is applied to the training speaker space in order to create kernel eigenspace. Then, the adapted acoustic model to the new user is calculated in that space. One limitation of KPCA is its inability to define a precise pre-image of the model adapted in the kernel eigenspace, back to the speaker space. Therefore, a huge amount of computations is required to perform adaptation. The previously developed solutions for calculation of an approximate pre-image of the adapted model do not necessarily lead to the optimal conditions. Therefore, in this paper, we propose an efficient solution for this problem to construct more reliable pre-image of the adapted model in the speaker space. For this purpose, we benefit from the latent variable model to define a probabilistic model for description of the applied mapping between the kernel eigenspace and the speaker space. The experiments were conducted on two speech databases: FARSDAT, a Persian, and TIMIT, an English speech database. Implementing a typical HMM-based automatic speech recognition system, it was verified that the proposed method, utilizing about three seconds of adaptation data, achieves up to 4.4% and 7.6% relative phoneme recognition accuracy rate over the speaker-independent model on FARSDAT and TIMIT, respectively. Moreover, the proposed approach demonstrated superior performance compared to the other kernel eigenspace-based adaptation methods.

Online statistical hypothesis test for leak detection in water distribution networks

This paper aims at improving the operation of the water distribution networks (WDN) by developing a leak monitoring framework. To do that, an online statistical hypothesis test based on leak detection is proposed. The developed technique, the so-called exponentially weighted online reduced kernel generalized likelihood ratio test (EW-ORKGLRT), is addressed so that the modeling phase is performed using the reduced kernel principal component analysis (KPCA) model, which is capable of dealing with the higher computational cost. Then the computed model is fed to EW-ORKGLRT chart for leak detection purposes. The proposed approach extends the ORKGLRT method to the one that uses exponential weights for the residuals in the moving window. It might be able to further enhance leak detection performance by detecting small and moderate leaks. The developed method’s main advantages are first dealing with the higher required computational time for detecting leaks and then updating the KPCA model according to the dynamic change of the process. The developed method’s performance is evaluated and compared to the conventional techniques using simulated WDN data. The selected performance criteria are the excellent detection rate, false alarm rate, and CPU time.

A Hybrid Approach for Process Monitoring: Improving Data-Driven Methodologies With Dataset Size Reduction and Interval-Valued Representation

Kernel principal component analysis (KPCA) is a well-established data-driven process modeling and monitoring framework that has long been praised for its performances. However, it is still not optimal for large-scale and uncertain systems. Applying KPCA usually takes a long time and a significant storage space when big data are utilized. In addition, it leads to a serious loss of information and ignores uncertainties in the processes. Consequently, in this paper, two uncertain nonlinear statistical fault detection methods using an interval reduced kernel principal component analysis (IRKPCA) are proposed. The main objective of the proposed methods is twofold. Firstly, reduce the number of observations in the data matrix through two techniques: a method, called IRKPCA $_{ED}$ , is based on Euclidean distance between samples as dissimilarity metric such that only one observation is kept in case of redundancy to build the reduced reference KPCA model, and another method, called IRKPCA $_{PCA}$ , is established on the PCA algorithm to treat the hybrid correlations between process variables and extract a reduced number of observations from the training data matrix. Secondly, address the problem of uncertainties in systems using a latent-driven technique based on interval-valued data. Taking into account sensors uncertainties via IRKPCA ensures better monitoring by reducing the computational and storage costs. The study demonstrated the feasibility and effectiveness of the proposed approaches for faults detection in two real world applications: Tennessee Eastman (TE) process and real air quality monitoring network (AIRLOR) data.

New reduced kernel PCA for fault detection and diagnosis in cement rotary kiln

Fault detection and diagnosis (FDD) based on data-driven techniques play a crucial role in industrial process monitoring. It intends to promptly detect and identify abnormalities and enhance the reliability and safety of the processes. Kernel Principal Component Analysis (KPCA) is a powerful FDD based data-driven method. It has gained much interest due to its ability in monitoring nonlinear systems. However, KPCA suffers from high computing time and large storage space when a large-sized training dataset is used. So, extracting and selecting the more relevant observations could provide a good solution to high computation time and memory requirements costs. In this paper, a new Reduced KPCA (RKPCA) approach is developed to address that issue. It aims to preserve one representative observation for each similar and selected Euclidean distance between training samples. Afterwards, the obtained reduced training dataset is used to build a KPCA model for FDD purposes. The developed RKPCA scheme is tested and evaluated across a numerical example and an actual involuntary process fault and various simulated sensor faults in a cement plant. The obtained results show high monitoring performance with highest robustness to false alarms and maximum fault detection sensitivity compared to conventional PCA, KPCA and other well-established RKPCA techniques. Furthermore, the unified contribution plot method demonstrates superior potentials in identifying faulty variables.

Machine Learning-Based Reduced Kernel PCA Model for Nonlinear Chemical Process Monitoring

Principal component analysis (PCA) is a popular tool for linear dimensionality reduction and fault detection. Kernel PCA (KPCA) is the nonlinear form of the PCA, which better exploits a complicated spatial structure of high-dimensional features, where a kernel function implicitly defines a nonlinear transformation into a feature space wherein standard PCA is performed. Despite its success and flexibility, conventional KPCA might not perform properly because the use of KPCA for a large-sized training dataset imposes a high computational load and a significant storage memory space since the required elements used for modelling have to be saved and used for monitoring, as well. To address this problem, a reduced KPCA (RKPCA) for fault detection of chemical processes is developed. RKPCA is a novel machine learning tool which merges dimensionality reduction, supervised learning as well as kernel selection. This novel method is used to reduce the size of recorded measurements while maintaining the most relevant data features. The removed observations, including redundant samples that are linearly correlated in the collected measurements, are described by only one sample. The obtained uncorrelated observations via PCA technique are then employed to identify the reduced KPCA model by which Hotelling $$T^2$$ and squared predictive error or Q statistics are extracted for detection purposes. Besides, their combination is also used as a detection index. The performance of the proposed process monitoring technique is illustrated through its application to Tennessee Eastman process. The obtained results demonstrate the effectiveness of the developed RKPCA technique in detecting various faults with remarkably reduced computation time and memory storage space.

Supervised process monitoring and fault diagnosis based on machine learning methods

Data-driven techniques have been receiving considerable attention in the industrial process monitoring field due to their major advantages of easy implementation and less requirement for the prior knowledge and process mechanism. Principal component analysis (PCA) method is known as a popular method for monitoring and fault detection in industrial systems but as it is basically a linear method. However, most practical systems are nonlinear. To make the extension to nonlinear systems, kernel PCA (KPCA) method has been proposed for process modeling and monitoring. We present in this paper an online reduced rank optimized KPCA (RR-KPCA) technique for fault detection in order to extend the advantages of the KPCA models to online processes. Following the fault detection, the identification of the variables correlated to the fault occurred is of great importance. For this purpose, it is proposed to extend the approaches of localization by partial PCA and by elimination in the linear case to the nonlinear case, by exploiting the solution of reduction of the dimension of the kernel matrix in the feature space. The partial RR-KPCA and the elimination sensor identification (ESI-RRKPCA) are generated based on the static RR-KPCA and the online RR-KPCA methods. The idea of these approaches is to generate partial RR-KPCA models with reduced sets of variables. In other words, their goal is to generate indices of fault detection sensitive to certain faults and insensitive to others. The proposed fault isolation methods are applied for monitoring an air quality monitoring network (AIRLOR) data.

Online fault detection and isolation of an AIR quality monitoring network based on machine learning and metaheuristic methods

The dynamic process monitoring is discussed in this paper. Kernel principal component analysis (KPCA) is a nonlinear monitoring method that cannot be applied for dynamic systems. Reduced online KPCA (OR-KPCA)is used for fault detection of dynamic processes, which is developed to built a dictionary according to the process status and then, it update the KPCA model and uses it for process monitoring. Also the Tabu search metaheuristic algorithm is used in order to determine the optimal parameter of the kernel function. In this paper, new approaches for online fault isolation, which is a challenging problem in nonlinear PCA, are formulated. An extension of partial PCA and the elimination sensor identification (ESI) to the case of nonlinear systems are presented in a feature space. The partial OR-KPCA and the elimination sensor identification (ESI-KPCA) are generated based on the OR-KPCA method and they consist of developing a set of sub-models. The sub-models are selected according to a pre-designed fault-to-residual structure matrix and by eliminating sequentially one variable from the set of the variables. The proposed fault isolation methods are applied for monitoring an air quality monitoring network. The simulation results show that the proposed fault isolation methods are effective for KPCA.

Parallel PCA–KPCA for nonlinear process monitoring

Both linear and nonlinear relationships may exist among process variables, and monitoring a process with such complex relationships among variables is imperative. However, individual principal component analysis (PCA) or kernel PCA (KPCA) may not be able to characterize these complex relationships well. This paper proposes a parallel PCA–KPCA (P-PCA–KPCA) modeling and monitoring scheme that incorporates randomized algorithm (RA) and genetic algorithm (GA) for efficient fault detection for a process with linearly correlated and nonlinearly related variables First, to determine the included variables in the parallel PCA (P-PCA) and the parallel KPCA (P-KPCA) models, GA-based optimization is performed, in which RA is used to generate faulty validation data. Second, monitoring statistics are established for the P-PCA and the P-KPCA models, in which the process status is determined. The proposed monitoring scheme discriminates the linear and nonlinear relationships among variables in a process and deals with nonlinear processes efficiently. We provide case studies on a numerical example and the continuous stirred tank reactor process. These case studies demonstrate that the proposed P-PCA–KPCA monitoring scheme is better than conventional PCA- or KPCA-based methods at performing nonlinear process monitoring.

Early Warning Technology in Drilling Muds Lost Circulation Anomaly Based on Data Analysis

To change the complex structure and the changeable working conditions of Daqing oil drilling engineering, as well as the phenomenon of misreporting for the drilling muds lost circulation warning system, in view of the early warning system of the drilling engineering in Daqing oilfield, the drilling well lost circulation fault detection method with multimode kernel principal component analysis (KPCA) based on data was proposed. First, the outlier elimination algorithmwas described in detail.The experiment verified the reliability of the adaptive determination of the length of the sliding window by using the inflection point of the elimination rate as the standard. Then, in view of the early warning system of oil drilling engineering, a new threshold classification algorithm, which can correctly classify each working condition in drilling, was proposed. Because the study object was nonlinear process, the fault detection method based on single KPCA was extended to multiple KPCA model fault detection methods which can be applied to oil drilling process. The research showed that the multimode KPCA drilling muds lost circulation detection method achieved accurate and sensitive fault detection for Daqing oilfield. It is concluded that the new fault detection method proposed in this paper can make the drilling lost circulation anomaly early warning system detect faults used in Daqing oilfield more accurately and efficiently, so as to avoid misreporting.

Fault diagnosis of bearing based on the kernel principal component analysis and optimized k-nearest neighbour model

Aiming to identify the bearing faults level effectively, a new method based on kernel principal component analysis and particle swarm optimization optimized k-nearest neighbour model is proposed. First, the gathered vibration signals are decomposed by time–frequency domain method, i.e., local mean decomposition; as a result, the product functions decomposed from the original signal are derived. Then, the entropy values of the product functions are calculated by Shannon method, which will work as the input features for k-nearest neighbour model. The kernel principal component analysis model is used to reduce the dimension of the features, and then the k-nearest neighbour model which was optimized by the particle swarm optimization method is used to identify the bearing fault levels. Case of test and actually collected signal are analysed. The results validate the effectiveness of the proposed algorithm.

Improved Statistical Fault Detection Technique and Application to Biological Phenomena Modeled by S-Systems.

In our previous work, we have demonstrated the effectiveness of the linear multiscale principal component analysis (PCA)-based moving window (MW)-generalized likelihood ratio test (GLRT) technique over the classical PCA and multiscale principal component analysis (MSPCA)-based GLRT methods. The developed fault detection algorithm provided optimal properties by maximizing the detection probability for a particular false alarm rate (FAR) with different values of windows, and however, most real systems are nonlinear, which make the linear PCA method not able to tackle the issue of non-linearity to a great extent. Thus, in this paper, first, we apply a nonlinear PCA to obtain an accurate principal component of a set of data and handle a wide range of nonlinearities using the kernel principal component analysis (KPCA) model. The KPCA is among the most popular nonlinear statistical methods. Second, we extend the MW-GLRT technique to one that utilizes exponential weights to residuals in the moving window (instead of equal weightage) as it might be able to further improve fault detection performance by reducing the FAR using exponentially weighed moving average (EWMA). The developed detection method, which is called EWMA-GLRT, provides improved properties, such as smaller missed detection and FARs and smaller average run length. The idea behind the developed EWMA-GLRT is to compute a new GLRT statistic that integrates current and previous data information in a decreasing exponential fashion giving more weight to the more recent data. This provides a more accurate estimation of the GLRT statistic and provides a stronger memory that will enable better decision making with respect to fault detection. Therefore, in this paper, a KPCA-based EWMA-GLRT method is developed and utilized in practice to improve fault detection in biological phenomena modeled by S-systems and to enhance monitoring process mean. The idea behind a KPCA-based EWMA-GLRT fault detection algorithm is to combine the advantages brought forward by the proposed EWMA-GLRT fault detection chart with the KPCA model. Thus, it is used to enhance fault detection of the Cad System in E. coli model through monitoring some of the key variables involved in this model such as enzymes, transport proteins, regulatory proteins, lysine, and cadaverine. The results demonstrate the effectiveness of the proposed KPCA-based EWMA-GLRT method over Q , GLRT, EWMA, Shewhart, and moving window-GLRT methods. The detection performance is assessed and evaluated in terms of FAR, missed detection rates, and average run length (ARL1) values.