Anomaly Detection Based on Kernel Principal Component and Principal Component Analysis

Feature extraction methods such as Kernel Principal Component Analysis (KPCA) and Principal Component Analysis (PCA), are often used for soft sensor modeling in industrial process with high dimensional data. A kind of soft sensor method based on Integrated Principal Component Analysis (Integrated PCA) is proposed for some weakness of KPCA and that of PCA. This approach combines nonlinear information extracted by KPCA with linear information extracted by PCA and it can not only reduce the dimensionality of input data, but also make full use of linear and nonlinear information. Partial Least Squares (PLS) is used to obtain the final soft sensor model and Particle Swarm Optimization (PSO) is applied to get the optimal parameters of Integrated PCA and those of KPCA. Finally, the proposed method is applied to build soft sensor models of diesel oil boiling point and other industrial objects and is proved to have better ability of generalization by being compared with other feature extraction methods.

Nonlinear process monitoring using kernel principal component analysis

Searching influence variables as well as forecasting the flowing of graduate employment is an ongoing activity of considerable significance. But the forecasting is complex due to the time series and complex factor inputs. The neural network method has been successfully employed to solve the multi factors problem. However the forecasting result is not ideal due to the nonlinearity and noise. In this work, a neural network model is presented by combining Recurrent Neural Network (RNN) with Kernel Principal Component Analysis (KPCA). And then try to forecast the area flowing of graduate employment using this model. In the model, RNN with Kernel Principal Component Analysis (KPCA) and Principal Component Analysis (PCA) as the feature extraction is introduced in as comparison. And then by an empirical study with actual data from some high school of China, it is shown that the proposed methods can both achieve good forecasting performance comparing with NN method. And the KPCA method performs better than the PCA method.

A method combined Kernel Principal Component Analysis (KPCA) with BP neural network is proposed for multispectral remote sensing image classification in this paper. Firstly, the KPCA transformation including Gaussian KPCA and polynomial KPCA is carried out to get the former three uncorrelated bands containing most information of the TM images with seven bands. Secondly, BP neural network classification is executed using the three bands data after KPCA transformation. For testifying, both the classical PC A and the KPCA are applied to the multispectral Landsat TM data for feature extraction. The results demonstrate that the method proposed in this paper can improve the classification accuracy compared with that of principal component analysis (PCA) and BP neural network. Keywords: Kernel principal component analysis, BP neural network, Multispectral remote sensing, Classification 1. INTRODUCTION Feature reduction in a remote sensing dataset is often desirable to decrease the processing time required to perform a classification. Principal component analysis (PCA) is a common method for image enhancement and compression. PCA maximizes the projection variance in the previous r vectors (r refers to the number of dimension which need to be reduced) according to search this r orthogonal eigenvectors (corresponding to previous r maximal eigenvalues) [1]. However, PCA method is a linear mapping algorithm in nature; it only extracts the linear features but loss of the non-linear features. Therefore, kernel principal component analysis (KPCA) is been put forward to deal with the non-linear problems in some re ferences [2, 3]. Recently, kernel-based learning algorithms which had been proved to be a promising method for tackling nonlinear systems have attracted much attention of researcher s in the field of machines learning. KPCA is applied to many fields including failure detection in waste water treatment plants [4–6], data denoising [7], recognition of ha ndwritten digits [8] and classification of genetic data [9] an so on. In recent years, there are some papers using KPCA method for feature extraction in image processing. A kernel machine-based discriminant analysis method presented by Juwei Lu et al deals with the nonlinearity of the face patterns' distribution [11]. The application of KPCA for dimension reduction on remote sensin g datasets with inherent non-linear structure was present by John Tan et al [12]. A method combined KPCA and SAM provided by Zhang Youjing et al. has been shown to yield high classification accuracy for city’s vegetation [13]. KPCA feature extraction based on Mahalanobis distance Fuzzy C-Means genetic algorithm provided by Chang Ruichun has been shown to yield high classification accuracy in extracting desertification nonlinear feature [14].

It is hard to search the influence variables and to forecast the flowing of graduate employment due to the time series and complex factor inputs. Recently the neural network method has been successfully employed to solve the problem. However the forecasting result is not ideal due to the nonlinearity and noise. In this work, by combining Recurrent Neural Network (RNN) with Kernel Principal Component Analysis (KPCA), a KRNN model is presented, based on which, the area flowing of graduate employment is tried to be forecasted, and both the complex factor problem and time series problem has been dealt with. In the model, RNN with Kernel Principal Component Analysis (KPCA) and Principal Component Analysis (PCA) as the feature extraction is introduced in as comparison. And then by an empirical study with actual data from some high school of China, it is shown that the proposed methods can both achieve good forecasting performance comparing with NN method. And the KPCA method performs better than the PCA method.

In view of the fact that the production process of Polyvinyl chloride (PVC) polymerization has more fault types and its type is complex, a fault diagnosis algorithm based on the hybrid Dynamic Kernel Principal Component Analysis-Fisher Discriminant Analysis (DKPCA-FDA) method is proposed in this paper. Kernel principal component analysis and Dynamic Kernel Principal Component Analysis are used for fault diagnosis of Polyvinyl chloride (PVC) polymerization process, while Fisher Discriminant Analysis (FDA) method was adopted to make failure data for further separation. The simulation results show that the Dynamic Kernel Principal Component Analyses to fault diagnosis of Polyvinyl chloride (PVC) polymerization process have better diagnostic accuracy, the Fisher Discriminant Analysis (FDA) can further realize the fault isolation, and the actual fault in the process of Polyvinyl chloride (PVC) polymerization production can be monitored by Dynamic Kernel Principal Component Analysis.

In the last decade, the effectiveness of kernel-based methods for object detection and recognition have been reported Fukui et al. (2006); Hotta (2008c); Kim et al. (2002); Pontil & Verri (1998); Shawe-Taylor & Cristianini (2004); Yang (2002). In particular, Kernel Principal Component Analysis (KPCA) took the place of traditional linear PCA as the first feature extraction step in various researches and applications. KPCA can cope with non-linear variations well. However, KPCAmust solve the eigen value problem with the number of samples × the number of samples. In addition, the computation of kernel functions with all training samples are required to map a test sample to the subspace obtained by KPCA. Therefore, the computational cost is the main drawback. To reduce the computational cost of KPCA, sparse KPCA Tipping (2001) and the use of clustering Ichino et al. (2007 (in Japanese) were proposed. Ichino et al. Ichino et al. (2007 (in Japanese) reported that KPCA of cluster centers is more effective than sparse KPCA. However, the computational cost becomes a big problem again when the number of classes is large and each class has one subspace. For example, KPCA of visual words (cluster centers of local features) Hotta (2008b) was effective for object categorization but the computational cost is high. In this method, each category of 101 categories has one subspace constructed by 400 visual words. Namely, 40, 400 (= 101 categorizes × 400 visual words) kernel computations are required to map a local feature to all subspaces. On the other hand, traditional linear PCA is independent of the number of samples when the dimension of a feature is smaller than the number of samples. This is because the size of eigen value problem depends on the minimum number of the feature dimension and the number of samples. To map a test sample to a subspace, only inner products between basis vectors and the test sample are required. Therefore, in general, the computational cost of linear PCA is much lower than KPCA. In this paper, we propose how to use non-linearity of KPCA and computational cost of linear PCA simultaneously Hotta (2008a). Kernel-based methods map training samples to high dimensional space as x → φ(x). Nonlinearity is realized by linear method in high dimensional space. The dimension of mapped feature space of the Radial Basis Function (RBF) kernel becomes infinity, and we can not describe the mapped feature explicitly. However, the mapped feature φ(x) of the polynomial kernel can be described explicitly. This means that KPCA with the polynomial kernel can be solved directly by linear PCA of mapped features. Unfortunately, in general, the dimension of mapped features is too high to solve by linear PCA even if the polynomial kernel with 2nd degrees K(x, y) = (1+ xTy)2 is used. The dimension of mapped features of the polynomial 5

It is hard to search the influence variables and to classify the flowing areas of graduate employment due to the complex factor inputs. Recently the neural network method has been successfully employed to solve the problem. However the classification result is not ideal due to the nonlinearity and noise. In this work, by combining Recurrent Neural Network (RNN) with Kernel Principal Component Analysis (KPCA), a KRNN model is presented, based on which, the flowing areas of graduate employment is tried to be classified, and the complex factor problem has been well dealt with. In the model, RNN with Kernel Principal Component Analysis (KPCA) and Principal Component Analysis (PCA) as the feature extraction is introduced in as comparison. And then by an empirical study with actual data, it is shown that the proposed methods can both achieve good classification performance comparing with NN method. And the Kernel Principal Component Analysis method performs better than the Principal Component Analysis method.

For the complexity of the automation system roll on, sensors should have to be more intelligent. Recently, Neural Network is widely used to intelligentize sensors for its well performance on capturing the information of the data. But due to its intrinsic linear character, it doesn't perform well in nonlinear data processing. In this paper, RNN with Kernel Principal Component Analysis (KPCA) and Principal Component Analysis (PCA) as the feature extraction is introduced in as comparison. And then an experimental system is set up with pressure sensor. By examining the data of the example, it is shown that the proposed methods can both achieve good performance comparing with NN method. And the KPCA method performs better than the PCA method.

In order to monitor nonlinear production process effectively, multivariate statistical process control based on kernel principal component analysis is applied to process monitoring and diagnosis. Squared prediction error (SPE) statistic of the kernel principal component analysis (KPCA) model is used for process monitoring, and the fault causes of the production process could be tracked by the methods of data reconstruction and the optimal neighbor selection strategy. Simulation data and Tennessee Eastman process data are used for model validation, as a result the proposed method has better performance on abnormality detecting, compared with multivariate statistical process control based on linear principal component analysis. What is more, the causes of the faults are tracked effectively, thus the production process can be adjusted to prevent substandard products.

Principal component analysis (PCA) and kernel principal component analysis (KPCA) are classical feature extraction methods. However, PCA and KPCA are unsupervised learning methods which always maximize the overall variance and ignore the information of within-class and between-class. In this paper, we propose a simple yet effective strategy to improve the performance of PCA and then this strategy is generalized to KPCA. The proposed methods utilize within-class auxiliary training samples, which are constructed through linear interpolation method. These within-class auxiliary training samples are used to train and get the principal components. In contrast with conventional PCA and KPCA, our proposed methods have more discriminant information. Several experiments are respectively conducted on XM2VTS face database, United States Postal Service (USPS) handwritten digits database and three UCI repository of machine learning databases, experimental results illustrate the effectiveness of the proposed method.

Global increases in the occurrence and frequency of flood have highlighted the need for resilience approaches to deal with future floods. The principal component analysis (PCA) has been used widely to understand the resilience of the urban system to floods. Based on feature extraction and dimensionality reduction, the PCA reduces datasets to representations consisting of principal components. Kernel PCA (KPCA) is the nonlinear form of PCA, which efficiently presents a complicated data in a lower dimensional space. In this work the KPCA techniques was applied to measure and map flood resilience across a local level. Therefore, it aims to improve the performance achieved by non-linear PCA application, compared to standard PCA. Twenty-one resilience indicators were gathered, including social, economic, physical, and natural components into a composite index (Flood resilience Index). The experimental results demonstrate the KPCA performance to get a better Flood Resilience Index, guiding q decision making to strengthen the flood resilience in our case of study of M’diq-Fnideq and martil municipalities in Northern of Morocco.

The identification of between-group differences and changes in gait mechanics are useful for injury prevention. Previous studies suggest the differences in gait biomechanical variables may interact in a complex non-linear fashion rather than a simple linear fashion. A non-linear multivariate analysis technique is therefore necessary to unravel the inherent structure in the gait data. Kernel principal component analysis (KPCA) is an extension of principal component analysis (PCA), the most widely used method in this field, and can provide inside into non-linear relationships of the variables of interest. Despite the growing use of PCA in running gait research, no prior studies have evaluated the utility of KPCA in determining the between-group differences in running gait patterns. Therefore, the objective of this study was to compare the performance of KPCA and PCA in identifying differences between male and female runners and as well between young and older runners in running gait kinematics. Running kinematic data were analysed on a gender group (female, n = 50; male, n = 50) and an age group (young, n = 50; older, n = 50) for discrete variables and waveform data. The following key results were obtained: (1) the performance of the KPCA for the identification of gender and age differences in running gait kinematics increased as compared to using the linear PCA; (2) the Gaussian function often performed better than the polynomial function for these two experiments; (3) there was no consistent optimal value of either the width σ of the Gaussian kernel or the degree d of the polynomial kernel for different data types and experiments. These results suggest that non-linear features extracted by KPCA could be potentially useful in identifying and discriminating between-group differences and changes in running gait patterns, and could provide useful information about the intrinsic non-linear dynamics of running movement.

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

Statistical control charts are essential to ensure both safety and efficient operation of many industrial processes. Many dimensionality reduction techniques such as principal component analysis (PCA) and Partial Least Squares (PLS) regression exist, and are often employed for modeling purposes as they are relatively easy to compute. However, these techniques are only effective for modeling and monitoring linear processes. The Kernel Principal Component Analysis (KPCA) method is an extension of PCA that helps deal with any nonlinearities in the process data. However, KPCA-based fault detection methods may result in a higher false alarm rate than the conventional method. In this paper, an improved KPCA method is developed in order to tackle the issue of high false alarm rates, by utilizing a mean filter to smoothen the detection statistics that are obtained from the KPCA method. The advantages presented by the developed method are illustrated using a simulated nonlinear model. The results clearly show that the improved KPCA method provides improved fault detection results with low missed detection and false alarm rates, and smaller ARL1 values compared to the conventional methods.