Kernel Locality Preserving Projections Research Articles

Process Monitoring Using Kernel PCA and Kernel Density Estimation-Based SSGLR Method for Nonlinear Fault Detection

Fault monitoring is often employed for the secure functioning of industrial systems. To assess performance and enhance product quality, statistical process control (SPC) charts such as Shewhart, CUSUM, and EWMA statistics have historically been utilized. When implemented to multivariate procedures, unfortunately, such univariate control charts demonstrate low fault sensing ability. Due to some limitations of univariate charts, numerous process monitoring techniques dependent on multivariate statistical approaches such as principal component analysis (PCA) and partial least squares (PLS) have been designed. Yet, in some challenging scenarios in industrial chemical and biological processes with notably nonlinear properties, PCA works poorly, according to its presumption that the dataset generally be linear. However, Kernel Principal Component Analysis (KPCA) is a reliable and precise nonlinear process control methodology, but the interaction mainly through upper control limits (UCLs) dependent on the Gaussian distribution may weaken its output. This article introduces time-varying statistical error tracking through Kernel Principal Component Analysis (KPCA) based on Generalized Likelihood Ratio statistics (GLR) using a sequential sampling scheme named KPCA-SSGLR for nonlinear fault detection. The main issue of employing just T2 and Q statistic in KPCA is that they cannot correctly give practitioners the change point of the system fault, preventing practitioners from diagnosing the issue. Based on this perspective, this study attempts to incorporate KPCA with sequential sampling Generalized Likelihood Ratio (SSGLR) for monitoring the nonlinear fault in multivariate systems. The KPCA is utilized for dimension reduction, while the SSGLR is employed as a tracking statistic. The kernel density estimation (KDE) was employed to approximate UCLs for variational system operation relying on KPCA. The testing efficiency of the corresponding KPCA-KDE-SSGLR technique was then analyzed and competed with KPCA and kernel locality preserving projection (KLPP), the UCLs of which were focused on the Gaussian distribution. The purpose of this analysis is to enhance the development of KPCA-KDE-SSGLR to accomplish future enhancements and to advance the practical use of the established model by implementing the sequential sampling GLR approach. The fault monitoring efficiency is demonstrated through different simulation scenarios, one utilizing synthetic data, the other from the Tennessee Eastman technique, and lastly through a hot strip mill. The findings indicate the applicability of the KPCA-KDE-based SSGLR system over the KLPP and KPCA-KDE methods by its two T2 and Q charts to recognize the faults.

Fault detection and diagnosis of the air handling unit via an enhanced kernel slow feature analysis approach considering the time-wise and batch-wise dynamics

Air handling unit (AHU) is a typical special batch control process, exhibiting strong nonlinear property and two-directional dynamic characteristics which are the time-wise and batch-wise dynamic characteristics. Specifically, the time-wise dynamic characteristic corresponds to the evolution of different operating modes caused by the underlying driving forces which vary slowly in each running day (a batch run), while the batch-wise dynamic characteristic relates to the dynamic variations and deviations among different running days (batch runs). In order to further improve the AHU FDD performance through capturing the underlying driving forces of the AHU system and tackling the batch-wise dynamic property between different batch runs, in this paper, an enhanced kernel slow feature analysis (SFA) based FDD scheme is developed to detect and identify the faults of the nonlinear AHU system. Firstly, a three-way data based kernel SFA (TBKSFA) approach is proposed to detect the faults. In the proposed TBKSFA approach, the kernel trick is adopted in the SFA to sufficiently deal with the nonlinearity and the time-wise dynamic characteristic, and the multiway data analysis is employed to cope with the batch-wise dynamics among different batch runs by converting the three-way training dataset into a variable-wise unfolding two-way matrix. In addition, to handle the tough problem of nonlinearly identifying the fault pattern, a novel kernel discriminant SFA (KDSFA) model is further built by combining the kernel SFA with the discriminant analysis method. In the fault pattern diagnosis process, the proposed KDSFA is pairwisely implemented on the normal and fault datasets to calculate the fault direction, and the fault is then identified by computing the similar degrees of its own fault direction and the historical fault directions. At last, experiments and comparisons on the FDD performance of the developed approach are made using the experimental data provided by ASHRAE Research Project RP-1312. To be specific, the proposed TBKSFA based fault detection method is compared with the popular kernel principal component analysis (KPCA) method, the closely related kernel SFA method and the emerging manifold learning based kernel locality preserving projections (KLPP) method. While the developed KDSFA based fault pattern diagnosis scheme is compared with the conventional jointed angle analysis technique, the strongly linked DSFA based method and the rising artificial neural networks based long short-term memory(LSTM) classifier. Experimental results demonstrate that significant improvements can be achieved by the proposed approach compared with some other popular methods.

Surface roughness prediction using kernel locality preserving projection and Bayesian linear regression

To further improve the prediction accuracy of surface roughness in milling process, this paper presents a new two-stage feature-fusion method by combining principal component analysis (PCA) and kernel locality preserving projection (KLPP). PCA is utilized for dimension-reduction while KLPP is utilized for dimension-increment. Vibration information of the workpiece, fixture and spindle is adopted as the monitoring signal. Firstly, the commonly-used time-domain features are extracted from the vibration signals. Then, the presented two-stage feature-fusion method is carried out for extracting more effective signal features. Besides, two types of Bayesian linear regression (BLR) model (Standard_BLR and Standard_SBLR) are utilized for model construction. Before the two-stage feature-fusion, Standard_BLR is utilized to determine the optimum dimension of PCA-based fusion features and the model parameters of KLPP. After the two-stage feature-fusion, Standard_SBLR is utilized to construct the BLR-based surface roughness predictive model. Two types of milling experiment (down milling and up milling) are carried out to show the influence of the presented two-stage feature-fusion method on the predictive performance of Standard_SBLR. Experimental results show that KLPP is highly effective in improving the prediction accuracy and compressing the confidence interval (CI) of Standard_SBLR. Moreover, the comparison results show that the effectiveness of KLPP is not inferior to kernel principal component analysis (KPCA). This paper lays the foundation for accurate monitoring of surface roughness in real industrial settings.

Using Locality Preserving Projections to Improve the Performance of Kernel Clustering

Many clustering methods may have poor performance when the data structure is complex (i.e., the data has an aspheric shape or non-linear relationship). Inspired by this view, we proposed a clustering model which combines kernel function and Locality Preserving Projections (LPP) together. Specifically, we map original data into the high-dimensional feature space according to the idea of kernel function. Secondly, it is feasible to explore the local structure of data in clustering tasks. LPP is used to preserve the original local structure information of data to improve the validity of the clustering model. Finally, some outliers are often included in real data, so we embedded sparse regularization items in the model to adjust feature weights and remove outliers. In addition, we design a simple iterative optimization method to solve the final objective function and show the convergence of the optimization method in the experimental part. The experimental analysis of ten public data sets showed that our proposed method has better efficiency and performance in clustering tasks than existing clustering methods.

Object Recognition Using Non-Negative Matrix Factorization with Sparseness Constraint and Neural Network

UAVs (unmanned aerial vehicles) have been widely used in many fields, where they need to be detected and controlled. Small-sample UAV recognition requires an effective detecting and recognition method. When identifying a UAV target using the backward propagation (BP) neural network, fully connected neurons of BP neural network and the high-dimensional input features will generate too many weights for training, induce complex network structure, and poor recognition performance. In this paper, a novel recognition method based on non-negative matrix factorization (NMF) with sparseness constraint feature dimension reduction and BP neural network is proposed for the above difficulties. The Edgeboxes are used for candidate regions and Log-Gabor features are extracted in candidate target regions. In order to avoid the complexity of the matrix operation with the high-dimensional Log-Gabor features, preprocessing for feature reduction by downsampling is adopted, which makes the NMF fast and the feature discriminative. The classifier is trained by neural network with the feature of dimension reduction. The experimental results show that the method is better than the traditional methods of dimension reduction, such as PCA (principal component analysis), FLD (Fisher linear discrimination), LPP (locality preserving projection), and KLPP (kernel locality preserving projection), and can identify the UAV target quickly and accurately.

Non-destructive identification of maize haploid seeds using nonlinear analysis method based on their near-infrared spectra

The identification of maize haploid seeds is a significant process in genetic research and modern maize breeding. Adopting near-infrared spectroscopy technology to distinguish haploid seeds from hybrid seeds has the advantages of being non-destructive, rapid and low cost. However, due to the influence of light, temperature, humidity, near-infrared intensity, instrument and dynamic change of seed activity, the near-infrared spectra of maize seeds showed high dimensional nonlinear characteristics. In this study, to make full use of the class label information, a nonlinear feature analytical method for haploid maize seeds identification based on Supervised Virtual Sample Kernel Locality Preserving Projection (SVSKLPP) has been proposed. The experimental results showed that the nonlinear identification model SVSKLPP achieved strong classification performance to identify the maize haploid seeds. Moreover, compared with the linear feature extraction methods Principal Component Analysis, Orthogonal Linear Discriminant Analysis, Locality Preserving Projection, Supervised Virtual Sample Locality Preserving Projection and nonlinear feature extraction methods Kernel Locality Preserving Projection, Isomap, Locally Linear Embedding, Laplacian Eigenmaps and Local Tangent Space Alignment, the SVSKLPP model achieved a better performance. The average accuracy, sensitivity and specificity using method SVSKLPP were 97.1%, 98.8% and 95.4% respectively, and it also had high robustness. The overall results show that the SVSKLPP-NIR methodology was efficient in accurately identifying haploid maize seeds, thus demonstrating its capabilities for application in haploid breeding for crop variety improvement.

Prediction of human promoter with Least Square Support Vector Machine based on Kernel Locality Preserving Projection

The gene promoter region controls the transcription of a gene, so finding the gene promoter region is the most important step in gene regulation. Due to the huge amount and genetic diversity, although many algorithms have been proposed, the promoter recognition are rather complex with the performance still limited by low sensitivity and highly false positives. In this paper, we present a novel machine learning method for predicting promoter. First, the function motifs in different regions of Human promoter sequences have been recognized using Gaussian Mixture Model (GMM). The optimum number of GMM is given by the fuzzy cluster recognition algorithm based on fuzzy likelihood function without prior knowledge. Then the promoter sequences were mapped into the positional densities of oligonucleotides high dimension Bayes space. At last, Least Square Support Vector Machine classifier is built with Kernel Locality Preserving Projection to predict the promoter sequence, which simplifies the Least Square Support Vector Machine to form the Least Square model. Simulation results show that the performance is improved compared with other promoter classifiers and the proposed method can predict the unknown promoters with unknown similar genes in the database, and also the speed of the proposed method is significantly increased.

An integrated index for breast cancer identification using histogram of oriented gradient and kernel locality preserving projection features extracted from thermograms

Breast cancer is one of the prime causes of death in women worldwide. Thermography has shown a great potential in screening the breast cancer and overcomes the limitations of mammography. Moreover, interpretations of thermogram images are dependent on the specialists, which may lead to errors and uneven results. Preliminary screening method should detect the hazardous, destructive tumours effectively to improve the accuracy. The growth of malignant tumour can increase the internal temperature which can be captured by thermograms. Thus in this work, locally normalised histogram of oriented gradients (HOG) based preliminary screening computer aided diagnosis tool is proposed. HOG is able to record the minute internal variations in thermograms. In order to reduce the dimensions of extracted HOG descriptors kernel locality preserving projection (KLPP) is used. The resulting KLPP features are then ranked to form an efficient classification model. Various machine learning algorithms are used to validate the proposed method. Our method shows a promising performance with an average accuracy, sensitivity, specificity and area under curve of 98%, 96.66%,100% and 0.98 respectively. We have also developed a breast cancer risk index (BCRI) using significant KLPP features which can discriminate the two classes using a single integrated index. This can help the radiologists to discriminate the normal and malignant classes during screening to validate their findings.

Nonlinear process monitoring based on kernel global–local preserving projections

A new nonlinear dimensionality reduction method called kernel global–local preserving projections (KGLPP) is developed and applied for fault detection. KGLPP has the advantage of preserving global and local data structures simultaneously. The kernel principal component analysis (KPCA), which only preserves the global Euclidean structure of data, and the kernel locality preserving projections (KLPP), which only preserves the local neighborhood structure of data, are unified in the KGLPP framework. KPCA and KLPP can be easily derived from KGLPP by choosing some particular values of parameters. As a result, KGLPP is more powerful than KPCA and KLPP in capturing useful data characteristics. A KGLPP-based monitoring method is proposed for nonlinear processes. T2 and SPE statistics are constructed in the feature space for fault detection. Case studies in a nonlinear system and in the Tennessee Eastman process demonstrate that the KGLPP-based method significantly outperforms KPCA, KLPP and GLPP-based methods, in terms of higher fault detection rates and better fault sensitivity.

Application of Shearlet transform to classification of surface defects for metals

Surface defects are important factors of surface quality of industrial products. Most of the traditional machine vision based methods for surface defect recognition have some shortcomings such as low detection rate of defects and high rate of false alarms. Different types of defects have special information at some directions and scales of their images, while the traditional methods of feature extraction, such as Wavelet transform, are unable to get the information at all directions. In this study, Shearlet transform is introduced to provide efficient multi-scale directional representation, and a general framework has been developed to analyze and represent surface defect images with anisotropic information. The metal surface images captured from production lines are decomposed into multiple directional subbands with Shearlet transform, and features are extracted from all subbands and combined into a high-dimensional feature vector. Kernel Locality Preserving Projection is applied to the dimension reduction of the feature vector. The proposed method is tested with the surface images captured from different production lines, and the results show that the classification rates of surface defects of continuous casting slabs, hot-rolled steels, and aluminum sheets are 94.35%, 95.75% and 92.5% respectively.

Supervised Kernel Optimized Locality Preserving Projection with Its Application to Face Recognition and Palm Biometrics

Kernel Locality Preserving Projection (KLPP) algorithm can effectively preserve the neighborhood structure of the database using the kernel trick. We have known that supervised KLPP (SKLPP) can preserve within-class geometric structures by using label information. However, the conventional SKLPP algorithm endures the kernel selection which has significant impact on the performances of SKLPP. In order to overcome this limitation, a method named supervised kernel optimized LPP (SKOLPP) is proposed in this paper, which can maximize the class separability in kernel learning. The proposed method maps the data from the original space to a higher dimensional kernel space using a data-dependent kernel. The adaptive parameters of the data-dependent kernel are automatically calculated through optimizing an objective function. Consequently, the nonlinear features extracted by SKOLPP have larger discriminative ability compared with SKLPP and are more adaptive to the input data. Experimental results on ORL, Yale, AR, and Palmprint databases showed the effectiveness of the proposed method.

Surface Defect Recognition for Metals Based on Feature Fusion of Shearlets and Wavelets

Surface defects of metals are often with complexity and diversity. Wavelet transform is effective to detect singularities, it is applicable for recognition of point defects. As a relatively new method of multiscale geometric analysis(MGA), shearlet transform has good performance of directivity and optimal approximation, and it is applicable for recognition of defects with determined directions. A new recognition method of surface defects for metals is proposed, and it is based on feature fusion of shearlet transform and wavelet transform. Images of metals are decomposed into multiple directional sub-bands with shearlet transform and wavelet transform respectively. Then, means and variances of all sub-bands are computed and combined as a high dimensional feature vector. kernel locality preserving projection(KLPP) is applied to the high dimensional feature vector to remove redundant information between features. A low dimensional feature vector is generated and input to support vector machine(SVM) for classification of defects. Tested with samples captured from production lines of three typical metals, including high temperature slabs, medium and heavy plates and aluminium strips. Results show that classification rates of testing sets for high temperature slabs, medium and heavy plates and aluminium strips are 93.95%, 98.27% and 92.5% respectively. It is verified that the proposed method is a general algorithm of defect recognition for different types of metals.

Intelligent Credit Assessment System by Kernel Locality Preserving Projections and Manifold-Regularized SVM Models

Intelligent Credit Assessment System by Kernel Locality Preserving Projections and Manifold-Regularized SVM Models

Kernel self-optimization learning for kernel-based feature extraction and recognition

Kernel learning is becoming an important research topic in the area of machine learning, and it has wide applications in pattern recognition, computer vision, image and signal processing. Kernel learning provides a promising solution to nonlinear problems, including nonlinear feature extraction, classification and clustering. However, in kernel-based systems, the problem of the kernel function and its parameters remains to be solved. Methods of choosing parameters from a discrete set of values have been presented in previous studies, but these methods do not change the data distribution structure in the kernel-based mapping space. Accordingly, performance is not improved because the current kernel optimization does not change the data distribution. Based on this problem, this paper presents a uniform framework for kernel self-optimization with the ability to adjust the data structure. The data-dependent kernel is extended and applied to kernel learning, and optimization equations with two criteria for measuring data discrimination are used to solve the optimal parameter values. Some experiments are performed to evaluate the performance in popular kernel learning methods, including kernel principal components analysis (KPCA), kernel discriminant analysis (KDA) and kernel locality-preserving projection (KLPP). These evaluations show that the framework of kernel self-optimization is feasible for enhancing kernel-based learning methods.

Manifold subspace distance derived from kernel principal angles and its application to machinery structural damage assessment

Damage assessment of machinery structure is beneficial for identifying structural health states and preventing sudden failures. A novel scheme for damage assessment is presented by using manifold subspace distance in this study. Vibration response signals from the machinery structure are collected by accelerometers first, and feature matrices are extracted to characterize the acceleration response comprehensively. Thereafter, a manifold learning method, namely kernel locality preserving projection (KLPP), is performed to obtain manifold subspace from the feature matrix. KLPP is available to mine nonlinear information hidden in the feature matrix, which makes it different from the linear subspace analysis method. This merit enables KLPP to be more effective in exploring the intrinsic model of the feature matrix. Further, kernel principal angles that represent similarity between the manifold subspaces are calculated. Finally, a manifold subspace distance is derived from the kernel principal angles. This distance is an appropriate metric for measuring closeness or similarity between the subspaces embedded on a manifold. The distance between manifold subspaces from normal state and damage state of a machinery structure is defined as a damage index. Effectiveness of the proposed scheme is validated by two case studies with regard to damage assessment for different machinery structures. The results show that the defined damage index is not only sensitive to the occurrence of structural damage but also increases obviously with the increasing damage level. These positive results illustrate that the proposed scheme has promise for future performance and is a valuable method for damage assessment.

Surface Detection of Continuous Casting Slabs Based on Curvelet Transform and Kernel Locality Preserving Projections

Longitudinal cracks are common defects of continuous casting slabs and may lead to serious quality accidents. Image capturing and recognition of hot slabs is an effective way for on-line detection of cracks, and recognition of cracks is essential because the surface of hot slabs is very complicated. In order to detect the surface longitudinal cracks of the slabs, a new feature extraction method based on Curvelet transform and kernel locality preserving projections (KLPP) is proposed. First, sample images are decomposed into three levels by Curvelet transform. Second, Fourier transform is applied to all sub-band images and the Fourier amplitude spectrum of each sub-band is computed to get features with translational invariance. Third, five kinds of statistical features of the Fourier amplitude spectrum are computed and combined in different forms. Then, KLPP is employed for dimensionality reduction of the obtained 62 types of high-dimensional combined features. Finally, a support vector machine (SVM) is used for sample set classification. Experiments with samples from a real production line of continuous casting slabs show that the algorithm is effective to detect longitudinal cracks, and the classification rate is 91.89%.

Sparse Kernel Locality Preserving Projection and Its Application in Nonlinear Process Fault Detection

Locality preserving projection (LPP) is a newly emerging fault detection method which can discover local manifold structure of a data set to be analyzed, but its linear assumption may lead to monitoring performance degradation for complicated nonlinear industrial processes. In this paper, an improved LPP method, referred to as sparse kernel locality preserving projection (SKLPP) is proposed for nonlinear process fault detection. Based on the LPP model, kernel trick is applied to construct nonlinear kernel model. Furthermore, for reducing the computational complexity of kernel model, feature samples selection technique is adopted to make the kernel LPP model sparse. Lastly, two monitoring statistics of SKLPP model are built to detect process faults. Simulations on a continuous stirred tank reactor (CSTR) system show that SKLPP is more effective than LPP in terms of fault detection performance.

Application of multi-scale feature extraction to surface defect classification of hot-rolled steels

Feature extraction is essential to the classification of surface defect images. The defects of hot-rolled steels distribute in different directions. Therefore, the methods of multi-scale geometric analysis (MGA) were employed to decompose the image into several directional subbands at several scales. Then, the statistical features of each subband were calculated to produce a high-dimensional feature vector, which was reduced to a lower-dimensional vector by graph embedding algorithms. Finally, support vector machine (SVM) was used for defect classification. The multi-scale feature extraction method was implemented via curvelet transform and kernel locality preserving projections (KLPP). Experiment results show that the proposed method is effective for classifying the surface defects of hot-rolled steels and the total classification rate is up to 97.33%.

Full-viewpoint 3D Space Object Recognition Based on Kernel Locality Preserving Projections

Space object recognition plays an important role in spatial exploitation and surveillance, followed by two main problems: lacking of data and drastic changes in viewpoints. In this article, firstly, we build a three-dimensional (3D) satellites dataset named BUAA Satellite Image Dataset (BUAA-SID 1.0) to supply data for 3D space object research. Then, based on the dataset, we propose to recognize full-viewpoint 3D space objects based on kernel locality preserving projections (KLPP). To obtain more accurate and separable description of the objects, firstly, we build feature vectors employing moment invariants, Fourier descriptors, region covariance and histogram of oriented gradients. Then, we map the features into kernel space followed by dimensionality reduction using KLPP to obtain the submanifold of the features. At last, k-nearest neighbor (kNN) is used to accomplish the classification. Experimental results show that the proposed approach is more appropriate for space object recognition mainly considering changes of viewpoints. Encouraging recognition rate could be obtained based on images in BUAA-SID 1.0, and the highest recognition result could achieve 95.87%.

Automatic Recognition Method of Surface Defects Based on Gabor Wavelet and Kernel Locality Preserving Projections

Principles of Gabor wavelet transform and kernel locality preserving projections(KLPP) are studied and characteristics of surface defects on hot-rolled steel plates are analyzed.A feature extraction method based on Gabor wavelet and KLPP is presented and applied to automatic recognition of hot-rolled steel plate surface defects.Surface images is decomposed into 40 complex-value components at 5 scales and 8 orientations by Gabor wavelet transform,then means and standard deviations of real parts and imaginary parts of the components and the original image are calculated as features respectively to produce a feature vector with 162 dimensions,which is then reduced to 21 dimensions by KLPP.A multi-layer perceptron classifier is constructed to classify the samples with the 21-dimensional feature vector.The feature extraction method presented in this paper has low computational complexity,high computational parallelism,and can discriminate edges and textures along different directions.The method is examined with samples of surface defects captured from a hot-rolled steel plate production line,and the classification rate is 93.87%.