Kernel Principal Component Analysis Feature Research Articles

Application of kernel principal component analysis for optical vector atomic magnetometry

Vector atomic magnetometers that incorporate electromagnetically induced transparency (EIT) allow for precision measurements of magnetic fields that are sensitive to the directionality of the observed field by virtue of fundamental physics. However, a practical methodology of accurately recovering the longitudinal angle of the local field through observations of EIT spectra has not been established. In this work, we address this problem of angle determination with an unsupervised machine learning algorithm utilizing nonlinear dimensionality reduction. The proposed algorithm was developed to interface with spectroscopic measurements from an EIT-based atomic rubidium magnetometer and uses kernel principal component analysis (KPCA) as an unsupervised feature extraction tool. The resulting KPCA features allow each EIT spectrum measurement to be represented by a single coordinate in a new reduced dimensional feature space, thereby streamlining the process of angle determination. A supervised support vector regression (SVR) machine was implemented to model the resulting relationship between the KPCA projections and field direction. If the magnetometer is configured so that the azimuthal angle of the field is defined with a polarization lock, the KPCA-SVR algorithm is capable of predicting the longitudinal angle of the local magnetic field within 1 degree of accuracy and the magnitude of the absolute field with a resolution of 70 nT. The combined scalar and angular sensitivity of this method make the KPCA-enabled EIT magnetometer competitive with conventional vector magnetometry methods. © 2023. All rights reserved.

Target Recognition of SAR Images Based on SVM and KSRC.

A synthetic aperture radar (SAR) target recognition method combining linear and nonlinear feature extraction and classifiers is proposed. The principal component analysis (PCA) and kernel PCA (KPCA) are used to extract feature vectors of the original SAR image, respectively, which are classical and reliable feature extraction algorithms. In addition, KPCA can effectively make up for the weak linear description ability of PCA. Afterwards, support vector machine (SVM) and kernel sparse representation-based classification (KSRC) are used to classify the KPCA and PCA feature vectors, respectively. Similar to the idea of feature extraction, KSRC mainly introduces kernel functions to improve the processing and classification capabilities of nonlinear data. Through the combination of linear and nonlinear features and classifiers, the internal data structure of SAR images and the correspondence between test and training samples can be better investigated. In the experiment, the performance of the proposed method is tested based on the MSTAR dataset. The results show the effectiveness and robustness of the proposed method.

LightGBM Indoor Positioning Method Based on Merged Wi-Fi and Image Fingerprints.

Smartphones are increasingly becoming an efficient platform for solving indoor positioning problems. Fingerprint-based positioning methods are popular because of the wide deployment of wireless local area networks in indoor environments and the lack of model propagation paths. However, Wi-Fi fingerprint information is singular, and its positioning accuracy is typically 2–10 m; thus, it struggles to meet the requirements of high-precision indoor positioning. Therefore, this paper proposes a positioning algorithm that combines Wi-Fi fingerprints and visual information to generate fingerprints. The algorithm involves two steps: merged-fingerprint generation and fingerprint positioning. In the merged-fingerprint generation stage, the kernel principal component analysis feature of the Wi-Fi fingerprint and the local binary pattern features of the scene image are fused. In the fingerprint positioning stage, a light gradient boosting machine (LightGBM) is trained with mutually exclusive feature bundling and histogram optimization to obtain an accurate positioning model. The method is tested in an actual environment. The experimental results show that the positioning accuracy of the LightGBM method is 90% within a range of 1.53 m. Compared with the single-fingerprint positioning method, the accuracy is improved by more than 20%, and the performance is improved by more than 15% compared with other methods. The average locating error is 0.78 m.

Derivation of a Dynamic Market Risk Signal Using Kernel PCA and Machine Learning

Kernel principal component analysis (PCA) is an extension of the conventional PCA method that employs a kernel transformation whereby hidden patterns in possibly multidimensional data may be detected and extracted more explicitly. In this article, the author applies the method of kernel PCA to a currency prediction case study and derives an aggregate market signal. It is observed that this signal has desirable information-compression properties and may be used as a predictive risk indicator in the return prediction models. Used alongside common drivers of exchange rates, a kernel PCA signal enhances in-sample and out-of-sample risk-adjusted performance across a range of machine learning strategies. In particular, the author observes that a kernel PCA signal remains robust and predictive during volatile market conditions. The kernel PCA signal may be used as a machine learning feature to inform and support data-driven risk management strategies. TOPICS:Big data/machine learning, derivatives Key Findings • This article develops a risk signal based on the method of kernel PCA. • Kernel PCA conveys desirable information on market dynamics. • Currency machine learning strategies benefit from this kernel PCA feature.

A combination of CSP-based method with soft margin SVM classifier and generalized RBF kernel for imagery-based brain computer interface applications

Several methods utilizing common spatial pattern (CSP) algorithm have been presented for improving the identification of imagery movement patterns for brain computer interface applications. The present study focuses on improving a CSP-based algorithm for detecting the motor imagery movement patterns. A discriminative filter bank of CSP method using a discriminative sensitive learning vector quantization (DFBCSP-DSLVQ) system is implemented. Four algorithms are then combined to form three methods for improving the efficiency of the DFBCSP-DSLVQ method, namely the kernel linear discriminant analysis (KLDA), the kernel principal component analysis (KPCA), the soft margin support vector machine (SSVM) classifier and the generalized radial bases functions (GRBF) kernel. The GRBF is used as a kernel for the KLDA, the KPCA feature selection algorithms and the SSVM classifier. In addition, three types of classifiers, namely K-nearest neighbor (K-NN), neural network (NN) and traditional support vector machine (SVM), are employed to evaluate the efficiency of the classifiers. Results show that the best algorithm is the combination of the DFBCSP-DSLVQ method using the SSVM classifier with GRBF kernel (SSVM-GRBF), in which the best average accuracy, attained are 92.70% and 83.21%, respectively. Results of the Repeated Measures ANOVA shows the statistically significant dominance of this method at p < 0.05. The presented algorithms are then compared with the base algorithm of this study i.e. the DFBCSP-DSLVQ with the SVM-RBF classifier. It is concluded that the algorithms, which are based on the SSVM-GRBF classifier and the KLDA with the SSVM-GRBF classifiers give sufficient accuracy and reliable results.

Hyperspectral Image Classification Based on Two-Stage Subspace Projection

Hyperspectral image (HSI) classification is a widely used application to provide important information of land covers. Each pixel of an HSI has hundreds of spectral bands, which are often considered as features. However, some features are highly correlated and nonlinear. To address these problems, we propose a new discrimination analysis framework for HSI classification based on the Two-stage Subspace Projection (TwoSP) in this paper. First, the proposed framework projects the original feature data into a higher-dimensional feature subspace by exploiting the kernel principal component analysis (KPCA). Then, a novel discrimination-information based locality preserving projection (DLPP) method is applied to the preceding KPCA feature data. Finally, an optimal low-dimensional feature space is constructed for the subsequent HSI classification. The main contributions of the proposed TwoSP method are twofold: (1) the discrimination information is utilized to minimize the within-class distance in a small neighborhood, and (2) the subspace found by TwoSP separates the samples more than they would be if DLPP was directly applied to the original HSI data. Experimental results on two real-world HSI datasets demonstrate the effectiveness of the proposed TwoSP method in terms of classification accuracy.

Recognition of mixture control chart patterns based on fusion feature reduction and fireworks algorithm-optimized MSVM

Unnatural control chart patterns (CCPs) can be associated with the quality problems of the production process. It is quite critical to detect and identify these patterns effectively based on process data. Various machine learning techniques to CCPs recognition have been studied on the process only suffer from basic CCPs of unnatural patterns. Practical production process data may be the combination of two or more basic patterns simultaneously in reality. This paper proposes a mixture CCPs recognition method based on fusion feature reduction (FFR) and fireworks algorithm-optimized multiclass support vector machine (MSVM). FFR algorithm consists of three main sub-networks: statistical and shape features, features fusion and kernel principal component analysis feature dimensionality reduction, which make the features more effective. In MSVM classifier algorithm, the kernel function parameters play a very significant role in mixture CCPs recognition accuracy. Therefore, fireworks algorithm is proposed to select the two-dimensional parameters of the classifier. The results of the proposed algorithm are benchmarked with popular genetic algorithm and particle swarm optimization methods. Simulation results demonstrate that the proposed method can gain the higher recognition accuracy and significantly reduce the running time.

Kernel PCA feature extraction and the SVM classification algorithm for multiple-status, through-wall, human being detection

Ultra-wideband (UWB) radar with strong anti-jamming performance and high-range resolution can be used to separate multiple human targets in a complex environment. In recent years, through-wall human being detection with UWB radar has become relatively sophisticated. In this paper, the method of kernel principal component analysis (KPCA) feature extraction and the support vector machine (SVM) classification algorithm are applied to identify and classify the multiple statuses of through-wall human being detection. This method makes full use of the KPCA of powerful, nonlinear feature extraction and SVMs, which can solve the problem of multiple-status detection and nonlinear pattern recognition. The experimental data that come from KPCA feature extraction are used as input to the SVM classification algorithm, some of which are used to train the model and the others to test the model. Experimental results showed that KPCA feature extraction and the SVM classification algorithm effectively distinguished four statuses of through-wall human being detection and achieved the desired results.

Protein Tertiary Structure Classification based on its Physicochemical property using Neural Network and KPCA-SVM: A Comparative Study

Proteins are one of the most important molecules in living organisms so they play a vital structural role in the cells of living organism. They are constructed of several polypeptide chains of amino acids, which fold into complex tertiary Structure. The knowledge of the protein function is directly dependent on its three dimensional (tertiary) structure. The Physicochemical properties of proteins always guide to determine the quality of the protein tertiary structure. Therefore it has been rigorously used to distinguish native or native like structure from other predicted structure. The experiments were conducted on the CASP dataset to classify RMSD (target class) near to native protein tertiary structure or not. Kernel principal component analysis (KPCA) is used for feature extraction since it performs better than PCA on protein tertiary structure dataset due to their nonlinear structures. The proposed model compare with neural network classification method. The experiments conducted shows that support vector machine combined with KPCA feature extraction performs better than neural network classifier. More than our results show better performance in Gaussian KPCA feature extraction with respect to other kernels.

A KPCA texture feature model for efficient segmentation of RADARSAT-2 SAR sea ice imagery

Sea ice information obtained from synthetic aperture radar (SAR) images is crucial for ensuring safe marine navigation and supporting climate change studies in polar regions. We propose a kernel principal component analysis (KPCA) local texture feature model for efficient sea ice segmentation. The proposed KPCA texture feature model is significant for several reasons. First, it takes into account the multiplicative nature of SAR speckle noise. The resulting KPCA features therefore assume independent and identically distributed (i.i.d.) Gaussian-like noise, which satisfies the assumptions inherent in classical statistical models such as the k-means algorithm. Second, the KPCA features are compact and statistically independent and therefore capable of reducing data redundancy. Third, the KPCA texture features are discriminative. Finally, the extraction of KPCA features requires less computation. Based on the KPCA model, a segmentation scheme is implemented in four steps. First, local patch-based texture descriptors are extracted from the image. Second, KPCA is performed on the patch-based texture descriptors to obtain compact and discriminative texture features with Gaussian-like noise characteristics. Third, the k-means algorithm is performed on extracted KPCA features to segment the image. Finally, a neighbourhood-based majority voting scheme is employed to determine the final label of each pixel. We compared the proposed method with several other popular sea ice segmentation approaches. The results demonstrated that our method is robust to speckle noise level, is accurate and fast and may thus better support the operational segmentation of sea ice in SAR imagery.

Data-driven fault detection with process topology for fault identification

In this paper a fault diagnosis framework based on detection with feature extraction methods and identification based on data-driven process topology methods was investigated. A simulation of a simple system consisting of two tanks with heat exchangers was used to generate data for normal operating conditions and a number of faults. Fault detection methods included principal component analysis and kernel principal component analysis feature extraction with Shewhart, cumulative sum and exponentially weighted moving average monitoring charts. Process topology information was extracted with linear cross-correlation, partial cross-correlation and transfer entropy. Connectivity maps were constructed to identify possible fault propagation paths to aid root cause analysis and changes in connectivity structure due to faults were exploited for fault identification. Kernel principal component analysis with a CUSUM chart gave the best detection performance, while connectivity graphs based on partial correlation gave an accurate representation of the system and assisted fault identification.

An efficient KPCA algorithm based on feature correlation evaluation

Classic kernel principal component analysis (KPCA) is less computationally efficient when extracting features from large data sets. In this paper, we propose an algorithm, that is, efficient KPCA (EKPCA), that enhances the computational efficiency of KPCA by using a linear combination of a small portion of training samples, referred to as basic patterns, to approximately express the KPCA feature extractor, that is, the eigenvector of the covariance matrix in the feature extraction. We show that the feature correlation (i.e., the correlation between different feature components) can be evaluated by the cosine distance between the kernel vectors, which are the column vectors in the kernel matrix. The proposed algorithm can be easily implemented. It first uses feature correlation evaluation to determine the basic patterns and then uses these to reconstruct the KPCA model, perform feature extraction, and classify the test samples. Since there are usually many fewer basic patterns than training samples, EKPCA feature extraction is much more computationally efficient than that of KPCA. Experimental results on several benchmark data sets show that EKPCA is much faster than KPCA while achieving similar classification performance.

A Bayesian Network Method for Automatic Classification of Eddy Current NDE Signals

Eddy current nondestructive evaluation (ECNDE) techniques are widely used in structural integrity and health monitoring. A novel algorithm was proposed for characterizing eddy current (EC) signals. In scanning inspection, the EC signals responding to impedance change were pre-processed for noise elimination and feature extraction. After feature extraction, Bayesian networks (BNs) were carried out to classify EC signals. It is shown by extensive experiments that kernel principal component analysis (KPCA) is better than principal component analysis (PCA) for feature extraction. The methods using BNs by KPCA feature extraction can perform better than the other classification methods.

Kernel-based representation for 2D/3D motion trajectory retrieval and classification

This paper proposes a novel kernel-space representation for motion trajectories. Contrasted to most trajectory representation methods in the literature, our method is more generic in the sense that it can be applied to either 2D or 3D trajectories. In the proposed method, a trajectory is firstly projected by the Kernel Principal Component Analysis (KPCA) which can be considered as an implicit mapping to a much higher-dimensional feature space. The high dimensionality can effectively improve the accuracy in recognizing motion trajectories. Then, Nonparametric Discriminant Analysis (NDA) is used to extract the most discriminative features from the KPCA feature space. The synergistic effect of KPCA and NDA leads to better class separability and makes the proposed trajectory representation a more powerful discriminator. The experimental validation of the proposed method is conducted on the Australian Sign Language (ASL) data set. The results show that our method performs significantly better, in both trajectory classification and retrieval, than the state-of-the-art techniques.

A Semi-Supervised SAR Target Recognition Based on Laplacian Regularized Least Squares Classification

提出了一种基于核主成分分析(kernel principal component analysis,简称KPCA)和拉普拉斯正则化最小二乘(Laplacian regularized least squares,简称LapRLS)的合成孔径雷达(synthetic aperture radar,简称SAR)目标识别方法.KPCA特征提取方法不仅能够提取目标主要特征,而且有效地降低了特征维数.Laplacian正则化最小二乘分类是一种半监督学习方法,将训练集样本作为有标识样本,测试集样本作为无标识样本,在学习过程中将测试集样本包含进来以获得更高的识别率.在MSTAR实测SAR地面目标数据上进行实验,结果表明,该方法具有较高的识别率,并对目标角度间隔具有鲁棒性.与模板匹配法、支撑矢量机以及正则化最小二乘监督学习方法相比,具有更高的SAR目标识别正确率.此外,还通过实验分析了不同情况下有标识样本数目对目标识别性能的影响.;A Synthetic Aperture Radar (SAR) target recognition approach based on KPCA (kernel principal component analysis) and Laplacian regularized least squares classification is proposed. KPCA feature extraction method can not only extract the main characteristics of target, but also reduce the input dimension effectively. Laplacian regularized least squares classification is a semi-supervised learning method. In the target recognition process, training set is treated as labeled samples and test set as unlabeled samples. Since the test samples are considered in the learning process, high recognition accuracy is obtained. Experimental results on MSTAR (moving and stationary target acquisition and recognition) SAR datasets show its good performance and robustness to azimuth interval. Compared with template matching, support vector machine and regularized least squares learning method, the proposed method gets more SAR target recognition accuracy. In addition, the effect of the number of labeled points on target identification performance is analyzed at different conditions.

Automatic classification of eddy current signals based on kernel methods

Eddy current nondestructive evaluation techniques are widely used in structural integrity and health monitoring. A novel algorithm based on kernel methods was proposed for characterising eddy current (EC) signals. In scanning inspection, the EC signals responding to the impedance change were pre-processed for noise elimination using the wavelet packet analysis method. Then, the Morlet wavelet was employed to perform the decomposition of one-dimension differential signals onto the coefficients of the wavelet transforms at different scales as input to kernel principal component analysis (KPCA) for feature extraction. After feature extraction, support vector machine (SVM) was carried out to classify EC signals. It is shown by extensive experiments that KPCA is better than the principal component analysis for feature extraction. The kernel methods using SVM by KPCA feature extraction can perform better than the other classification methods.

Customer churn prediction on kernel principal component analysis feature abstraction

ion method was presented. The prediction model was designed by combining Kernel Principal Component Analysis (KPCA) and Logistic regression. Experimental results of customer churn prediction for a telecommunication carrier show that the proposed method is superior to original attributes and PCA feature abstraction in hit rate, covering rate, accuracy rate and lift coefficient, which indicates that KPCA abstracts nonlinear features of customer data and provides an effective measurement for customer churn prediction.

Customer churn prediction on kernel principal component analysis feature abstraction

ion method was presented. The prediction model was designed by combining Kernel Principal Component Analysis (KPCA) and Logistic regression. Experimental results of customer churn prediction for a telecommunication carrier show that the proposed method is superior to original attributes and PCA feature abstraction in hit rate, covering rate, accuracy rate and lift coefficient, which indicates that KPCA abstracts nonlinear features of customer data and provides an effective measurement for customer churn prediction.

Subspace-based gearbox condition monitoring by kernel principal component analysis

Feature extraction is a key step for gearbox condition monitoring. The statistical features of the measured vibrations can be used to characterise gearbox conditions; however, their regularity and sensitivity in pattern space are different and may vary considerably under different operating conditions. This paper addresses the non-linear feature extraction scheme from the time-domain features with wavelet packet preprocessing and frequency-domain features of the vibration signals using the kernel principal component analysis (KPCA). Then two different KPCA-based subspace structures are constructed for representing and classifying the gearbox conditions. The proposed methods can extract the non-linear features of gearbox conditions using KPCA effectively, and perform conveniently with low computational complexity based on subspace methods. Experimental analysis with a fatigue test of an automobile transmission gearbox shows that the KPCA features outperform PCA features in terms of clustering capability, and both the two KPCA-based subspace methods can be effectively applied to gearbox condition monitoring.

A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine

Recently, support vector machine (SVM) has become a popular tool in time series forecasting. In developing a successful SVM forecastor, the first step is feature extraction. This paper proposes the applications of principal component analysis (PCA), kernel principal component analysis (KPCA) and independent component analysis (ICA) to SVM for feature extraction. PCA linearly transforms the original inputs into new uncorrelated features. KPCA is a nonlinear PCA developed by using the kernel method. In ICA, the original inputs are linearly transformed into features which are mutually statistically independent. By examining the sunspot data, Santa Fe data set A and five real futures contracts, the experiment shows that SVM by feature extraction using PCA, KPCA or ICA can perform better than that without feature extraction. Furthermore, among the three methods, there is the best performance in KPCA feature extraction, followed by ICA feature extraction.