Kernel Principal Component Analysis Research Articles

Entropy Feature Fusion-Based Diagnosis for Railway Point Machines Using Vibration Signals Based on Kernel Principal Component Analysis and Support Vector Machine

Entropy Feature Fusion-Based Diagnosis for Railway Point Machines Using Vibration Signals Based on Kernel Principal Component Analysis and Support Vector Machine

Feature Extraction Algorithm of Massive Rainstorm Debris Flow Based on Ecological Environment Telemetry

In order to accurately extract the characteristics of debris flow caused by group rainstorms, effectively identify the on-site information of debris flow, and provide a scientific basis for debris flow monitoring, early warning and disaster control, this paper proposes a method for extracting the characteristics of heavy rainstorm debris flow using multiregional ecological environment remote sensing. In the ecological environment where debris flows occur frequently, remote sensing data of heavy rainstorm debris flows are preprocessed using remote sensing technology, providing an important basis for the feature extraction of debris flows. The kernel principal component analysis method and Gabor filters are innovatively used to extract the spectral and texture features of rainstorm and debris flow remote sensing images, and the convolutional neural network structure is improved based on the open source deep learning framework, integrating multilevel features to generate debris flow feature maps. The improved convolution neural network is then used to extract the secondary features of the fusion feature map, and the feature extraction of heavy rainstorm debris flow is realized. The experiment shows that this method can accurately extract the characteristics of heavy rainstorm debris flow. Fused remote sensing images of debris flow effectively ameliorate the problem of insufficient informational content in a single image and improve image clarity. When the Gabor kernel function has eight different directions, the feature extraction effect of the debris flow image in each direction of the heavy rainstorm is the best.

Ordinal classification for interval-valued data and interval-valued functional data

The aim of ordinal classification is to predict the ordered labels of the output from a set of observed inputs. Interval-valued data refers to data in the form of intervals. For the first time, interval-valued data and interval-valued functional data are considered as inputs in an ordinal classification problem. Six ordinal classifiers for interval data and interval-valued functional data are proposed. Three of them are parametric, one of them is based on ordinal binary decompositions and the other two are based on ordered logistic regression. The other three methods are based on the use of distances between interval data and kernels on interval data. One of the methods uses the weighted k-nearest-neighbor technique for ordinal classification. Another method considers kernel principal component analysis plus an ordinal classifier. And the sixth method, which is the method that performs best, uses a kernel-induced ordinal random forest. They are compared with naïve approaches in an extensive experimental study with synthetic and original real data sets, about human global development, and weather data. The results show that considering ordering and interval-valued information improves the accuracy. The source code and data sets are available at https://github.com/aleixalcacer/OCFIVD.

Metaverse-IDS: Deep learning-based intrusion detection system for Metaverse-IoT networks

Combining the metaverse and the Internet of Things (IoT) will lead to the development of diverse, virtual, and more advanced networks in the future. The integration of IoT networks with the metaverse will enable more meaningful connections between the 'real' and 'virtual' worlds, allowing for real-time data analysis, access, and processing. However, these metaverse-IoT networks will face numerous security and privacy threats. Intrusion Detection Systems (IDS) offer an effective means of early detection for such attacks. Nevertheless, the metaverse generates substantial volumes of data due to its interactive nature and the multitude of user interactions within virtual environments, posing a computational challenge for building an intrusion detection system. To address this challenge, this paper introduces an innovative intrusion detection system model based on deep learning. This model aims to detect most attacks targeting metaverse-IoT communications and combines two techniques: KPCA (Kernel Principal Component Analysis which was used for attack feature extraction and CNN (Convolutional Neural Networks for attack recognition and classification. The efficiency of this proposed IDS model is assessed using two widely recognized benchmark datasets, BoT-IoT and ToN-IoT, which contain various IoT attacks potentially targeting IoT communications. Experimental results confirmed the effectiveness of the proposed IDS model in identifying 12 classes of attacks relevant to metaverse-IoT, achieving a remarkable accuracy of 99.8% and a False Negative Rate FNR less than 0.2. Furthermore, when compared with other models in the literature, our IDS model demonstrates superior performance in attack detection accuracy.

Modeling Method for Aerobic Zone of A2O Based on KPCA-PSO-SCN

Sewage treatment plants face significant problems as a result of the annual growth in urban sewage discharge. Substandard sewage discharge can also be caused by rising sewage treatment expenses and unpredictable procedures. The most widely used sewage treatment process in urban areas is the Anaerobic–Anoxic–Oxic (A2O) sewage treatment process. Therefore, modeling the sewage treatment process and predicting the effluent quality are of great significance. A process modeling method based on Kernel Principal Component Analysis–Particle Swarm Optimization–Stochastic Configuration Network (KPCA-PSO-SCN) is proposed for the A2O aerobic wastewater treatment process. Firstly, eight auxiliary variables were determined through mechanism analysis, including Chemical Oxygen Demand (COD) and ammonia nitrogen (NH4+) and nitrate nitrogen (NO3−) of influent water, pH, temperature (T), Mixed Liquor Suspended Solid (MLSS), Dissolved Oxygen (DO) and hydraulic residence time (HRT) in the aerobic zone. Dimensionality reduction was carried out using the kernel principal component analysis method based on the Gaussian function, and the eight-dimensional data were changed to five-dimensional data, which improved the running speed and efficiency of subsequent models. Then, according to the advantages of the particle swarm optimization algorithm, such as low calculation cost and fast convergence, combined with the advantages of stochastic configuration network general approximation performance, the PSO-SCN model was established to predict the three water quality indexes of effluent COD, NH4+, and NO3− for the aerobic zone. The experimental results proved the effectiveness of the model. Compared with classic water quality prediction algorithm models such as SCN, PSO-BP, RBF, PSO-RBF, etc., the superiority of the PSO-SCN algorithm model was demonstrated.

Principal components-based hidden Markov model for automatic detection of whale vocalisations

Over the years, researchers have continued to put forward solutions to lessen the threats faced by whales within their ecosystem. The correct detection of the different species of whale is important in the search for solutions that will lessen the threats. In order to accurately detect and classify whale species, a number of techniques have been proposed over the years, with varying degrees of success. This research seeks to improve the performance of the hidden Markov models (HMMs), which is one of the most consistent methods for the detection and classification of whale vocalisations. The performance of HMMs is affected by the quality of the feature vectors fed into them. Thus, this research proposes feature extraction (FE) techniques based on principal component analysis. The principal components (PC) computed from PCA and kernel PCA were uniquely transformed into feature vector structures suitable for the HMMs. The emerging models, PCA-HMMs and kPCA-HMMs, were experimented with on passive acoustic monitoring (PAM) datasets containing southern right whale and humpback whale vocalisations. The results from the experiments showed that the kPCA-HMMs outperformed PCA-HMMs. This is due to kPCA’s ability to find non-linear subspaces that may exist in whale vocalisations. Furthermore, the results of the developed PC-HMMs were compared with other existing FE techniques used with HMMs in the literature for the detection of whale vocalisations. The proposed PC-HMMs did not only outperform the existing FE-HMMs but also offered lower computational complexity than the existing HMMs for the detection of whale vocalisations.

An enhanced fault diagnosis method for fuel cell system using a kernel extreme learning machine optimized with improved sparrow search algorithm

Proton exchange membrane fuel cells (PEMFC) have a broad development prospect in the fields of vehicles, drones and ships due to their high efficiency and cleanliness. However, the problems of insufficient reliability and durability have severely restricted their industrialization process. To improve the safety, reliability and durability of fuel cell system, a fault diagnosis method that combined kernel principal component analysis (KPCA) with an improved sparrow search algorithm (ISSA) and an optimized kernel extreme learning machine (KELM) was proposed in this study. Firstly, KPCA is utilized to extract nonlinear features from fault indicators and obtain the fault feature vector of the fuel cell system. Then, by incorporating logistic mapping and Cauchy Gaussian mutation strategies to improve the Sparrow Search Algorithm (SSA), ISSA was used to optimize the kernel parameters and regularization coefficient in KELM. The experimental results show that the KPCA-ISSA-KELM method for normal conditions, hydrogen leakage and membrane drying are 100%, 98.5% and 100%, respectively, with an overall accuracy of 99.5% and an operation time of 0.97s. The diagnostic accuracy of the proposed method is 10.4%, 5.7%, 4.8%, 4.2%, 3.0%, 1.8% higher than support vector machine (SVM), back propagation neural network (BPNN), KELM, genetic algorithm-based KELM (GA-KELM), particle swarm optimization-based KELM (PSO-KELM) and SSA-KELM, respectively, and the operation time is only slightly higher than that of the SVM model and KELM model.

Multimodal Emotion Recognition Using Bi-LG-GCN for MELD Dataset

Emotion recognition using multimodal data is a widely adopted approach due to its potential to enhance human interactions and various applications. By leveraging multimodal data for emotion recognition, the quality of human interactions can be significantly improved. We present the Multimodal Emotion Lines Dataset (MELD) and a novel method for multimodal emotion recognition using a bi-lateral gradient graph neural network (Bi-LG-GNN) and feature extraction and pre-processing. The multimodal dataset uses fine-grained emotion labeling for textual, audio, and visual modalities. This work aims to identify affective computing states successfully concealed in the textual and audio data for emotion recognition and sentiment analysis. We use pre-processing techniques to improve the quality and consistency of the data to increase the dataset’s usefulness. The process also includes noise removal, normalization, and linguistic processing to deal with linguistic variances and background noise in the discourse. The Kernel Principal Component Analysis (K-PCA) is employed for feature extraction, aiming to derive valuable attributes from each modality and encode labels for array values. We propose a Bi-LG-GCN-based architecture explicitly tailored for multimodal emotion recognition, effectively fusing data from various modalities. The Bi-LG-GCN system takes each modality's feature-extracted and pre-processed representation as input to the generator network, generating realistic synthetic data samples that capture multimodal relationships. These generated synthetic data samples, reflecting multimodal relationships, serve as inputs to the discriminator network, which has been trained to distinguish genuine from synthetic data. With this approach, the model can learn discriminative features for emotion recognition and make accurate predictions regarding subsequent emotional states. Our method was evaluated on the MELD dataset, yielding notable results in terms of accuracy (80%), F1-score (81%), precision (81%), and recall (81%) when using the MELD dataset. The pre-processing and feature extraction steps enhance input representation quality and discrimination. Our Bi-LG-GCN-based approach, featuring multimodal data synthesis, outperforms contemporary techniques, thus demonstrating its practical utility.

Evaluating Autoencoders as a Dimensionality Reduction Mechanism to Support Clustering Brazilian Agricultural Diversity

Brazilian agricultural production presents high spatial diversity, challenging the conception of public policies. This article proposes an approach for grouping Brazilian municipalities according to their agricultural production. We combine a feature extraction using autoencoders and clustering based on k-means and Self-Organizing Maps. We used panel data from IBGE’s annual estimates of the production value of permanent and temporary crops, animal products, aquaculture, plant extractivism, forestry, planted areas, and herd population between 1999 and 2018. We analyzed different structures of simple stacked and incomplete autoencoders, varying the number of layers and neurons in each, and evaluated the asymmetric exponential linear loss function to handle the sparse data. We applied the Isomap, Kernel PCA, Truncated SVD, and MDS dimensionality reduction methods for comparative purposes. Results showed that the autoencoders could extract characteristics from the transformed raw data to allow the clustering of municipalities to reveal regional and evenintra-regional patterns. The autoencoders improved comparative performance as the intrinsic dimensionality increased.

A novel fault detection framework integrated with variable importance analysis for quality-related nonlinear process monitoring

Monitoring the quality-related variables is an essential task in industrial processes. However, large amounts of variables and complex relationship between measured and quality variables pose challenges in improving fault detection performance. This paper proposes a novel quality-related fault detection framework based on variable importance analysis for nonlinear process monitoring. In the proposed framework, a new nonlinear variable division index called improved variable importance in the projection (IVIP) is designed to better identify quality-related variables. First, measured variables are divided into quality-related and independent spaces according to their IVIP values. Then, considering different data characteristics, a hybrid detection model combined multivariate exponentially weighted moving average and kernel principal component analysis (MEKPCA) is constructed in these two spaces to monitor abnormal events. To provide an integrated decision, the Bayesian inference strategy is finally utilized to fuse useful information from different spaces. Three case studies, including a simulated numerical system, the Tennessee Eastman process, and a wastewater treatment process, demonstrate that our proposed IVIP-MEKPCA outperforms other comparison methods.

MK-DCCA-Based Fault Diagnosis for Incipient Faults in Nonlinear Dynamic Processes

Incipient fault diagnosis is particularly important in process industrial systems, as its early detection helps to prevent major accidents. Against this background, this study proposes a combined method of mixed kernel principal components analysis and dynamic canonical correlation analysis (MK-DCCA). The robust generalization performance of this approach is demonstrated through experimental validation on a randomly generated dataset. Furthermore, comparative experiments were conducted on a CSTR Simulink model, comparing the MK-DCCA method with DCCA and DCVA methods, demonstrating its excellent detection performance for incipient faults in nonlinear and dynamic systems. Meanwhile, fault identification experiments were conducted, validating the high accuracy of the fault identification method based on contribution. The experimental findings demonstrate that the method possesses a certain industrial significance and academic relevance.

Hand gesture classification framework leveraging the entropy features from sEMG signals and VMD augmented multi-class SVM

To improve the classification accuracy of hand movements from sEMG signals, this paper puts forward a unified hand gesture classification framework which exploits the potentials of variational mode decomposition (VMD) and multi-class support vector machine (SVM). Acquiring the sEMG signals from 25 intact subjects for ten functional activities in real-time, we implement a non-recursive adaptive decomposition technique to sEMG signals and perform power spectral analysis to identify the dominant narrow-band intrinsic mode functions (IMFs) that contain prominent biomarkers. Subsequently, to compute the optimal feature vectors from a set of entropy measures, this work investigates the performance of two techniques namely minimum redundancy and maximum relevance (MRMR) technique and kernel principal component analysis (kPCA). After extracting the optimal set of entropy features, the proposed approach implements a multi-class SVM based on one-vs-one (OVO) strategy to classify the hand gestures. The performance of the multi-class SVM compared with those of the K-nearest neighbor (KNN) and naïve bayes (NB) classifiers highlight that multi-class SVM offers superior performance with an average classification accuracy of 99.98%. Moreover, for statistical analysis of the experimental results, this work performs Friedman test to analyze the significance of the SVM, KNN and NB classifier performances. Finally, the performance comparison of the proposed approach with those of the state-of-the-art techniques highlights the superiority of the proposed framework to improve the hand gesture classification accuracy.

Fault detection and quantitative assessment of wheel diameter difference based on ensemble adaptive tunable Q-factor wavelet transform and mixed kernel principal component analysis

Tread wear is inevitable for railway vehicles. Because of the complicated railway condition, the wear rates of the two wheels of a wheelset are usually unequal, which leads to the wheel diameter difference (WDD). The WDD shortens the service life of the wheelset and deteriorates the dynamic performance of railway vehicles. Therefore, it has long been desired to monitor and quantify the WDD condition. However, influenced by the random irregularity, the effective vibration features induced by the WDD may be completely submerged in noise. In this paper, an adaptive TQWT decomposition method is proposed for denoise and feature enhancement. In this method, a new evaluation index is constructed and used to determine parameters. The influence of the WDD on the vibration of the wheelset can be more obvious after denoised by the adaptive TQWT decomposition. Mixed kernel principal component analysis is introduced into the condition monitoring and quantitative assessment of the WDD. By calculating the Hoteling’s and SPE statistics, control charts are employed to identify and assess the WDD conditions quantitatively in conjunction with the corresponding control limits. The effectiveness of the proposed method is validated by the simulation and experimental study. For the experimental datasets, the proposed method gives the average fault detection rate (FDR), false alarm rate (FAR), false alarm rate (PA) of 92.74%, 2.23%, 97.47% for the statistic and the average FDR, FAR, PA of 96.57%, 2.60%, 97.29% for the SPE statistic. The results show that the proposed method can effectively monitor the WDD conditions.

A new monitoring approach of time-varying and nonlinear processes with application to penicillin fermentation process

In the actual production process, time-varying and nonlinear problems are numerous important problems to be considered, in view of these problems, a process monitoring approach based on locally weighted probabilistic kernel principal component analysis (LWPKPCA) is proposed. First, the method selects the normal process data with a high similarity to the test samples as training data of the local model, and continuously updates the local model according to the test samples to build an accurate time-varying model. Second, by weighting the data of different importance, the role of data similar to test samples in the modeling process is strengthened. Third, the LWPKPCA model is applied to process monitoring, the monitoring indicators are established in a high-dimensional space and used to detect faults. Finally, on the basis of LWPKPCA, the penicillin fermentation process (PFP) is taken to evaluate the monitoring performance of the proposed methods. According to the comparison of the experiment results, the detection rate and accuracy rate of the LWPKPCA method is considerably better than those of probabilistic principal component analysis and probabilistic kernel principal component analysis methods. The results demonstrate that the proposed method is suitable for processing time-varying data with nonlinear characteristics, and the LWPKPCA process monitoring method is effective for improving the performance of fault detection.

Bayesian optimization of multiscale kernel principal component analysis and its application to model Gas-to-liquid (GTL) process data

Kernel methods map the data from original space into a higher-dimensional space in which linear methods are applied. In many applications, the inverse mapping is also important, and the pre-image of a feature vector must be found in the original space. Kernel principal component analysis (KPCA) based kernel density estimation (KDE) has been developed to solve this problem. However, the performance of the KPCA technique greatly depends on the choice of some parameters which can lead to poor modeling performance when these parameters are not well identified. Thus, fully Bayesian optimization KPCA (BOKPCA) is proposed to enhance the performance of the KPCA model. BOKPCA method aims to automatically select the best parameters of the KPCA model. Generally, kernel methods struggle to handle nonlinear data contaminated with high levels of noise. This is because the noise affects every principal component, making it challenging to mitigate its influence during the reconstruction step. Consequently, to further enhance the ability of KPCA and BOKPCA models, we propose to integrate multi-scale filtering with these two models. The efficiency of the proposed methods are evaluated using a simulated nonlinear process and real data generated from a bench-scale Fischer–Tropsch (FT) process.

Kernel-Based KPI-Oriented Fault Detection Approach in the Presence of Outliers

Kernel partial least squares (KPLS) has poor robustness and cannot achieve effective monitoring for key performance indicators (KPI). This study investigates a new KPI-oriented robust KPLS approach to mitigate these drawbacks. In this methodology, the robust KPLS inspired by the gradient boosting principle incorporates a weighting matrix into the KPLS to mitigate the influence of outliers. Simultaneously, the associated coefficient matrix is derived in detail. Then, a decomposition approach is used to separate the process variable space into two orthogonal parts. Two strategies are discussed to obtain the unknown projection matrices based on the kernel principal component analysis and the elastic network frameworks. Finally, the performance of the proposed methods in terms of prediction, monitoring and robustness to outliers is evaluated by the Tennessee Eastman process and the three-phase flow facility. The results show that the proposed methods have good prediction accuracy and monitoring performance in the presence of outliers, demonstrating the effectiveness and advantages of the proposed approaches.

A Relevant Variable Selection and SVDD-Based Fault Detection Method for Process Monitoring

This study investigates the sample value imbalance problem of process monitoring. A fault detection approach based on variable selection and support vector data description (SVDD) is developed for efficient process monitoring. First, Kullback&#x2013;Leibler divergence serves as the variable selection algorithm, which highlights the most beneficial information about the concerned faults. The attained variables are segmented by block division to avoid faults information being covered in single space monitoring, so that the relevant variables and the most beneficial information are concentrated in the same block. Then, Kernel principal component analysis is applied in each block to address the challenge that variables may still be high-dimensional and nonlinear. After that, the monitoring result is given based on the proposed SVDD with a restructured radius index, which is more sensitive to the fault. As demonstrated from experimental results on the Tennessee Eastman process, this method is effective and outperforms counterparts with higher mean fault detection rate. <i>Note to Practitioners</i>&#x2014;Recently, multivariate statistical process monitoring (MSPM) has attracted much attention. In general, MSPM incorporates all variables for the large-scale process. However, only a small number of variables are fault-dependent. Namely, the sample value imbalance problem is encountered in application. In this scenario, the monitoring performance degrades and the online computational complexity increases. To this end, a SVDD-based fault detection method, which considers the fault-related variables, is proposed for process monitoring. The proposed method is verified by the Tennessee Eastman process and it is more sensitive to the concerned fault.

Probabilistic Combination of Non-Linear Eigenprojections For Ensemble Classification

The emergence of new technologies has changed the way clinicians perform diagnosis. Medical imaging play a crucial role in this process, given the amount of information that they usually provide as non-invasive techniques. Despite the high quality offered by these images and the expertise of clinicians, the diagnostic process is not a straightforward task since different pathologies can have similar signs and symptoms. For this reason, it is extremely useful to assist this process with the inclusion of an automatic tool that reduces the bias when analyzing this kind of images. In this work, we propose an ensemble classifier based on probabilistic Support Vector Machine (SVM) in order to identify relevant patterns while providing information about the reliability of the classification. Specifically, each image is divided into patches and features contained in each one of them are extracted by applying kernel principal component analysis (PCA). The use of base classifiers within an ensemble allows our system to identify the informative patterns regardless of their size or location. Decisions of each individual patch are then combined according to the reliability of each individual classification: the lower the uncertainty, the higher the contribution. Performance is evaluated in a real scenario where distinguishing between pneumonia patients and controls from chest Computed Tomography (CCT) images, yielding an accuracy of 97.86%. The large performance obtained and the simplicity of the system (use of deep learning in CCT images would highly increase the computational cost) evidence the applicability of our proposal in a real-world environment.

Quality design based on kernel trick and Bayesian semiparametric model for multi-response processes with complex correlations

ABSTRACT Processes or products are typically complex systems with numerous interrelated procedures and interdependent components. This results in complex relationships between responses and input factors, as well as complex nonlinear correlations among multiple responses. If the two types of complex correlations in the quality design cannot be properly dealt with, it will affect the prediction accuracy of the response surface model, as well as the accuracy and reliability of the recommended optimal solutions. In this paper, we combine kernel trick-based kernel principal component analysis, spline-based Bayesian semiparametric additive model, and normal boundary intersection-based evolutionary algorithm to address these two types of complex correlations. The effectiveness of the proposed method in modeling and optimisation is validated through a simulation study and a case study. The results show that the proposed Bayesian semiparametric additive model can better describe the process relationships compared to least squares regression, random forest regression, and support vector basis regression, and the proposed multi-objective optimisation method performs well on several indicators mentioned in the paper.

A retrieval model of surface geochemistry composition based on remotely sensed data

Abstract The geochemical sampling work in the difficult and dangerous areas is very hard; hence, it can be greatly improved by combining with the remotely sensed data. Thus, a retrieval model is proposed by Kernel Principal Component Analysis and Artificial Bee Colony (ABC) optimized Support Vector Machine (SVM) models based on Landsat 8 remotely sensed data and the geochemical data in the study area. The analysis results show that the geochemical data delineate the areas with relatively enriched elements, but indicate the low-abnormal ore (chemical) points, and the anomalies delineated by the inversion data are better for this purpose, for better indication. At the same time, the distribution and intensity of the corresponding abnormal areas found that the abnormal areas delineated by the inversion data basically contain the abnormal areas delineated by the original data, and the anomalies located at the ore spots are obviously enhanced; it shows that the SVM model of ABC Optimization can establish the relation between geochemistry data and remote sensing data, can supply the original data effectively, and can also provide the direction for the next prospecting work.