Kernel Partial Least Research Articles

IKPLS: Improved Kernel Partial Least Squares and Fast Cross-Validation Algorithms for Python with CPU and GPU Implementations Using NumPy and JAX

IKPLS: Improved Kernel Partial Least Squares and Fast Cross-Validation Algorithms for Python with CPU and GPU Implementations Using NumPy and JAX

Novel intrusion detection system based on a downsized kernel method for cybersecurity in smart agriculture

Smart farming aims to ameliorate the quantity and quality of farms products using modern information and communication technologies. Agriculture has always used cutting-edge technology to improve profitability, efficiency, and safety as well as creating the opportunity for more environmentally friendly operations. In smart agriculture network communication between smart devices is necessary to managing the agriculture process. Consequently, the possibility of information being exposed to hacking is very likely, which makes the information security system extremely important and cannot be neglected. Even though many solutions have been developed to overcome cyberattacks in smart networks, some limitations are observed, such as solving multi-class problems, high feature dimensions, and overhead computation. To address the aforementioned issues, an intelligent Intrusion Detection System (IDS) for identifying cyberattacks in the Internet of Agriculture Things (IoAT) is proposed. The developed model uses a proposed reduced kernel method, the Downsized Kernel Partial Least Square (DKPLS) to extract and reduce data feature dimension to improve detection performance. The DRKPLS approach is used to reduce the dimension of a kernel matrix generated using the Kernel Partial Least Square (KPLS) technique by selecting the most important features. In classification phase, the Kernel Extreme Learning Machine (KELM) is used to classify data for binary and multi-class classification. The proposed IDS is evaluated on a new Industrial IoT Dataset, X-IIoTID. The developed approach achieved higher performances for binary classification and for multi–class classification compared to others machine learning and deep learning approaches. The proposed approach achieved an accuracy rate of 99.92% for binary classification and 99.99% for multi–class classification. Furthermore, the higher percentage in multi-class classification is obtained for the two types of attacks Ransom Denial of Service (RDoS) and Command_Control respectively.

An Improved Deep Kernel Partial Least Squares and Its Application to Industrial Data Modeling

An Improved Deep Kernel Partial Least Squares and Its Application to Industrial Data Modeling

Exploring the Path of Teacher Team Building for Innovation and Entrepreneurship in Colleges and Universities in the Information Age

Abstract With the in-depth promotion of education informatization, colleges and universities focus on developing innovation and entrepreneurship education and are committed to cultivating talents with innovative spirit and technology, which has become the key to the reform of higher education. This article profoundly analyzes the characteristics of the current innovation and entrepreneurship education teacher team, and proceeds to optimize and build a dual-creative teacher team with information literacy. Aiming at the construction effect of this path, the article constructs an evaluation index system of the construction effect of the dual-creation teacher team from multiple aspects. It combines the Kernel Partial Least Squares (KPLS) and Particle Swarm Optimization Algorithm (PSO) to empower the indexes to assess the effectiveness of the construction path. The research analysis shows that the proposed faculty building path effectively promotes the rejuvenation of the faculty team, and the percentage of teachers under 35 years old rises to 34%, significantly improving teachers’ acceptance of new things. Regarding teaching results, the weight of students’ participation in entrepreneurial practice reaches 0.146. The weight of filing innovation patents is 0.101. The results show that the construction path has a positive impact on improving the faculty team’s teaching quality and students’ innovation ability.

Chemometric-driven quantification of construction and demolition waste using Raman spectroscopy and SWIR: Enhancing sustainability in the ceramic sector

Construction and demolition wastes (CDW) face low recycling rates due to the absence of reliable and industrially feasible quantification method for valorization. This study explores quantifying mixed construction materials through their homogenization, a novel proposal, and chemometric modeling. The problem materials were homogenized in known proportions for the acquisition of a statistical number of chemical data. Short-wave infrared and Raman spectroscopy (applied here for the first time) provided data for two independent models. Principal Component Analysis and Improved Kernel Partial Least Square Discriminant Analysis were employed for model construction. These models were successfully applied to a real CDW sample from a Spanish management plant, yielding comparable results. This cost-effective methodology allows the quantification of stony aggregates within CDW, enhancing their recycling potential in the ceramic sector. Addressing the scarcity of reliable CDW quantification procedures, this approach and the use of Raman technique hold promise for advancing CDW valorization and sustainability.

Colorimetric Characterization of Color Imaging System Based on Kernel Partial Least Squares

Colorimetric characterization is the basis of color information management in color imaging systems. In this paper, we propose a colorimetric characterization method based on kernel partial least squares (KPLS) for color imaging systems. This method takes the kernel function expansion of the three-channel response values (RGB) in the device-dependent space of the imaging system as input feature vectors, and CIE-1931 XYZ as output vectors. We first establish a KPLS color-characterization model for color imaging systems. Then we determine the hyperparameters based on nested cross validation and grid search; a color space transformation model is realized. The proposed model is validated with experiments. The CIELAB, CIELUV and CIEDE2000 color differences are used as evaluation metrics. The results of the nested cross validation test for the ColorChecker SG chart show that the proposed model is superior to the weighted nonlinear regression model and the neural network model. The method proposed in this paper has good prediction accuracy.

Quality-related Fault Detection Based on Approximate Kernel Partial Least Squares Method

Quality-related Fault Detection Based on Approximate Kernel Partial Least Squares Method

Remote exploration and monitoring of geothermal sources: A novel method for foliar element mapping using hyperspectral (VNIR-SWIR) remote sensing

Hyperspectral remote sensing is an emerging technique to develop new cost- and time-effective geophysical mapping methods. To overcome challenges introduced by plant cover in geothermal systems globally, we hypothesise that foliage can be used as a proxy to map underlying surface geothermal activity and heat-flux due to their capability on elemental uptake from geothermal fluids and host rock/soil. This study shows for the first time that foliar elemental mapping can be used to image geothermal systems using both high-resolution airborne and satellite hyperspectral images.This study has specifically targeted kanuka shrub (kunzea ericoides var. microflora) as a proxy media to develop air- and spaceborne hyperspectral solutions to monitor inaccessible, biologically and culturally sensitive geothermal areas. Using high resolution airborne AisaFENIX and PRISMA hyperspectral data, foliar element maps for Ag, As, Ba and Sb have been developed using Kernel Partial Least Squares Regression and Random Forest classification models to track their foliar distribution and develop a conceptual model for metal and thermal induced changes in plants. Our study shows evidence that the created foliar element maps are in concordance with independent LiDAR-retrieved canopy structure and height as well as temperature effects of the underlying geothermal field. This study has proven air- and spaceborne hyperspectral sensors can indeed capture critical information within the VNIR and SWIR regions (e.g. ∼452, ∼500, ∼670, ∼820, ∼970, ∼1180, ∼1400 and ∼2000 nm) that can be used to identify metal and thermal-induced spectral changes in plants reliably (overall accuracy of 0.41–0.66) with remotely sensed imagery. Our non-invasive method using hyperspectral remote sensing can complement existing practices for exploration and management of renewable geothermal resources through timely monitoring from air- and spaceborne platforms.

A Deep Supervised Learning Framework Based on Kernel Partial Least Squares for Industrial Soft Sensing

Kernel partial least squares (KPLS) is a widely used soft sensor modeling method for nonlinear industrial processes. However, the traditional KPLS is considered as the shallow learning machine and may not capture the vital information hidden among data. In order to exploit the intrinsic data feature information, in this article, we propose a deep supervised learning framework based on KPLS, which is referred to as deep KPLS (DeKPLS). First, inspired by the deep learning mechanism, a hierarchical feature extraction framework based on KPLS is proposed, where the KPLS is served as the basic feature extraction module. Then, a layer-wise feedforward training strategy is designed for the determination of model architecture. Finally, two actual industrial processes are utilized to demonstrate the effectiveness of the proposed DeKPLS.

Kernel Partial Least Squares Feature Selection Based on Maximum Weight Minimum Redundancy

Feature selection refers to a vital function in machine learning and data mining. The maximum weight minimum redundancy feature selection method not only considers the importance of features but also reduces the redundancy among features. However, the characteristics of various datasets are not identical, and thus the feature selection method should have different feature evaluation criteria for all datasets. Additionally, high-dimensional data analysis poses a challenge to enhancing the classification performance of the different feature selection methods. This study presents a kernel partial least squares feature selection method on the basis of the enhanced maximum weight minimum redundancy algorithm to simplify the calculation and improve the classification accuracy of high-dimensional datasets. By introducing a weight factor, the correlation between the maximum weight and the minimum redundancy in the evaluation criterion can be adjusted to develop an improved maximum weight minimum redundancy method. In this study, the proposed KPLS feature selection method considers the redundancy between the features and the feature weighting between any feature and a class label in different datasets. Moreover, the feature selection method proposed in this study has been tested regarding its classification accuracy on data containing noise and several datasets. The experimental findings achieved using different datasets explore the feasibility and effectiveness of the proposed method which can select an optimal feature subset and obtain great classification performance based on three different metrics when compared with other feature selection methods.

Towards the Development of a Diagnostic Test for Autism Spectrum Disorder: Big Data Meets Metabolomics.

Autism spectrum disorder (ASD) is defined as a neurodevelopmental disorder which results in impairments in social communications and interactions as well as repetitive behaviors. Despite current estimates showing that approximately 2.2% of children are affected in the United States, relatively little about ASD pathophysiology is known in part due to the highly heterogenous presentation of the disorder. Given the limited knowledge into the biological mechanisms governing its etiology, the diagnosis of ASD is performed exclusively based on an individual's behavior assessed by a clinician through psychometric tools. Although there is no readily available biochemical test for ASD diagnosis, multivariate statistical methods show considerable potential for effectively leveraging multiple biochemical measurements for classification and characterization purposes. In this work, markers associated with the folate dependent one-carbon metabolism and transulfuration (FOCM/TS) pathways analyzed via both Fisher Discriminant Analysis and Support Vector Machine showed strong capability to distinguish between ASD and TD cohorts. Furthermore, using Kernel Partial Least Squares regression it was possible to assess some degree of behavioral severity from metabolomic data. While the results presented need to be replicated in independent future studies, they represent a promising avenue for uncovering clinically relevant ASD biomarkers.

Dynamic prediction of multivariate functional data based on Functional Kernel Partial Least Squares

With the flourishment of sensing technology, a huge mass of functional data can be acquired to describe the manufacturing process and predict the product quality. But these data simultaneously bring in modeling challenges of multi-type data, high-dimensional variables, data irregularity, and complex correlations. To address these challenges, this work proposes the Functional Kernel Partial Least Squares (FKPLS) methodology, which is a function-on-function regression. The FKPLS method first smooths the discrete sequences to continuous functions via functional data analysis (FDA) to retain the dynamic characteristics of variables, and then relies on basis function expansion to estimate the regression coefficient functions in order to avoid the ill-posed problem in directly estimating the functional eigenequation. An artificial dataset and a real-world steelmaking process are used to validate the effectiveness of the FKPLS method. The results demonstrate lower mean relative prediction error of the FKPLS method than the traditional methods, which is no more than 0.36% on simulation data and no more than 14.88% on industrial data. As the regression coefficients are functions, the FKPLS method is also effective in predicting dynamic mapping between predictors and responses.

Face Recognition Through Multi-Resolution Images

In this paper, we introduce a new method for face recognition in multi-resolution images. The proposed method is composed of two phases: an off-line phase and an inference phase. In the off-line phase, we built the Kernel Partial Least Squares (KPLS) regression model to map the LR facial features to HR ones. The KPLS predictor was then used in the inference phase to map HR features from LR features. We applied in both phases the Block-Based Discrete Cosine Transform (BBDCT) descriptor to enhance the facial feature description. Finally, the identity matching was carried out with the K-Nearest Neighbor (KNN) classifier. Experimental study was conducted on the AR and ORL databases and the obtained results proved the efficiency of the proposed method to deal with LR and VLR face recognition problem.

An enhanced CAD system based on machine Learning Algorithm for brain MRI classification

The development of Computer-Aided Design (CAD) and Computer-Aided Manufacturing (CAM) systems in the past decade has led to a remarkable advance in biomedical applications and devices. Particularly, CAM and CAD systems are employed in medical engineering, robotic surgery, clinical medicine, dentistry and other biomedical areas. Hence, the accuracy and precision of the CAD and CAM systems are extremely important for proper treatment. This work suggests a new CAD system for brain image classification by analyzing Magnetic Resonance Images (MRIs) of the brain. Firstly, we use the proposed Downsized Rank Kernel Partial Least Squares (DR-KPLS) as a feature extraction technique. Then, we perform the classification using Support Vector Machines (SVM) and we validate with a k-fold cross validation approach. Further, we utilize the Tabu search metaheuristic approach in order to determine the optimal parameter of the kernel function. The proposed algorithm is entitled DR-KPLS+SVM. The algorithm is tested on the OASIS MRI database. The proposed kernel-based classifier is found to be better performant than the existing methods.

Improved Dynamic Optimized Kernel Partial Least Squares for Nonlinear Process Fault Detection

We suggest in this article a dynamic reduced algorithm in order to enhance the monitoring abilities of nonlinear processes. Dynamic fault detection using data-driven methods is among the key technologies, which shows its ability to improve the performance of dynamic systems. Among the data-driven techniques, we find the kernel partial least squares (KPLS) which is presented as an interesting method for fault detection and monitoring in industrial systems. The dynamic reduced KPLS method is proposed for the fault detection procedure in order to use the advantages of the reduced KPLS models in online mode. Furthermore, the suggested method is developed to monitor the time-varying dynamic system and also update the model of reduced reference. The reduced model is used to minimize the computational cost and time and also to choose a reduced set of kernel functions. Indeed, the dynamic reduced KPLS allows adaptation of the reduced model, observation by observation, without the risk of losing or deleting important information. For each observation, the update of the model is available if and only if a further normal observation that contains new pertinent information is present. The general principle is to take only the normal and the important new observation in the feature space. Then the reduced set is built for the fault detection in the online phase based on a quadratic prediction error chart. Thereafter, the Tennessee Eastman process and air quality are used to precise the performances of the suggested methods. The simulation results of the dynamic reduced KPLS method are compared with the standard one.

Kernel Partial Least Square Regression with High Resistance to Multiple Outliers and Bad Leverage Points on Near-Infrared Spectral Data Analysis

Multivariate statistical analysis such as partial least square regression (PLSR) is the common data processing technique used to handle high-dimensional data space on near-infrared (NIR) spectral datasets. The PLSR is useful to tackle the multicollinearity and heteroscedasticity problem that can be commonly found in such data space. With the problem of the nonlinear structure in the original input space, the use of the classical PLSR model might not be appropriate. In addition, the contamination of multiple outliers and high leverage points (HLPs) in the dataset could further damage the model. Generally, HLPs contain both good leverage points (GLPs) and bad leverage points (BLPs); therefore, in this case, removing the BLPs seems relevant since it has a significant impact on the parameter estimates and can slow down the convergence process. On the other hand, the GLPs provide a good efficiency in the model calibration process; thus, they should not be eliminated. In this study, robust alternatives to the existing kernel partial least square (KPLS) regression, which are called the kernel partial robust GM6-estimator (KPRGM6) regression and the kernel partial robust modified GM6-estimator (KPRMGM6) regression are introduced. The nonlinear solution on PLSR was handled through kernel-based learning by nonlinearly projecting the original input data matrix into a high-dimensional feature mapping that corresponded to the reproducing kernel Hilbert spaces (RKHS). To increase the robustness, the improvements on GM6 estimators are presented with the nonlinear PLSR. Based on the investigation using several artificial dataset scenarios from Monte Carlo simulations and two sets from the near-infrared (NIR) spectral dataset, the proposed robust KPRMGM6 is found to be superior to the robust KPRGM6 and non-robust KPLS.

Key-Performance-Indicator-Related Process Monitoring Based on Improved Kernel Partial Least Squares

Although the partial least squares approach is an effective fault detection method, some issues of nonlinear process monitoring related to key performance indicators (KPIs) still exist. To address the nonlinear characteristics in the industrial processes, kernel partial least squares (KPLS) method was proposed in the literature. However, the KPLS method also faces some difficulties in fault detection. None of the existing KPLS methods can accurately decompose measurements into KPI-related and KPI-unrelated parts, and these methods usually ignore the fact that the residual subspace still contains some KPI-related information. In this article, a new improved KPLS method, which considers the KPI-related information in the residual subspace, has been proposed for KPI-related process monitoring. First, the proposed method performs general singular value decomposition (GSVD) on the calculable loadings based on the kernel matrix. Next, the kernel matrix can be suitably divided into KPI-related and KPI-unrelated subspaces. Besides, we present the design of two statistics for process monitoring as well as a detailed algorithm performance analysis for kernel methods. Finally, a numerical case and Tennessee Eastman benchmark process demonstrate the efficacy and merits of the improved KPLS-based method.

A new monitoring scheme of an air quality network based on the kernel method

Air pollution is classified as one of the most dangerous type on the human health, the environment, and the ecosystem. However, air pollution results in climate change and affects people’s health. For a number of years, monitoring the air quality has become a very urgent and necessary topic. Moreover, safety and health have been attracting attention as one of the important topics to evaluate, firstly, the degree of air pollution and predict pollutant concentrations accurately. Then, it is crucial to establish a more scientific air quality monitoring to ensure the quality of life. In this paper, new reduced air quality monitoring is suggested to enhance the Fault Detection (FD) of an air quality monitoring network. Furthermore, a sensor FD procedure based on Reduced Kernel Partial Least Squares (RKPLS) is proposed to monitor an air quality monitoring network. The main contribution of the suggested procedure is to enhance the FD of an air quality monitoring network in terms of computation time and false alarm rate, using just the important latent components, compared to standard Kernel Partial Least Squares (KPLS).

A Nonlinear Method for Component Separation of Dam Effect Quantities Using Kernel Partial Least Squares and Pseudosamples

Existing component separation methods fail to consider the complex nonlinear relationship between dam effect quantities and environmental variables. In this study, a novel nonlinear component separation method for the effect quantities is proposed by combining kernel partial least squares (KPLS) and pseudosamples. By this method, a nonlinear monitoring model is established based on KPLS, and the complicated nonlinear relationship between the effect quantities and environmental variables can be determined accurately through the model. Furthermore, special pseudosamples are constructed to separate independent components and coupling influence components of environmental factors from the KPLS model. These methods have been applied into a super‐high arch dam, and the separated displacement components conform to the general deformation law. The presented results indicate that it is more reliable than traditional multiple linear regression models.

Motor imagery and mental fatigue: inter-relationship and EEG based estimation.

Even though it has long been felt that psychological state influences the performance of brain-computer interfaces (BCI), formal analysis to support this hypothesis has been scant. This study investigates the inter-relationship between motor imagery (MI) and mental fatigue using EEG: a. whether prolonged sequences of MI produce mental fatigue and b. whether mental fatigue affects MI EEG class separability. Eleven participants participated in the MI experiment, 5 of which quit in the middle because of experiencing high fatigue. The growth of fatigue was monitored using the Kernel Partial Least Square (KPLS) algorithm on the remaining 6 participants which shows that MI induces substantial mental fatigue. Statistical analysis of the effect of fatigue on motor imagery performance shows that high fatigue level significantly decreases MI EEG separability. Collectively, these results portray an MI-fatigue inter-connection, emphasizing the necessity of developing adaptive MI BCI by tracking mental fatigue.