Kernel Partial Least Squares Research Articles

Prediction of octanol–water partition coefficients of organic chemicals by QSAR models

A quantitative structure–activity relationship was developed using the partial least square (PLS), kernel PLS (KPLS) and Levenberg–Marquardt artificial neural network (L-M ANN) approach for chemometrics study. The logarithm octanol–water partition coefficients (logKow) of 47 organic chemicals in effluents were obtained by reversed-phase high-performance liquid chromatography. A genetic algorithm (GA) was employed as factor selection procedure for PLS and KPLS modeling. For comparison, GA-PLS descriptors were selected for L-M ANN. A model with low prediction errors and good correlation coefficients was obtained by L-M ANN.

Fault Identification of Nonlinear Processes

In this paper, a new kernel partial least-squares (KPLS)-based fault identification method is proposed. Although KPLS is superior to PLS in fault detecting of nonlinear processes, the fault identification methods for KPLS are limited. In this paper, the contributions are (1) The relationship between the input and the output variables are considered and each variable’s contribution is measured using the gradient of kernel function. In the existing work, only input variables are concerned; (2) The complex computation is avoided since the new computation method of the partial derivative in the kernel matrix is introduced. The proposed method has two advantages: the ability to identify faulty variables in nonlinear process and guarantee correct diagnosis of simple sensor faults, compared with PLS using conventional contribution plots. In the end, case study on a numerical example and the electro-fused magnesia furnace (EFMF) is employed to illustrate the effectiveness of the proposed method, where the compari...

Application of a Combination Model Based on Wavelet Transform and KPLS-ARMA for Urban Annual Water Demand Forecasting

AbstractA combination of models including wavelet transform and kernel partial least squares-autoregressive moving average (KPLS-ARMA) is proposed to explore the nonstationarity of the urban annual water demand series, the nonlinear relationships between water demand series and its determinants, and the high correlations among those determinants, based on which a novel forecast model is proposed for urban annual water demand. First, by Mallat algorithm, a nonstationary urban annual water demand series is decomposed and reconstructed into one low-frequency component and one or several high-frequency components. Following that, the kernel partial least squares (KPLS) model is applied to simulating the low-frequency component. An autoregressive moving average (ARMA) model is constructed for each of the high-frequency components. The combined models are applied to understanding the nonstationarity and forecasting the annual water demand of Dalian City. The results are then compared with those from other sever...

A Symmetric Kernel Partial Least Squares Framework for Speaker Recognition

I-vectors are concise representations of speaker characteristics. Recent progress in i-vectors related research has utilized their ability to capture speaker and channel variability to develop efficient automatic speaker verification (ASV) systems. Inter-speaker relationships in the i-vector space are non-linear. Accomplishing effective speaker verification requires a good modeling of these non-linearities and can be cast as a machine learning problem. Kernel partial least squares (KPLS) can be used for discriminative training in the i-vector space. However, this framework suffers from training data imbalance and asymmetric scoring. We use “one shot similarity scoring” (OSS) to address this. The resulting ASV system (OSS-KPLS) is tested across several conditions of the NIST SRE 2010 extended core data set and compared against state-of-the-art systems: Joint Factor Analysis (JFA), Probabilistic Linear Discriminant Analysis (PLDA), and Cosine Distance Scoring (CDS) classifiers. Improvements are shown.

Modeling Load Parameters of Ball Mill in Grinding Process Based on Selective Ensemble Multisensor Information

Due to complex dynamic characteristics of the ball mill system, it is difficult to measure load parameters inside the ball mill. It has been noticed that the traditional single-model and ensemble-model based soft sensor approaches demonstrate weak generalization power. Also, mill motor current, feature subsets of the shell vibration and acoustical frequency spectra contain different useful information. To achieve better solutions and overcome these problems mentioned above, a selective ensemble multisource information approach is proposed in this paper. Only the useful feature subsets of vibration and acoustical frequency spectra are portioned and selected. Some modeling techniques, such as fast Fourier transform (FFT), mutual information (MI), kernel partial least square (KPLS), brand and band (BB), and adaptive weighting fusion (AWF), are combined effectively to model the mill load parameters. The simulation is conducted using real data from a laboratory-scale ball mill. The results show that our proposed approach can effectively fusion the shell vibration, acoustical and mill motor current signals with improved model generalization.

Robust Online Monitoring Based on Spherical-Kernel Partial Least Squares for Nonlinear Processes with Contaminated Modeling Data

KPLS is a very efficient technique for tackling complex nonlinear data sets by mapping an original input space into a high-dimensional feature space. However, KPLS may not function well when the model set is contaminated to a large extent by outliers. In this Article, a robust version of kernel partial least squares (KPLS) called spherical kernel partial least squares (SKPLS) is introduced for monitoring nonlinear processes. The key idea of SKPLS is to project all of the feature vectors in the feature space onto a unit sphere and then to perform KPLS on the sphered feature vectors. The effects of the outliers in the original data are eliminated or diminished because of the sphering. The robust monitoring statistics and robust control limits are derived for process monitoring purposes. The simulation results show that the proposed process monitoring strategy not only works well when the modeling data set does not contain outliers but also offers a satisfactory efficiency when the modeling data set is highl...

Prediction of Retention Behavior of Pesticides in Fruits and Vegetables in Low-Pressure Gas Chromatography–Time-of-Flight Mass Spectrometry

Pesticides are substances or mixture of substances intended for preventing, destroying, repelling, or mitigating any pest. A pesticide may be a chemical substance, biological agent (such as a virus or bacterium), antimicrobial, disinfectant, or device used against any pest. Although the use of pesticides has many benefits, there are also drawbacks, such as potential toxicity to humans and other animals. A quantitative structure–retention relationship (QSRR) was developed using the partial least squares (PLS), kernel PLS (KPLS), and Levenberg–Marquardt artificial neural network (L-M ANN) approach for chemometrics study. The data which contained retention time (RT) of the 150 pesticides in tomato, strawberry, potato, orange, and lettuce samples were obtained by low-pressure gas chromatography–time-of-flight mass spectrometry. Genetic algorithm was employed as a factor selection procedure for PLS and KPLS modeling methods. By comparing the results, genetic algorithm–PLS descriptors are selected for L-M ANN. Finally, a model with a low prediction error and a good correlation coefficient was obtained by L-M ANN. The described model does not require experimental parameters and potentially provides useful prediction for the RT of new compounds. This is the first research on the QSRR of pesticides in fruits and vegetables using the chemometrics models.

KPLS-RWBFNN model for MFL 2D defect profile reconstruction

Kernel partial least squares (KPLS) is normally very efficient for tackling nonlinear systems by mapping an original input space into a high-dimensional feature space and creating a linear PLS model in the feature space. Unlike other nonlinear PLS techniques, KPLS does not entail any nonlinear optimisation procedures. However, due to the linear inner model of PLS, KPLS is still inappropriate for describing the significant nonlinear characteristic data structure while dealing with complex physical systems in practical situations. Under this circumstance, radial wavelet basic function neural network (RWBFNN) can replace the linear inner model of PLS in the nonlinear kernel-based algorithm. Thus, KPLS-RWBFNN model is proposed in this paper and applied to multi-resolution approximation reconstruction of 2D defect profiles in magnetic flux leakage testing. The reconstructions of 2D defect profiles by this method are implemented, and the comparisons among reconstructions by KPLS, RWBFNN and the proposed approach are also undertaken. Meanwhile, the reconstructions of 2D defects by RWBFNN and the proposed approach at different SNR are also executed. The results indicate that KPLS-RWBFNN model could simplify the structure of the network while holding well-behaved generalisation and multi-resolution approximation and predict the 2D defect profiles accurately and rapidly with good robustness.

A novel tree kernel partial least squares for modeling the structure–activity relationship

Kernel partial least squares (KPLS) has become a popular technique for regression and classification of complex data sets, which is a nonlinear extension of linear PLS in which training samples are transformed into a feature space via a nonlinear mapping. The PLS algorithm can then be carried out in the feature space. In the present study, we attempt to develop a novel tree KPLS (TKPLS) classification algorithm by constructing an informative kernel on the basis of decision tree ensembles. The constructed tree kernel can effectively discover the similarities of samples and select informative features by variable importance ranking in the process of building the kernel. Simultaneously, TKPLS can also handle nonlinear relationships in the structure–activity relationship data by such a kernel. Finally, three data sets related to different categorical bioactivities of compounds are used to evaluate the performance of TKPLS. The results show that the TKPLS algorithm can be regarded as an alternative and promising classification technique. Copyright © 2013 John Wiley & Sons, Ltd.

Enhanced batch process monitoring using just-in-time-learning based kernel partial least squares

Batch process has the characteristics of dynamic behavior, strong nonlinearity and multiplicity of operation phases. In this paper, an integrated framework consisting of just-in-time-learning (JITL) method and kernel partial least squares (KPLS) is described for the monitoring of the performance of batch processes. The training data set for modeling is defined by using JITL, and KPLS is employed to build the model for process monitoring purpose. The JITL model based on local neighborhoods of similar samples is very accurate and sensitive because it can well track the change of the process. Therefore, the proposed method is suited for multiphase and time-varying batch processes. Meanwhile, KPLS is a very efficient technique for tacking complex nonlinear data sets. Furthermore, the proposed monitoring strategy does not require the estimation of future values of the process variables during online application. The effectiveness of the proposed monitoring approach is demonstrated through two sets of benchmark data. The comparison of monitoring results shows that the proposed JITL–KPLS approach is superior to multiway KPLS and hierarchical KPLS, and it can achieve accurate detection of various types of process faults occurring in the batch operation.

Estimation value of plasma amino acid target analysis to the acute radiation injury early triage in the rat model

Rapid radiation injury early triage for large-scale people after radiation exposure is vital for limited medical resources allocation and early treatment of a large number of wounded after a nuclear accident. Owing to the high-throughput analysis and minimally invasive of collection sample, radiation metabolomics has been recently applied to radiation damage researches. Here, exploration the feasibility of estimating the acute radiation injury early triage by means of plasma amino acid target analysis was attempted using high performance liquid chromatography–electrospray tandem mass spectrometry (HPLC–ESI–MS/MS) technique. The nonlinear kernel partial least squares (KPLS) model was used to classify the radiation damage levels. The classification accuracy of without radiation exposure was 92.3 % at 5 h after exposure. At 24 h after exposure, the triage accuracies were all above 83 % in the different doses of irradiated groups, the correct classification rates of moderate and severe radiation injury were 91.7 and 92.3 % respectively. At 72 h after exposure, the classification accuracies of all levels of radiation injury were more than 90 %, the correct classification rates of moderate and severe groups were up to 100 %. This approach is useful for early predicting different levels of radiation exposure and for developing metabolomics strategies for radiation biodosimetry in humans, but need more data to consummate.

AN IMPROVED QSPR STUDY OF REVERSE FACTOR OF NANOPARTICLES IN ROADSIDE ATMOSPHERE ON KERNEL PARTIAL LEAST SQUARES AND GENETIC ALGORITHM

Thermal desorption-comprehensive two-dimensional gas chromatography high-resolution time-of-flight mass spectrometry (TD–GC × GC–HRTOF-MS) is one of the most powerful tools in analytical nanoparticle compounds. Genetic algorithm and partial least square (GA-PLS) and kernel PLS (GA-KPLS) models were used to investigate the correlation between reverse factor (RF) and descriptors for 50 nanoparticles fraction with a diameter of 29–58 nm in roadside atmosphere which obtained by TD–GC×GC–HRTOF-MS. The correlation coefficient leave-group-out cross validation (LGO-CV (Q2)) of prediction for the GA-PLS and GA-KPLS models for training and test sets were (0.761 and 0.718) and (0.825 and 0.814), respectively, revealing the reliability of these models. This is the first research on the quantitative structure-property relationship (QSPR) of the nanoparticles in roadside atmosphere using the GA-PLS and GA-KPLS.

Application of Improved Three-Dimensional Kernel Approach to Prediction of Protein Structural Class

Kernel methods, such as kernel PCA, kernel PLS, and support vector machines, are widely known machine learning techniques in biology, medicine, chemistry, and material science. Based on nonlinear mapping and Coulomb function, two 3D kernel approaches were improved and applied to predictions of the four protein tertiary structural classes of domains (all-α, all-β, α/β, and α + β) and five membrane protein types with satisfactory results. In a benchmark test, the performances of improved 3D kernel approach were compared with those of neural networks, support vector machines, and ensemble algorithm. Demonstration through leave-one-out cross-validation on working datasets constructed by investigators indicated that new kernel approaches outperformed other predictors. It has not escaped our notice that 3D kernel approaches may hold a high potential for improving the quality in predicting the other protein features as well. Or at the very least, it will play a complementary role to many of the existing algorithms in this regard.

Product quality modelling and prediction based on wavelet relevance vector machines

In order to predict product quality and optimize production process, the product quality models need to be built. However, there are complex nonlinear relationship among the product quality parameters and the production process variables. The common methods cannot model the production process with high accuracy and the prediction intervals cannot be given by those methods. In contrast, the kernel methods can transform the original input data into a feature space via kernel function, and then the linear methods can be used to resolve the nonlinear problem accurately. Moreover, the relevance vector machine as a kernel method can give the prediction intervals, and wavelet kernel can inherit the ability of local analysis and feature extraction from the wavelet function. The product quality models based on wavelet relevance vector machine are proposed in this paper. A simulation data set, two chemistry data sets and a real field data set of zinc coating weights from strip hot-dip galvanizing are used to validate the model. The results demonstrate that the model based on wavelet relevance vector machines has a higher prediction precision than the common methods such as partial least squares(PLS), orthogonal signal correction-partial least squares(OSC-PLS), Quadratic-PLS, kernel partial least squares(KPLS), orthogonal signal correction-kernel partial least squares(OSC-PLS), least squares-support vector machines (LS-SVM) and ordinary relevance vector machines(RVM). The prediction intervals are also given by the presented model. Mexican, Morlet and Difference of Gaussian (DOG) wavelet relevance vector machines (WRVMs) for multi-group data show superior prediction performance compared to other methods mentioned above.

Kernelized Extreme Learning Machine with Levenberg-Marquardt Learning Approach towards Intrusion Detection

and system security is of vital importance in the present data communication environment. Hackers and intruders can create many successful attempts to cause the crash of the networks and web services by unauthorized intrusion. New threats and associated solutions to prevent these threats are emerging together with the secured system evolution. Intrusion Detection Systems (IDS) are one of these solutions. The main function of Intrusion Detection System is to protect the resources from threats. It analyzes and predicts the behaviours of users, and then these behaviours will be considered an attack or a normal behaviour. There are several techniques which exist at present to provide more security to the network, but most of these techniques are static. On the other hand, intrusion detection is a dynamic one, which can give dynamic protection to the network security by observing the attack. In recent times, Extreme Learning Machine (ELM) has been extensively applied to provide potential solutions for the IDS problem. But, the practicability of ELM is affected because of the complexity in choosing the suitable ELM parameters. Hence, in this paper sigma ( ) of the radial basis kernel function is tuned using Levenberg-Marquardt (LM) learning and proposed kernelized Extreme Learning Machine with LM. In order to obtain a converged solution, LM learning is utilized. The experiment is carried out with the help of WEKA by using KDD Cup 1999 dataset and the results indicate that the proposed technique can achieve higher detection rate, very low false alarm rate and to achieve high accuracy than the regular ELM algorithms. This method is used to decrease the space densisty of the data. KeywordsDetection System (IDS), Extreme Learning Machine (ELM), Radial Basis Function (RBF), Kernel Function, Leave-One-Out (LOO), Kernel Partial Least Squares (K-PLS)

Multiway Gaussian Mixture Model Based Adaptive Kernel Partial Least Squares Regression Method for Soft Sensor Estimation and Reliable Quality Prediction of Nonlinear Multiphase Batch Processes

The predictive model based soft sensor technique has become increasingly important to provide reliable online measurements, facilitate advanced process control and optimization, and improve product quality in process industries. The conventional soft sensors are normally single-model based and thus may not be appropriate for processes with shifting operating phases or conditions and the underlying changing dynamics. In this study, a multiway Gaussian mixture model (MGMM) based adaptive kernel partial least-squares (AKPLS) method is proposed to handle online quality prediction of batch or semibatch processes with multiple operating phases. The three-dimensional measurement data are first preprocessed and unfolded into two-dimensional matrix. Then, the multiway Gaussian mixture model is estimated in order to identify and isolate different operating phases. Further, the process and quality measurements are separated into multiple segments corresponding to those identified phases, and the various localized kernel PLS models are built in the high-dimensional nonlinear feature space to characterize the shifting dynamics across different operating phases. Using Bayesian inference strategy, each process measurement sample of a new batch is classified into a particular phase with the maximal posterior probability, and thus, the local kernel PLS model representing the identical phase can be adaptively chosen for online quality variable prediction. The presented soft sensor modeling method is applied to a simulated multiphase penicillin fermentation process, and the computational results demonstrate that the proposed MGMM-AKPLS approach is superior to the conventional kernel PLS model in terms of prediction accuracy and model reliability.

Linear and non-linear chemometric modeling of THM formation in Barcelona's water treatment plant

The complex behavior observed for the dependence of trihalomethane formation on forty one water treatment plant (WTP) operational variables is investigated by means of linear and non-linear regression methods, including kernel-partial least squares (K-PLS), and support vector machine regression (SVR). Lower prediction errors of total trihalomethane concentrations (lower than 14% for external validation samples) were obtained when these two methods were applied in comparison to when linear regression methods were applied. A new visualization technique revealed the complex nonlinear relationships among the operational variables and displayed the existing correlations between input variables and the kernel matrix on one side and the support vectors on the other side. Whereas some water treatment plant variables like river water TOC and chloride concentrations, and breakpoint chlorination were not considered to be significant due to the multi-collinear effect in straight linear regression modeling methods, they were now confirmed to be significant using K-PLS and SVR non-linear modeling regression methods, proving the better performance of these methods for the prediction of complex formation of trihalomethanes in water disinfection plants.

Detection and Quantification of Bacterial Spoilage in Milk and Pork Meat Using MALDI-TOF-MS and Multivariate Analysis

Microbiological safety is one of the cornerstones of quality control in the food industry. Identification and quantification of spoilage bacteria in pasteurized milk and meat in the food industry currently relies on accurate and sensitive yet time-consuming techniques which give retrospective values for microbial contamination. Matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOF-MS), a proven technique in the field of protein and peptide identification and quantification, may be a valuable alternative approach for the rapid assessment of microbial spoilage. In this work we therefore developed MALDI-TOF-MS as a novel analytical approach for the assessment of food that when combined with chemometrics allows for the detection and quantification of milk and pork meat spoilage bacteria. To develop this approach, natural spoilage of pasteurized milk and raw pork meat samples incubated at 15 °C and at room temperature, respectively, was conducted. Samples were collected for MALDI-TOF-MS analysis (which took 4 min per sample) at regular time intervals throughout the spoilage process, with concurrent calculation and documentation of reference total viable counts using traditional microbiological methods (these took 2 days). Multivariate statistical techniques such as principal component discriminant function analysis, canonical correlation analysis, partial least-squares (PLS) regression, and kernel PLS (KPLS) were used to analyze the data. The results from MALDI-TOF-MS combined with PLS or KPLS gave excellent bacterial quantification results for both milk and meat spoilage, and typical root mean squared errors for prediction in test spectra were between 0.53 and 0.79 log unit. Overall these novel findings strongly indicate that MALDI-TOF-MS when combined with chemometric approaches would be a useful adjunct for routine use in the milk and meat industry as a fast and accurate viable bacterial detection and quantification method.

Performance modeling of centrifugal compressor using kernel partial least squares

The performance modeling of centrifugal (turbo) compressor was performed in this paper by applying kernel partial least squares (KPLS). Firstly, steady-state compressor data sets were collected from a real gas turbine power plant and a simulation study respectively. Then the two data sets were used to train the KPLS regression model for predicting the centrifugal compressor operating parameters such as pressure ratio and efficiency. The prediction performance of KPLS model was compared to that of a three-layer back-propagation (BP) neural network with validation data, and the KPLS model showed slightly better performance than the BP network. Furthermore, results showed that, with high accuracy, KPLS could be used to smoothly predict the compressor map, which was useful in the preliminary design phase of any centrifugal compression system.

Chemometrics analysis for investigation of retention behavior of hazardous compounds in effluents

The toxic substances, pesticides, and organic contaminants in effluents can potentially be causing damage that includes increased cancer risk; liver, kidney, stomach, nervous system, and immune system problems; reproductive difficulties; cataracts; and anemia. A quantitative structure-retention relationship (QSRR) was developed using the partial least square (PLS), kernel PLS (KPLS), and Levenberg-Marquardt artificial neural network (L-M ANN) approach for chemometrics study. The data which contained retention time (RT) of the 47 hazardous compounds in effluents were obtained by reverse-phase high-performance liquid chromatography. Genetic algorithm was employed as a factor selection procedure for PLS and KPLS modeling methods. By comparing the results, GA-PLS descriptors are selected for L-M ANN. Finally, a model with a low prediction error and a good correlation coefficient was obtained by L-M ANN. The described model does not require experimental parameters and potentially provides useful prediction for RT of new compounds. This is the first research on the QSRR of hazardous compounds in effluents using the chemometrics models.