Kernel Partial Least Research Articles

Research on Relevance Vector Machine Model Based on Kernel Partial Least Square in Properties of Engineering Materials

To improve the prediction of properties of engineering materials, a Relevance Vector Machine (RVM) regression algorithm based on Kernel Partial Least Squares (KPLS) is proposed. In the algorithm, firstly execute the feature extraction from the original samples using KPLS, and then use obtained feature to realize RVM regression. The simulation shows that the hybrid regression algorithm can effectively reduce the difficulty on RVM modeling and has a wide application in prediction of properties of engineering materials.

Quantifying anatomical shape variations in neurological disorders.

We develop a multivariate analysis of brain anatomy to identify the relevant shape deformation patterns and quantify the shape changes that explain corresponding variations in clinical neuropsychological measures. We use kernel Partial Least Squares (PLS) and formulate a regression model in the tangent space of the manifold of diffeomorphisms characterized by deformation momenta. The scalar deformation momenta completely encode the diffeomorphic changes in anatomical shape. In this model, the clinical measures are the response variables, while the anatomical variability is treated as the independent variable. To better understand the "shape-clinical response" relationship, we also control for demographic confounders, such as age, gender, and years of education in our regression model. We evaluate the proposed methodology on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database using baseline structural MR imaging data and neuropsychological evaluation test scores. We demonstrate the ability of our model to quantify the anatomical deformations in units of clinical response. Our results also demonstrate that the proposed method is generic and generates reliable shape deformations both in terms of the extracted patterns and the amount of shape changes. We found that while the hippocampus and amygdala emerge as mainly responsible for changes in test scores for global measures of dementia and memory function, they are not a determinant factor for executive function. Another critical finding was the appearance of thalamus and putamen as most important regions that relate to executive function. These resulting anatomical regions were consistent with very high confidence irrespective of the size of the population used in the study. This data-driven global analysis of brain anatomy was able to reach similar conclusions as other studies in Alzheimer's disease based on predefined ROIs, together with the identification of other new patterns of deformation. The proposed methodology thus holds promise for discovering new patterns of shape changes in the human brain that could add to our understanding of disease progression in neurological disorders.

Online Prediction Model Based on New Kernel Method

This paper proposes a new online kernel identification method of a nonlinear system. The proposed algorithm entitled online RKPLS-RN kernel method uses the technique Reduced Kernel Partial Least Square (RKPLS) in an offline phase to construct a RKHS model with reduced parameter number. Then the Regularized Network (RN) method is used on online phase to update the reduced parameters of the RKHS model that determined in offline one using the RKPLS technique. The proposed algorithm has been tested to identify an electronic nonlinear system with a Wiener Hammerstein structure

Speech Emotion Recognition with MPCA and Kernel Partial Least Squares Regression

Speech signal is one of the major means for communication which carries not only semantic, but personal information, such as genders and emotions. The researches about speech emotion have become more and more important to human-computer interaction. To this end, from speech, the long-term and short-term emotional features are extracted, the dimensionality of which is then reduced by virtue of the multi linear PCA algorithm. Finally, the kernel partial least squares regression is used for speech emotional recognition. The results show that in comparison with other current classifiers, the algorithm proposed herein can improve recognition rates by about 6% to 23%.

Variable Selection Method for Aluminum Electrolytic Process Based on FNN and RM in KPLS Feature Space

Selecting the process variables is an important prerequisite for establishing an accurate model of aluminum electrolytic process. A variable selection method is researched and proposed based on the False Nearest Neighbors (FNN) and Randomization Method (RM) (FR) in KPLS(Kernel Partial Least Squares) feature space. Firstly, the KPLS is employed to transform the original space to the PLS feature space; secondly, in the new feature space, the FNN is used to calculate the similarity measure of each variable which first is retained and then reset to zero for evaluateing the importance to the dependent variable; then, the RM is utilized to test the significance level of the importance for each variable in turn, so the redundant variables would be excluded; lastly, technical energy consumption model of aluminum electrolytic process is built to verify the presented method. The experimental results show that the method selects out the best process variables of aluminum electrolytic process. Therefore, the research provides a new method of the variable selection for metallurgical industrial processes.

Image Super-resolution Reconstruction Based on Kernel Partial Least Squares and Weighted Boosting

核偏最小二乘(KPLS)算法对每个图像块选用全部主元成分进行图像重建，导致图像超分辨率算法的计算量大。兼顾图像重建质量和时间效率，该文提出一种加权Boosting的图像超分辨率重建算法。为自适应地选取每个图像块主元成分的最佳数目，利用加权Boosting原理对KPLS回归预测量进行补偿，推导给出补偿权重系数的数学表达式。讨论不同Boosting阈值情况下的重建性能，在合适的下，选取出主元成分的最佳数目m更好地满足KPLS回归模型的精度要求。实验结果表明，该文算法的超分辨率重建质量优于传统算法。

A Symmetric Kernel Partial Least Squares Framework for Speaker Recognition

I-vectors are concise representations of speaker characteristics. Recent progress in i-vectors related research has utilized their ability to capture speaker and channel variability to develop efficient automatic speaker verification (ASV) systems. Inter-speaker relationships in the i-vector space are non-linear. Accomplishing effective speaker verification requires a good modeling of these non-linearities and can be cast as a machine learning problem. Kernel partial least squares (KPLS) can be used for discriminative training in the i-vector space. However, this framework suffers from training data imbalance and asymmetric scoring. We use “one shot similarity scoring” (OSS) to address this. The resulting ASV system (OSS-KPLS) is tested across several conditions of the NIST SRE 2010 extended core data set and compared against state-of-the-art systems: Joint Factor Analysis (JFA), Probabilistic Linear Discriminant Analysis (PLDA), and Cosine Distance Scoring (CDS) classifiers. Improvements are shown.

Near-Infrared Spectroscopy Coupled with Kernel Partial Least Squares-Discriminant Analysis for Rapid Screening Water Containing Malathion

Near-infrared spectroscopy coupled with kernel partial least squares-discriminant analysis was used to rapidly screen water containing malathion. In the wavenumber of 4348 cm-1 to 9091 cm-1, the overall correct classification rate of kernel partial least squares-discriminant analysis was 100% for training set, and 100% for test set, with the lowest concentration detected malathion residues in water being 1 μg·ml-1. Kernel partial least squares-discriminant analysis was able to have a good performance in classifying data in nonlinear systems. It was inferred that Near-infrared spectroscopy coupled with the kernel partial least squares-discriminant analysis had a potential in rapid screening other pesticide residues in water.

Prostate Cancer Classification from Mass Spectrometry Data by Using Wavelet Analysis and Kernel Partial Least Squares Algorithm

Prostate Cancer Classification from Mass Spectrometry Data by Using Wavelet Analysis and Kernel Partial Least Squares Algorithm

Kernelized Extreme Learning Machine with Levenberg-Marquardt Learning Approach towards Intrusion Detection

and system security is of vital importance in the present data communication environment. Hackers and intruders can create many successful attempts to cause the crash of the networks and web services by unauthorized intrusion. New threats and associated solutions to prevent these threats are emerging together with the secured system evolution. Intrusion Detection Systems (IDS) are one of these solutions. The main function of Intrusion Detection System is to protect the resources from threats. It analyzes and predicts the behaviours of users, and then these behaviours will be considered an attack or a normal behaviour. There are several techniques which exist at present to provide more security to the network, but most of these techniques are static. On the other hand, intrusion detection is a dynamic one, which can give dynamic protection to the network security by observing the attack. In recent times, Extreme Learning Machine (ELM) has been extensively applied to provide potential solutions for the IDS problem. But, the practicability of ELM is affected because of the complexity in choosing the suitable ELM parameters. Hence, in this paper sigma ( ) of the radial basis kernel function is tuned using Levenberg-Marquardt (LM) learning and proposed kernelized Extreme Learning Machine with LM. In order to obtain a converged solution, LM learning is utilized. The experiment is carried out with the help of WEKA by using KDD Cup 1999 dataset and the results indicate that the proposed technique can achieve higher detection rate, very low false alarm rate and to achieve high accuracy than the regular ELM algorithms. This method is used to decrease the space densisty of the data. KeywordsDetection System (IDS), Extreme Learning Machine (ELM), Radial Basis Function (RBF), Kernel Function, Leave-One-Out (LOO), Kernel Partial Least Squares (K-PLS)

Research on the Line Loss Rate Prediction Technology based on the Kernel Partial Least Squares

Research on the Line Loss Rate Prediction Technology based on the Kernel Partial Least Squares

Interpretation and Visualization of Non-Linear Data Fusion in Kernel Space: Study on Metabolomic Characterization of Progression of Multiple Sclerosis

BackgroundIn the last decade data fusion has become widespread in the field of metabolomics. Linear data fusion is performed most commonly. However, many data display non-linear parameter dependences. The linear methods are bound to fail in such situations. We used proton Nuclear Magnetic Resonance and Gas Chromatography-Mass Spectrometry, two well established techniques, to generate metabolic profiles of Cerebrospinal fluid of Multiple Sclerosis (MScl) individuals. These datasets represent non-linearly separable groups. Thus, to extract relevant information and to combine them a special framework for data fusion is required.MethodologyThe main aim is to demonstrate a novel approach for data fusion for classification; the approach is applied to metabolomics datasets coming from patients suffering from MScl at a different stage of the disease. The approach involves data fusion in kernel space and consists of four main steps. The first one is to extract the significant information per data source using Support Vector Machine Recursive Feature Elimination. This method allows one to select a set of relevant variables. In the next step the optimized kernel matrices are merged by linear combination. In step 3 the merged datasets are analyzed with a classification technique, namely Kernel Partial Least Square Discriminant Analysis. In the final step, the variables in kernel space are visualized and their significance established.ConclusionsWe find that fusion in kernel space allows for efficient and reliable discrimination of classes (MScl and early stage). This data fusion approach achieves better class prediction accuracy than analysis of individual datasets and the commonly used mid-level fusion. The prediction accuracy on an independent test set (8 samples) reaches 100%. Additionally, the classification model obtained on fused kernels is simpler in terms of complexity, i.e. just one latent variable was sufficient. Finally, visualization of variables importance in kernel space was achieved.

A Multiway Gaussian Mixture Model based Adaptive Kernel Partial Least Squares Regression Approach for Inferential Quality Predictions of Batch Processes

Soft sensor technique has become increasingly important to provide reliable on-line measurements, facilitate advanced process control and improve product quality in process industries. The conventional soft sensors are normally single-model based and thus may not be appropriate for processes with shifting operating conditions or phases. In this study, a multiway Gaussian mixture model (MGMM) based kernel partial least squares (PLS) method is proposed to handle multiple operating phases in batch or semibatch processes. The measurement data are first projected onto high-dimensional kernel feature space to account for the process nonlinearity. Then the multiway Gaussian mixture model is estimated with multiple Gaussian clusters in the kernel space. Thus various localized PLS models can be built within each Gaussian cluster to characterize the dynamics in the particular operating phase. Using Bayesian inference strategy, the soft sensor models for all the test samples are adaptively selected from the multiple localized kernel PLS models representing different phases and further used for online quality predictions. The presented soft sensor method is applied to the multiphase penicillin fermentation process and the computational results demonstrate that its performance is superior to the conventional multiway kernel PLS model.

Asymmetric Learning Based on Kernel Partial Least Squares for Software Defect Prediction

Asymmetric Learning Based on Kernel Partial Least Squares for Software Defect Prediction

Improved prediction of biomass composition for switchgrass using reproducing kernel methods with wavelet compressed FT-NIR spectra

Fourier transform near-infrared (FT-NIR) technique is an effective approach to predict chemical properties and can be applied to online monitoring in bio-energy industry. High dimensionality and collinearity of the FT-NIR spectral data makes it difficult in some applications to construct the reliable prediction model. In this study, two nonlinear kernel methods with wavelet-compressed data, Kernel Partial Least Squares (KPLS) regression and Kernel Ridge Regression (KRR), are presented to resolve those data into a few predictors and then, more sophisticated models are created to capture the nonlinear relationships between the spectral data and concentrations determined by wet chemistry. A wavelet transform is adopted as a preprocessing procedure to reduce the data size for supporting real-time implementation of assessing biomass properties with FT-NIR spectroscopy. A real-life data of switchgrass is presented to illustrate the performance of the developed models and the results advocated that the use of nonlinear kernel procedure with wavelet compression improved the prediction performance of the model.

基于核偏最小二乘方法的点模式匹配

Point pattern matching is an essential step in many image processing applications. This letter investigates the spectral approaches of point pattern matching, and presents a spectral feature matching algorithm based on kernel partial least squares (KPLS). Given the feature points of two images, we define position similarity matrices for the reference and sensed images, and extract the pattern vectors from the matrices using KPLS, which indicate the geometric distribution and the inner relationships of the feature points. Feature points matching are done using the bipartite graph matching method. Experiments conducted on both synthetic and real-world data demonstrate the robustness and invariance of the algorithm.

Analysis of complex, processed substances with the use of NIR spectroscopy and chemometrics: Classification and prediction of properties — The potato crisps example

An NIR spectroscopic method was researched and developed for the analysis of potato crisps (chips) chosen as an example of a common, cheap but complex product. Four similar types of the ‘original flavour’ potato chips from different manufacturers were analysed by NIR spectroscopy; as well, the quality parameters — fat, moisture, acid and peroxide values of the extracted oil were predicted. Principal component analysis (PCA) of the NIR data displayed the clustering of objects with respect to the type of chips. NIR spectra were rank-ordered with the use of the sparingly applied multiple criteria decision making (MCDM) ranking methods, PROMETHEE (Preference Ranking Organization METHod for Enrichment Evaluation) and GAIA (Geometrical Analysis for Interactive Aid), and a comprehensive quantitative description of the data was obtained. The four traditional parameters were predicted on the basis of the NIR spectra; the performance of the Partial Least Squares (PLS), and Kernel Partial Least Squares (KPLS) calibrations was compared with those from Least Squares-Support Vector Machines (LS-SVM) method. The LS-SVM calibrations, which model better data linearity and non-linearity, successfully predicted all four parameters. This work has demonstrated that NIR methodology with the use of chemometrics can describe comprehensively qualitative and quantitative properties of complex, processed substances as illustrated by the potato chips example, and indicated that this approach may be applied to other similar complex samples.

Application of Kernel Partial Least Square to Predict Biological Hydrogen Production by Enterobacter aerogenes

Hydrogen production by dark fermentation, where hydrogenase enzymes catalyze the oxidation or evolution of molecular hydrogen from two protons (H + ) and electrons, is an economic and environmentally friendly technology for producing clean energy. However, the long-term operations of a continuous anaerobic reactor for fermentative hydrogen production were frequently unstable. In this study, a kernel partial least squares (KPLS) algorithm is employed to develop an online estimation of the key process variables in a biological hydrogen production process by Enterobacter aerogenes in minimal time and with minimal cost. The KPLS approach is potentially very efficient for predicting key quality variables of nonlinear processes by mapping an original input space into a high-dimensional feature space. The proposed kernel-based algorithm could effectively capture the nonlinear relationship in the process variables and show far better performance in the prediction of the key process variable compared with the conventional linear PLS and other nonlinear PLS methods.

Decentralized Fault Diagnosis of Large-Scale Processes Using Multiblock Kernel Partial Least Squares

In this paper, a decentralized fault diagnosis approach of complex processes is proposed based on multiblock kernel partial least squares (MBKPLS). To solve the problem posed by nonlinear characteristics, kernel partial least squares (KPLS) approaches have been proposed. In this paper, MBKPLS algorithm is first proposed and applied to monitor large-scale processes. The advantages of MBKPLS are: 1) MBKPLS can capture more useful information between and within blocks compared to partial least squares (PLS); 2) MBKPLS gives nonlinear interpretation compared to MBPLS; 3) Fault diagnosis becomes possible if number of sub-blocks is equal to the number of the variables compared to KPLS. The proposed methods are applied to process monitoring of a continuous annealing process. Application results indicate that the proposed decentralized monitoring scheme effectively captures the complex relations in the process and improves the diagnosis ability tremendously.

Prediction for Biodegradability of Chemicals by Kernel Partial Least Squares

Prediction for Biodegradability of Chemicals by Kernel Partial Least Squares