Kernel Partial Least Squares Regression Research Articles

Mango quality prediction based on near-infrared spectroscopy using multi-predictor local polynomial regression modeling

Background pH and total soluble solids (TSS) are important quality parameters of mangoes; they represent the acidity and sweetness of the fruit, respectively. This study predicts the pH and TSS of intact mangoes based on near-infrared (NIR) spectroscopy using multi-predictor local polynomial regression (MLPR) modeling. Herein, the prediction performance of kernel partial least square regression (KPLSR), support vector machine regression (SVMR), and MLPR is compared. Methods For this purpose, 186 intact mango samples at three different maturity stages are used. Prediction models are built using MLPR, KPLSR, and SVMR based on untreated and treated spectra. The best regression model for predicting pH is MLPR based on Gaussian filter smoothing spectra. Moreover, the TSS value is more accurately predicted using MLPR based on Savitzky–Golay smoothing. Results The findings reveal that MLPR is highly accurate in estimating the pH and TSS of mangoes, with mean absolute percentage error (MAPE) values less than 10 %. In addition, the MLPR model has the best predictive performance with the lowest Mean Squared error (MSE) and root mean squared error (RMSE) values and the highest R2 value. Conclusions The use of NIR spectroscopy in combination with multi-predictor local polynomial regression could provide a quick and non-destructive technique for predicting mango quality. Thus, the results of this study help support sustainable production as a sustainable development goal.

Mango quality prediction based on near-infrared spectroscopy using multi-predictor local polynomial regression modeling

Background: pH and total soluble solids (TSS) are important quality parameters of mangoes; they represent the acidity and sweetness of the fruit, respectively. This study predicts the pH and TSS of intact mangoes based on near-infrared (NIR) spectroscopy using multi-predictor local polynomial regression (MLPR) modeling. Herein, the prediction performance of kernel partial least square regression (KPLSR), support vector machine regression (SVMR), and MLPR is compared. Methods: For this purpose, 186 intact mango samples at three different maturity stages are used. Prediction models are built using MLPR, KPLSR, and SVMR based on untreated and treated spectra. The best regression model for predicting pH is MLPR based on Gaussian filter smoothing spectra. Moreover, the TSS value is more accurately predicted using MLPR based on Savitzky–Golay smoothing. Results: The findings reveal that MLPR is highly accurate in estimating the pH and TSS of mangoes, with mean absolute percentage error (MAPE) values less than 10 %. In addition, the MLPR model has the best predictive performance with the lowest Mean Squared error (MSE) and root mean squared error (RMSE) values and the highest R2 value. Conclusions: The use of NIR spectroscopy in combination with multi-predictor local polynomial regression could provide a quick and non-destructive technique for predicting mango quality. Thus, the results of this study help support sustainable production as a sustainable development goal.

Improved Predictive Ability of KPLS Regression with Memetic Algorithms

Kernel partial least squares regression (KPLS) is a non-linear method for predicting one or more dependent variables from a set of predictors, which transforms the original datasets into a feature space where it is possible to generate a linear model and extract orthogonal factors also called components. A difficulty in implementing KPLS regression is determining the number of components and the kernel function parameters that maximize its performance. In this work, a method is proposed to improve the predictive ability of the KPLS regression by means of memetic algorithms. A metaheuristic tuning procedure is carried out to select the number of components and the kernel function parameters that maximize the cumulative predictive squared correlation coefficient, an overall indicator of the predictive ability of KPLS. The proposed methodology led to estimate optimal parameters of the KPLS regression for the improvement of its predictive ability.

Comprehensive 2D-Quantitative Structure-Activity Relationship Study on Monobactam Analogues Against Gram-Negative Bacteria.

Human health has been severely affected by infections resulting from multidrug-resistant (MDR) gram-negative bacteria (GNB). Monobactam antibiotics are known to be effective against such infections. This study aimed to construct a predictive two-dimensional quantitative structure-activity relationship (2D-QSAR) model for the rational design of new monobactams based on the 65 known monobactams against Escherichia coli (Eco) and Klebsiella pneumonia (Kpn) strains using the kernel partial least squares regression (KPLS) algorithm. The total performance of Eco and Kpn KPLS modes was shown as RMSE: 0.681/0.596, R²: 0.946/0.882, Q²: 0.922/0.877, and RMSU: 0.625/0.593. Thirty-four monobactams reported in our lab were chosen as external data to predict their activities against Eco and Kpn using the newly established models, by which the R² between the experimental and predicted values was 0.878 and 0.871, respectively. The models developed and verified in this study provide a powerful design strategy for novel monobactams that are effective against MDR gram-negative bacterial infections.

Investigation of multivariate seismic surrogate demand modeling for multi-response structural systems

Probabilistic seismic risk assessment of complex multi-response structural systems requires accurate estimations of multiple engineering demand parameters (EDPs) that are related to different failure modes. One conventional way is developing multiple separate univariate seismic demand models for each of the EDPs. However, such an approach lacks sufficient consideration of the correlations among different EDPs and also requires repetitive model tuning and training. Another alternative, which has not yet been extensively studied, is to develop a single multivariate surrogate demand model (MvSDM) that is able to jointly provide the estimates for all the EDPs. To facilitate more accurate and realistic seismic demand modeling of multi-response structural systems, the present study conducts a comprehensive investigation of different MvSDMs, which are constituted by a systematic trend model that characterizes the mean response hypersurface, and a covariance model that quantifies the correlated model errors. Three types of systematic trend models including multivariate linear regression (MvLR), partial least squares regression (PLSR) and artificial neural network (ANN); along with three model error covariance estimation methods including joint probabilistic demand model (JPSDM) method, sample regression residual covariance (SCOV) method and multi-output Gaussian process regression (MOGP), are considered. The influence of trend model and error covariance model selection on seismic demand and fragility modeling is then studied based on a typical three-span concrete-girder highway bridge structure in the Central and Southeastern US. The results suggest that kernel PLSR and ANN as the systematic trend models, and the SCOV method as the error covariance model are promising alternatives that can contribute to accurate and reliable multivariate surrogate demand modeling yet with good computational efficiency.

KPLS optimization approach using genetic algorithms

Abstract Kernel partial least squares regression (KPLS) is a technique widely used in the construction of predictive models. However, the adjustment of both the parameter of the kernel function and the number of components supposes for the researcher an unavoidable additional task. This paper presents a procedure that optimizes the generalization capacity of KPLS multivariate models using genetic algorithms (GA), selects the values of the kernel function parameter and the number of components for which the value of the cross-validation coefficient Q2cum is maximum, adds preliminary tests to configure the GA and defines a convergence criterion in terms of dispersion in the estimates. GA has demonstrated a good performance in the task of optimizing KPLS with convergent solutions towards a global optimum.

KPLS Optimization With Nature-Inspired Metaheuristic Algorithms

Kernel partial least squares regression (KPLS) is a technique used in several scientific areas because of its high predictive ability. This article proposes a methodology to simultaneously estimate both the parameters of the kernel function and the number of components of the KPLS regression to maximize its predictive ability. A metaheuristic optimization problem was proposed taking the cumulative cross-validation coefficient as an objective function to be maximized. It was solved using nature-inspired metaheuristic algorithms: the genetic algorithm, particle swarm optimization, grey wolf optimization and the firefly algorithm. To validate the results and have a reference measure of the efficiency of the nature-inspired metaheuristic algorithms, derivative-free optimization algorithms were also applied: Hooke-Jeeves and Nelder-Mead. The metaheuristic algorithms estimated optimal values of both of the kernel function parameters and the number of components in the KPLS regression.

Orders of magnitude speed increase in partial least squares feature selection with new simple indexing technique for very tall data sets

Feature selection is a challenging combinatorial optimization problem that tends to require a large number of candidate feature subsets to be evaluated before a satisfying solution is obtained. Because of the computational cost associated with estimating the regression coefficients for each subset, feature selection can be an immensely time‐consuming process and is often left inadequately explored. Here, we propose a simple modification to the conventional sequence of calculations involved when fitting a number of feature subsets to the same response data with partial least squares (PLS) model fitting. The modification consists in establishing the covariance matrix for the full set of features by an initial calculation and then deriving the covariance of all subsequent feature subsets solely by indexing into the original covariance matrix. By choosing this approach, which is primarily suitable for tall design matrices with significantly more rows than columns, we avoid redundant (identical) recalculations in the evaluation of different feature subsets. By benchmarking the time required to solve regression problems of various sizes, we demonstrate that the introduced technique outperforms traditional approaches by several orders of magnitude when used in conjunction with PLS modeling. In the supplementary material, we provide code for implementing the concept with kernel PLS regression.

A real-time and registration-free framework for dynamic shape instantiation.

Real-time 3D navigation during minimally invasive procedures is an essential yet challenging task, especially when considerable tissue motion is involved. To balance image acquisition speed and resolution, only 2D images or low-resolution 3D volumes can be used clinically. In this paper, a real-time and registration-free framework for dynamic shape instantiation, generalizable to multiple anatomical applications, is proposed to instantiate high-resolution 3D shapes of an organ from a single 2D image intra-operatively. Firstly, an approximate optimal scan plane was determined by analyzing the pre-operative 3D statistical shape model (SSM) of the anatomy with sparse principal component analysis (SPCA) and considering practical constraints. Secondly, kernel partial least squares regression (KPLSR) was used to learn the relationship between the pre-operative 3D SSM and a synchronized 2D SSM constructed from 2D images obtained at the approximate optimal scan plane. Finally, the derived relationship was applied to the new intra-operative 2D image obtained at the same scan plane to predict the high-resolution 3D shape intra-operatively. A major feature of the proposed framework is that no extra registration between the pre-operative 3D SSM and the synchronized 2D SSM is required. Detailed validation was performed on studies including the liver and right ventricle (RV) of the heart. The derived results (mean accuracy of 2.19mm on patients and computation speed of 1ms) demonstrate its potential clinical value for real-time, high-resolution, dynamic and 3D interventional guidance.

Linear guided autoencoder: Representation learning with linearity

In materials science, good representations of materials are important for use with prediction models in order to ensure accurate prediction of the properties of the output. In this paper, in order to address this issue, we use a learning system, linear guided autoencoder (LGAE) we call, which consists of an autoencoder and a linear predictor. For the autoencoder, we adopt a variant of the denoising autoencoder. In the LGAE, the learning addresses the unsupervised and supervised tasks simultaneously. Thus, the LGAE can be regarded as a form of nonlinear partial least squares (PLS) regression. Previous studies have not found the optimal solution for the encoder for an objective that contains both tasks. Our main contributions are a first-order approximation of the optimal solution and determination of the condition for linear solution that applies to the LGAE after training, in order to acquire knowledge from the nonlinear model (i.e., the LGAE). The main drawback of nonlinear PLS regression is that it is difficult to interpret the latent representation. Therefore, we propose a technical method for interpreting the latent representation. Experiments on benchmark datasets are conducted in order to compare the LGAE with kernel PLS regression, which is a powerful nonlinear PLS regression method. We also applied the LGAE to a dataset of methane storage materials in order to interpret the methane uptake based on the input variables and obtained reasonable results.

Mapping of macro and micro nutrients of mixed pastures using airborne AisaFENIX hyperspectral imagery

On-farm assessment of mixed pasture nutrient concentrations is important for animal production and pasture management. Hyperspectral imaging is recognized as a potential tool to quantify the nutrient content of vegetation. However, it is a great challenge to estimate macro and micro nutrients in heterogeneous mixed pastures. In this study, canopy reflectance data was measured by using a high resolution airborne visible-to-shortwave infrared (Vis–SWIR) imaging spectrometer measuring in the wavelength region 380–2500nm to predict nutrient concentrations, nitrogen (N) phosphorus (P), potassium (K), sulfur (S), zinc (Zn), sodium (Na), manganese (Mn) copper (Cu) and magnesium (Mg) in heterogeneous mixed pastures across a sheep and beef farm in hill country, within New Zealand. Prediction models were developed using four different methods which are included partial least squares regression (PLSR), kernel PLSR, support vector regression (SVR), random forest regression (RFR) algorithms and their performance compared using the test data. The results from the study revealed that RFR produced highest accuracy (0.55⩽R2CV⩽0.78; 6.68%⩽nRMSECV⩽26.47%) compared to all other algorithms for the majority of nutrients (N, P, K, Zn, Na, Cu and Mg) described, and the remaining nutrients (S and Mn) were predicted with high accuracy (0.68⩽R2CV⩽0.86; 13.00%⩽nRMSECV⩽14.64%) using SVR. The best training models were used to extrapolate over the whole farm with the purpose of predicting those pasture nutrients and expressed through pixel based spatial maps. These spatially registered nutrient maps demonstrate the range and geographical location of often large differences in pasture nutrient values which are normally not measured and therefore not included in decision making when considering more effective ways to utilized pasture.

A Quality-Based Nonlinear Fault Diagnosis Framework Focusing on Industrial Multimode Batch Processes

This paper proposes a framework for quality-based fault detection and diagnosis for nonlinear batch processes with multimode operating environment. The framework seeks to address 1) the mode partition problem using a kernel fuzzy C-clustering method, and the optimal cluster number will be guaranteed by a between-within proportion index; 2) the diagnosis problem using a contribution rate method based on an improved kernel partial least squares (PLS) model, by which better detection and diagnosis performances are provided; and 3) the classification of online measurements using a hybrid kernel PLS regression and the Bayes inference theory, where the new coming measurement can be correctly assigned to its constituent mode. The whole framework is developed for batch processes, and applied to the hot strip mill rolling process. It is shown using the real industrial data that for faults affecting the thickness and flatness of the strip steel in this process, the detection and diagnosis abilities of the present methods are better compared with the existing methods.

A K-PLSR-based color correction method for TCM tongue images under different illumination conditions

In this paper a Kernel Partial Least Squares Regression (K-PLSR)-based color correction method for Traditional Chinese Medicine (TCM) tongue images under different illumination conditions has been proposed. The captured values under different illumination conditions and their reference values of 24 patches in the Munsell colorchecker are respectively considered as the input and the output. The mapping model between the input and the output is established by using the K-PLSR method in the device-independent CIE LAB color space. The mapping model is then applied to correct the captured tongue images. Experimental results show that, using the proposed method, the average color difference of each color patch is only 0.821 after correction. For the subjective results, tongue images under different illumination conditions can obtain consistent correction results, which is beneficial for subsequent standardized tongue image storage and automatic analysis in tongue diagnosis of TCM. Compared with the most commonly used polynomial-based correction method and support vector regression based correction method, whether for the subjective or objective evaluation, the proposed method can obtain a superior color correction performance.

Deviance residuals-based sparse PLS and sparse kernel PLS regression for censored data.

A vast literature from the past decade is devoted to relating gene profiles and subject survival or time to cancer recurrence. Biomarker discovery from high-dimensional data, such as transcriptomic or single nucleotide polymorphism profiles, is a major challenge in the search for more precise diagnoses. The proportional hazard regression model suggested by Cox (1972), to study the relationship between the time to event and a set of covariates in the presence of censoring is the most commonly used model for the analysis of survival data. However, like multivariate regression, it supposes that more observations than variables, complete data, and not strongly correlated variables are available. In practice, when dealing with high-dimensional data, these constraints are crippling. Collinearity gives rise to issues of over-fitting and model misidentification. Variable selection can improve the estimation accuracy by effectively identifying the subset of relevant predictors and enhance the model interpretability with parsimonious representation. To deal with both collinearity and variable selection issues, many methods based on least absolute shrinkage and selection operator penalized Cox proportional hazards have been proposed since the reference paper of Tibshirani. Regularization could also be performed using dimension reduction as is the case with partial least squares (PLS) regression. We propose two original algorithms named sPLSDR and its non-linear kernel counterpart DKsPLSDR, by using sparse PLS regression (sPLS) based on deviance residuals. We compared their predicting performance with state-of-the-art algorithms on both simulated and real reference benchmark datasets. sPLSDR and DKsPLSDR compare favorably with other methods in their computational time, prediction and selectivity, as indicated by results based on benchmark datasets. Moreover, in the framework of PLS regression, they feature other useful tools, including biplots representation, or the ability to deal with missing data. Therefore, we view them as a useful addition to the toolbox of estimation and prediction methods for the widely used Cox's model in the high-dimensional and low-sample size settings. The R-package plsRcox is available on the CRAN and is maintained by Frédéric Bertrand. http://cran.r-project.org/web/packages/plsRcox/index.html. pbastien@rd.loreal.com or fbertran@math.unistra.fr. Supplementary data are available at Bioinformatics online.

KPLS-based image super-resolution using clustering and weighted boosting

Kernel partial least squares (KPLS) algorithm for super-resolution (SR) has carried out a regression model to estimate a high-resolution (HR) feature patch from its corresponding low-resolution (LR) feature patch using a training database. However, KPLS may be time-consuming in the neighbor search and use of principal components. In this paper we propose a clustering and weighted boosting (CWB) framework to improve the efficiency in KPLS regression model construction without reducing SR reconstruction quality. First, the training LR-HR feature patch pairs are divided into a certain number of clusters. For each test LR feature patch, the neighbor search in the selected cluster saves more computational costs than that in the whole training database. Second, a weighted boosting scheme is used to adaptively construct the KPLS regression model with the best number of principal components (BNPC). Experimental results on natural scene images suggest that the proposed CWB method can effectively improve the efficiency of KPLS-based SR method while preserving reconstruction quality, and achieve better performance than the conventional KPLS method.

Eddy Current Inversion Models for Estimating Dimensions of Defects in Multilayered Structures

In eddy current nondestructive evaluation, one of the principal challenges is to determine the dimensions of defects in multilayered structures from the measured signals. It is a typical inverse problem which is generally considered to be nonlinear and ill-posed. In the paper, two effective approaches have been proposed to estimate the defect dimensions. The first one is a partial least squares (PLS) regression method. The second one is a kernel partial least squares (KPLS) regression method. The experimental research is carried out. In experiments, the eddy current signals responding to magnetic field changes are detected by a giant magnetoresistive (GMR) sensor and preprocessed for noise elimination using a wavelet packet analysis (WPA) method. Then, the proposed two approaches are used to construct the inversion models of defect dimension estimation. Finally, the estimation results are analyzed. The performance comparison between the proposed two approaches and the artificial neural network (ANN) method is presented. The comparison results demonstrate the feasibility and validity of the proposed two methods. Between them, the KPLS regression method gives a better prediction performance than the PLS regression method at present.

Performance modeling of centrifugal compressor using kernel partial least squares

The performance modeling of centrifugal (turbo) compressor was performed in this paper by applying kernel partial least squares (KPLS). Firstly, steady-state compressor data sets were collected from a real gas turbine power plant and a simulation study respectively. Then the two data sets were used to train the KPLS regression model for predicting the centrifugal compressor operating parameters such as pressure ratio and efficiency. The prediction performance of KPLS model was compared to that of a three-layer back-propagation (BP) neural network with validation data, and the KPLS model showed slightly better performance than the BP network. Furthermore, results showed that, with high accuracy, KPLS could be used to smoothly predict the compressor map, which was useful in the preliminary design phase of any centrifugal compression system.

Learning-based super resolution using kernel partial least squares

In this paper, we propose a learning-based super resolution approach consisting of two steps. The first step uses the kernel partial least squares (KPLS) method to implement the regression between the low-resolution (LR) and high-resolution (HR) images in the training set. With the built KPLS regression model, a primitive super-resolved image can be obtained. However, this primitive HR image loses some detailed information and does not guarantee the compatibility with the LR one. Therefore, the second step compensates the primitive HR image with a residual HR image, which is the subtraction of the original and primitive HR images. Similarly, the residual LR image is obtained from the down-sampled version of the primitive HR and original LR image. The relation of the residual LR and HR images is again modeled with KPLS. Integration of the primitive and the residual HR image will achieve the final super-resolved image. The experiments with face, vehicle plate, and natural scene images demonstrate the effectiveness of the proposed approach in terms of visual quality and selected image quality metrics.

MALDI-MS and multivariate analysis for the detection and quantification of different milk species

The extensive consumption of milk and dairy products makes these foodstuffs targets for potential adulteration with financial gains for unscrupulous producers. Such practices must be detected as these can impact negatively on product quality, labelling and even health. Matrix-assisted laser desorption/ionisation time-of-flight mass spectrometry (MALDI-ToF-MS) is a potentially useful technique, with proven abilities in protein identification and more recently through the use of internal standards for quantification purposes of specific proteins or peptides. In the current work, we therefore aim to explore the accuracy and attributes of MALDI-ToF-MS with chemometrics for the detection and quantification of milk adulteration. Three binary mixtures containing cows' and goats', cows' and sheep's, and goats' and sheep's milk and a fourth tertiary mixture containing all types of milk were prepared and analysed directly using MALDI-ToF-MS. In these mixtures, the milk concentrations of each milk varied from 0% to 100% in 5% steps. Multivariate statistical methods including partial least squares (PLS) regression and non-linear Kernel PLS regression were employed for multivariate calibration and final interpretation of the results. The results for PLS and KPLS were encouraging with between 2% and 13% root mean squared error of prediction on independent data; KPLS slightly outperformed PLS. We believe that these results show that MALDI-ToF-MS has excellent potential for future use in the dairy industry as a rapid method of detection and enumeration in milk adulteration.

Kernel partial robust M-regression as a flexible robust nonlinear modeling technique

Kernel partial least squares (KPLS) is potentially very efficient for tackling nonlinear systems by mapping an original input space into a high-dimensional feature space. Unlike other nonlinear modeling technique, KPLS does not involve any nonlinear optimization procedure and possesses low computational complexity similar to that of linear partial least squares (PLS). But when there are some outliers in the training data set, KPLS regression will have poor properties due to the sensitiveness of PLS to outliers. Under this circumstance, a more robust regression method such as partial robust M-regression (PRM) can be used instead of PLS in the nonlinear kernel-based algorithm. In this paper, a kernel partial robust M-regression (KPRM) is presented. Nonlinear structure in the original input space is transformed into linear one in the high-dimensional feature space; and through choosing appropriately weighting strategy, the proposed method becomes entirely robust to two types of outliers. The prediction performance of KPRM is compared to those of PLS, PRM, and KPLS using three examples; KPRM yields much lower prediction errors for these three data sets, and the loss in efficiency to be paid for is very small.