Kernel Partial Least Squares Research Articles

Quantitative analysis of organic acids in pomelo fruit using FT-NIR spectroscopy coupled with network kernel PLS regression

Pomelo is a high-class characteristic food in China. The fruit quality of pomelo is much important for people’s health and sense of taste. After being picked, the fruit quality deteriorates because of the spontaneous physiological reactions due to the metabolism and changes of organic acids (OA). The detection of OA concentration is essential for evaluating the pomelo fruit quality. In this paper, the OA concentrations in pomelo fruit samples were quantitatively determined by using Fourier transform near-infrared (FT-NIR) spectroscopy coupled with the kernel partial least square (kernel PLS) method. The network architectural kernel was proposed for extracting the spectral feature data in the machine learning manner. The number of network hidden nodes were tuned in combination of the optimally selection of the PLS latent variables. An error-feedback iteration mechanism was used for self-adaptively training the network linking weights. Model calibration results show that the most suitable network structure was identified with 130 hidden nodes and 20 output nodes. The optimal kernel improved the PLS model with 8 feature latent variables. The model training results were acquired with the root mean square error as 0.834 g/kg, and the correlation coefficient as 0.936. In comparison with common kernel functions, the network kernel performed better in the model training part and in the testing part. The model prediction results indicated that the proposed automatically training of the network architectural kernel is feasible to improve the PLS modeling effects for the FT-NIR rapid analysis of OA in pomelo fruit.

A Dynamic Model Incorporated with Delay Estimation and Variable Selection for Control of SCR De-NOx Process

In order to control NH3 injection for the selective catalytic reduction of nitrogen oxide (NOx) denitration (SCR de-NOx) process, a model that can accurately and quickly predict outlet NOx emissions is required. This paper presents a dynamic kernel partial least squares (KPLS) model incorporated with delay estimation and variable selection for outlet NOx emission and investigated control strategy for NH3 injection. First, k-nearest neighbor mutual information (KNN_MI) was used for delay estimation, and the effect of historical data lengths on KNN_MI was taken into account. Bidirectional search based on the change rate of KNN_MI (KNN_MI_CR) was used for variable selection. Delay–time difference update algorithm and feedback correction strategy were proposed. Second, the NH3 injection compensator (NIC) and the outlet NOx emission model constituted a correction controller. Then, its output and the output of the existing controller are added up to suitable NH3 injection. Finally, the KNN_MI_CR method was compared with different algorithms by benchmark dataset. The field data results showed that the KNN_MI_CR method could improve model accuracy for reconstructed samples. The final model can predict outlet NOx emissions in different operating states accurately. The control result not only meets the NOx emissions standard (50 mg/m3) but also keeps high de-NOx efficiency (80%). NH3 injection and NH3 escape are reduced by 11% and 39%.

Kernel partial least squares model for pectin content in peach using near‐infrared spectroscopy

SummaryPectin is a class of complex galacturonic acid‐rich polysaccharides that are related to the texture of fruit and vegetables. The objective of this study was to develop, using near‐infrared (NIR) spectroscopy, the best model for the determination of pectin content in peach fruit. A total of 100 samples divided into lossy and lossless samples were used to collect NIR raw spectra in the range of 1000–2500 nm. NIR absorption spectra were then obtained after pre‐processing. Finally, four methods were used to establish lossy and lossless spectral models. The 10‐fold cross‐validation coefficient of determination R2 of the lossy model was between 0.364 and 0.628, whereas that of the lossless model was between 0.187 and 0.288, indicating that the lossy model was better than the lossless model. Among all samples, the kernel partial least squares (KPLS) lossy model was better, with coefficient of determination R2 = 0.628, root mean square error (RMSE) = 0.069 and mean absolute error (MAE) = 0.061. This is the first study to evaluate the prediction of peach pectin content using NIR spectroscopy, and the model can be used for rough screening.

Application of non-linear partial least squares analysis on prediction of biomass of maize plants using hyperspectral images

With the application of hyperspectral imaging systems on high-throughput plant phenotyping, multivariable spectral data modelling methods have been proposed to predict plant physiological features such as biomass. However, most presented modelling methods are linear models such as the regression model between projected leaf area and plant biomass, which cannot accurately reflect some non-linear relationships between hyperspectral imaging information and the phenomes to be predicted. Therefore, it is important to develop a prediction model for plant physiological features that fully utilises the information in plant hyperspectral images. In this paper, a non-linear modelling method known as kernel partial least squares (KPLS) was investigated to improve the prediction performance of maize (US corn) biomass. The main idea of KPLS is mapping the original input features into higher dimensional feature spaces before obtaining the principle components for a standard PLS model. In the new mapping spaces, the original, nonlinear inputs may exhibit linear patterns with different kernel functions. This method was tested in a greenhouse assay containing 102 maize plants, which were subjected to a combination of three water treatments and two nitrogen treatments at three different growth stages. KPLS methods were successfully applied on the collected hyperspectral images. The results showed that compared with conventional linear models, statistics including R 2 of KPLS with radial basis function (RBF) kernel had improved accuracy (R 2 = 0.924). We expect that besides plant biomass, this new non-linear KPLS model could improve the quality of estimation of other plant features as well. • Proposed KPLS model with RBF kernel more accurately predicts maize fresh weight. • Kernel functions map original features into higher dimensional feature spaces. • Hyperspectral image provides image-derived features for biomass prediction. • Performances of projected leaf area, linear PLS and KPLS models are investigated. • We conduct a greenhouse phenotyping assay to explore maize biomass prediction models.

Approximate kernel partial least squares

As an extension of partial least squares (PLS), kernel partial least squares (KPLS) is an very important methods to find nonlinear patterns from data. However, the application of KPLS to large-scale problems remains a big challenge, due to storage and computation issues in the number of examples. To address this limitation, we utilize randomness to design scalable new variants of the kernel matrix to solve KPLS. Specifically, we consider the spectral properties of low-rank kernel matrices constructed as sums of random feature dot-products and present a new method called randomized kernel partial least squares (RKPLS) to approximate KPLS. RKPLS can alleviate the computation requirements of approximate KPLS with linear space and computation in the sample size. Theoretical analysis and experimental results show that the solution of our algorithm converges to exact kernel matrix in expectation.

Fault identification for quality monitoring of molten iron in blast furnace ironmaking based on KPLS with improved contribution rate

Blast furnace (BF) ironmaking is one of the most important production links in modern iron–steel making. The operation conditions of BF and the molten iron quality (MIQ) should be monitored and analyzed in real-time to realize high quality with low energy consumption. Aiming at the problems of strong nonlinearity and few fault samples in BF processes, a novel fault identification method for MIQ monitoring based on kernel partial least squares (KPLS) with improved contribution rate is proposed in this paper. First of all, a KPLS model is established with the actual historical data in order to detect the quality-related faults accurately. The T2 and SPE statistics are used to monitor the operation conditions of process from different aspects. Second, in view of the unclear physical meaning and complex computation of the existing fault identification methods based on KPLS with contribution rate, a scale factor vector is introduced into the new samples to calculate the T2 and SPE statistics, so as to construct the monitoring indicators functions. By performing Taylor approximation on these constructed functions near the scale factor with the value of 1, two new statistics are obtained from the absolute value of the first-order partial derivative representing the contribution rate of each variable. Finally, in order to further improve the effect of fault identification, the relative contribution rate of each variable is used to identify the final faulty variables. The tests of MIQ monitoring in BF ironmaking process using actual industrial data verify the validity and practicability of the proposed method.

Key-Performance-Indicator-Related Process Monitoring Based on Improved Kernel Partial Least Squares

Although the partial least squares approach is an effective fault detection method, some issues of nonlinear process monitoring related to key performance indicators (KPIs) still exist. To address the nonlinear characteristics in the industrial processes, kernel partial least squares (KPLS) method was proposed in the literature. However, the KPLS method also faces some difficulties in fault detection. None of the existing KPLS methods can accurately decompose measurements into KPI-related and KPI-unrelated parts, and these methods usually ignore the fact that the residual subspace still contains some KPI-related information. In this article, a new improved KPLS method, which considers the KPI-related information in the residual subspace, has been proposed for KPI-related process monitoring. First, the proposed method performs general singular value decomposition (GSVD) on the calculable loadings based on the kernel matrix. Next, the kernel matrix can be suitably divided into KPI-related and KPI-unrelated subspaces. Besides, we present the design of two statistics for process monitoring as well as a detailed algorithm performance analysis for kernel methods. Finally, a numerical case and Tennessee Eastman benchmark process demonstrate the efficacy and merits of the improved KPLS-based method.

Multiple genome pattern analysis and signature gene identification for the Caucasian lung adenocarcinoma patients with different tobacco exposure patterns.

BackgroundWhen considering therapies for lung adenocarcinoma (LUAD) patients, the carcinogenic mechanisms of smokers are believed to differ from those who have never smoked. The rising trend in the proportion of nonsmokers in LUAD urgently requires the understanding of such differences at a molecular level for the development of precision medicine.MethodsThree independent LUAD tumor sample sets—TCGA, SPORE and EDRN—were used. Genome patterns of expression (GE), copy number variation (CNV) and methylation (ME) were reviewed to discover the differences between them for both smokers and nonsmokers. Tobacco-related signature genes distinguishing these two groups of LUAD were identified using the GE, ME and CNV values of the whole genome. To do this, a novel iterative multi-step selection method based on the partial least squares (PLS) algorithm was proposed to overcome the high variable dimension and high noise inherent in the data. This method can thoroughly evaluate the importance of genes according to their statistical differences, biological functions and contributions to the tobacco exposure classification model. The kernel partial least squares (KPLS) method was used to further optimize the accuracies of the classification models.ResultsForty-three, forty-eight and seventy-five genes were identified as GE, ME and CNV signatures, respectively, to distinguish smokers from nonsmokers. Using only the gene expression values of these 43 GE signature genes, ME values of the 48 ME signature genes or copy numbers of the 75 CNV signature genes, the accuracies of TCGA training and SPORE/EDRN independent validation datasets all exceed 76%. More importantly, the focal amplicon in Telomerase Reverse Transcriptase in nonsmokers, the broad deletion in ChrY in male nonsmokers and the greater amplification of MDM2 in female nonsmokers may explain why nonsmokers of both genders tend to suffer LUAD. These pattern analysis results may have clear biological interpretation in the molecular mechanism of tumorigenesis. Meanwhile, the identified signature genes may serve as potential drug targets for the precision medicine of LUAD.

In silico identification of hydantoin derivatives: a novel natural prolyl hydroxylase inhibitor

Alzheimer’s disease (AD) is the most common dementia in late life memory related issues. It is estimated that worldwide 46.8 million people suffer from dementia. The hypoxia inducible factor (HIF) upregulation could be a potential neuroprotective strategy by modulating the multiple survival pathways in AD. Hence, the development of small molecules that induce HIF-α activation via inhibition of prolyl hydroxylase enzyme (PHD) has been suggested as a potentially useful therapeutic strategy for the treatment of AD. Thus, to unveil a novel human PHD inhibitor, 2D-QSAR (Quantitative Structure Activity Relationship) modeling was performed with the 213 PHD inhibitors reported in the USA patent database. Best model with the r2 score of 0.8273 and q2 value of 0.8284 with dendritic fingerprint was developed and visualized by kernel partial least squares (KPLS) methods, which were used for visualization and screening of natural/derivative compounds database for novel and effective drug or scaffold. The docking was performed as a secondary strategy for screening the top 5000 compounds retrieved by 2D-QSAR based screening. Further, the docked complexes were screened by molecular dynamics (MD) simulation and molecular mechanics/generalized Born surface area (MM-GBSA) based binding free energy calculations to determine the binding energy of the inhibitors and to identify crucial interacting energy contributors. Three leads have demonstrated good binding free energy and the better binding affinity for PHD compared to other selected ligands. Thus, the results obtained from QSAR, docking and MD simulations depicted that hydantoin scaffold could be effectively used as a potent inhibitor toward human PHD in AD therapeutics.Communicated by Ramaswamy H. Sarma

Reliable Fault Detection and Diagnosis of Large-Scale Nonlinear Uncertain Systems Using Interval Reduced Kernel PLS

Kernel partial least squares (KPLS) models are widely used as nonlinear data-driven methods for faults detection (FD) in industrial processes. However, KPLS models lead to irrelevant performance over long operation periods due to process parameters changes, errors and uncertainties associated with measurements. Therefore, in this paper, two different interval reduced KPLS (IRKPLS) models are developed for monitoring large scale nonlinear uncertain systems. The proposed IRKPLS models present an interval versions of the classical KPLS model. The two proposed IRKPLS models are based on the Euclidean distance between interval-valued observations as a dissimilarity metric to keep only the more relevant and informative samples. The first proposed IRKPLS technique uses the centers and ranges of intervals to estimate the interval model, while the second one is based on the upper and lower bounds of intervals for model identification. These obtained models are used to evaluate the monitored interval residuals. The aforementioned interval residuals are fed to the generalized likelihood ratio test (GLRT) chart to detect the faults. In addition to considering the uncertainties in the input-output systems, the new IRKPLS-based GLRT techniques aim to decrease the execution time when ensuring the fault detection performance. The developed IRKPLS-based GLRT approaches are evaluated across various faults of the well-known Tennessee Eastman (TE) process. The performance of the proposed IRKPLS-based GLRT methods is evaluated in terms of missed detection rate, false alarms rate, and execution time. The obtained results demonstrate the efficiency of the proposed approaches, compared with the classical interval KPLS.

Soft Sensor Modeling for Unobserved Multimode Nonlinear Processes Based on Modified Kernel Partial Least Squares With Latent Factor Clustering

To cope with the soft sensor modeling of unobserved multimode nonlinear processes, this paper proposes a modified kernel partial least squares (KPLS) by integrating latent factor clustering (LFC), called LFC-KPLS. In the proposed method, the process data are first divided into several batches orderly, and then projected onto the latent space by using the nonlinear functional expansion technology. In the latent space, partial least squares method is applied to compute the regression coefficients between the input variables and output variable of each batch. These regression coefficients, called the latent factors, can describe the functional relationships in the unobserved multimode data. Therefore, the latent factors are used for mode clustering so that the process data with similar functional relations can be clustered in one mode together. For each mode, the nonlinear soft sensor is established based on KPLS. To assign the mode of the online query sample, a mode identification strategy based on Bayesian inference is designed for the soft sensor online prediction. Finally, two cases studies are adopted to validate the proposed method.

Reflectance spectra reconstruction from trichromatic camera based on kernel partial least square method.

A novel spectral reflectance reconstruction method based on kernel partial least square (KPLS) regression is proposed. The proposed method integrates the partial least square algorithm and kernel function to estimate the reflectance spectra from 9-channel multispectral imaging system using commercial trichromatic camera. The performance of the proposed method is demonstrated in comparison with the existing methods using simulated and real camera responses from Munsell Matte color and IT8.7/3 dataset. The experimental results show that the proposed method is superior or at least equivalent to its counterparts and satisfactory enough for color management purpose.

Orders of magnitude speed increase in partial least squares feature selection with new simple indexing technique for very tall data sets

Feature selection is a challenging combinatorial optimization problem that tends to require a large number of candidate feature subsets to be evaluated before a satisfying solution is obtained. Because of the computational cost associated with estimating the regression coefficients for each subset, feature selection can be an immensely time‐consuming process and is often left inadequately explored. Here, we propose a simple modification to the conventional sequence of calculations involved when fitting a number of feature subsets to the same response data with partial least squares (PLS) model fitting. The modification consists in establishing the covariance matrix for the full set of features by an initial calculation and then deriving the covariance of all subsequent feature subsets solely by indexing into the original covariance matrix. By choosing this approach, which is primarily suitable for tall design matrices with significantly more rows than columns, we avoid redundant (identical) recalculations in the evaluation of different feature subsets. By benchmarking the time required to solve regression problems of various sizes, we demonstrate that the introduced technique outperforms traditional approaches by several orders of magnitude when used in conjunction with PLS modeling. In the supplementary material, we provide code for implementing the concept with kernel PLS regression.

Sustainable NO[formula omitted] emission reduction at a coal-fired power station through the use of online neural network modeling and particle swarm optimization

Meeting increasingly stringent environmental targets while conforming to new operational demands of load-ramping and low-load operation presents a challenge for coal-fired thermal power stations. Artificial neural networks (ANNs) have been shown to predict efficiency and emissions of power stations, however, existing literature is comprised of offline studies and short-duration tests. In practice, ANN based combustion optimization systems (COS) have experienced mixed success due to the degradation of model performance over time and a lack of customizability. The current work presents a COS directed at reducing NOx at a 490 MW tangentially coal-fired power station over two years of live, closed-loop operation. The system employs a novel software platform called the Griffin AI Toolkit™, which includes a graphical user interface for ANN design and meta-optimization. A unique ANN utilizing swappable synapse weights for improved modeling of operational sub-regimes is validated against support vector regression (SVR), random forests (RF), and kernel partial least squares (KPLS) models, then deployed within the COS. The system leverages particle swarm optimization (PSO) to interrogate the models and perform real-time optimization. This work demonstrates: 1. Substantial NOx emission reductions by the station (22.5%), 2. A hybrid optimization approach applying genetic algorithm (GA) to select neural network structure and PSO to perform combustion optimization, 3. The realization of other performance benefits in an 87% reduction of high-temperature exceedances due to encapsulating operational best-practices within the platform, and 4. The sustainability of the COS in achieving a high degree of acceptance, with a final service factor of 86%.

Dual-layer optimized selective information fusion using multi-source multi-component mechanical signals for mill load parameters forecasting

Abstract Ball mill is a heavy mechanical device necessary for grinding. Mill load parameters (MLP) relate to production economic indices and process safety. Mechanical signals of the ball mill are used to estimate MLP by domain experts. However, they can only estimate familiar mills effectively in certain time because of human limitation. A new dual-layer optimized selective information fusion is proposed based on the analysis of the characteristics of mill mechanical signals and cognitive behavior of the domain expert for MLP forecasting (MLPF). An ensemble construction strategy based on multi-component mechanical signals adaptive decomposition is employed to build candidate sub-models by using kernel partial least squares (KPLS). The dual-layer optimization strategy is proposed to build selective ensemble (SEN) KPLS (SENKPLS) with optimized ensemble sub-models and their coefficients, thus realizing the trade-off between prediction accuracies and diversity implicitly. The MLPF models based on SENKPLS are constructed by selective fusion multi-source multi-scale frequency spectral information in terms of the auditory perception process of the simulation domain experts. Results show that the proposed strategy can obtain better forecasting results than other state-of-the-art methods.

Dynamic modelling and predictive control for the sequential collaborative reactors of cobalt removal process under time‐varying conditions

The accuracy of the process model directly affects the performance of the model‐based controller. In zinc hydrometallurgy, the overall dynamics of the cobalt removal process can hardly be described by a fixed model since there are a series of interconnected reactors working together under time‐varying inlet and reaction conditions. In this study, an interacting continuously stirred tank reactors (ICSTR) model is developed to describe the cooperative relationship of these cascaded reactors. Considering the time‐varying inlet and reaction conditions, the reaction surface conversion coefficient is defined and incorporated into the ICSTR model, and the kernel partial least squares (KPLS) is employed to update the dynamic model online. The effectiveness of the time‐varying ICSTR model is validated using industrial data. Based on the proposed time‐varying ICSTR model, a predictive controller is designed to realize the optimal operation of the cobalt removal process. Simulation results indicate that compared with conventional predictive control and manual manipulation, the time‐varying ICSTR model‐based predictive control method can not only maintain the outlet cobalt ion concentration but also reduce the zinc dust dosage.

A feature-based soft sensor for spectroscopic data analysis

In the last few decades, spectroscopic techniques such as near-infrared (NIR) and UV/vis spectroscopies have gained wide applications. As a result, various soft sensors have been developed to predict sample properties from its spectroscopic readings. Because the readings at different wavelengths are highly correlated, it has been shown that variable selection could significantly improve a soft sensor’s prediction performance and reduce the model complexity. Currently, almost all variable selection methods focus on how to select the variables (i.e., wavelengths or wavelength segments) that are strongly correlated with the dependent variable to improve the prediction performance. Although many successful applications have been reported, such variable selection methods do have their limitations, such as high sensitivity to the choice of training data, and deteriorated performance when testing on new samples. One possible reason is the removal of useful wavelengths or segments of wavelengths during the calibration process, which could be “tilted” to overfit or capture the noise or unknown disturbances contained in the calibration data. As a result, the model prediction performance may deteriorate significantly when the model is extrapolated or applied to new samples. To address this limitation, we propose a feature-based soft sensor approach utilizing statistics pattern analysis (SPA). Instead of selecting certain wavelengths or wavelength segments, the SPA-based method considers the whole spectrum which is divided into segments, and extracts different features over each spectrum segment to build the soft sensor. In other words, the SPA model contains the complete information from the full spectrum without any selection or removal, which we believe is the main reason for the high robustness of the SPA-based method. In addition, we propose a Monte Carlo validation and testing (MCVT) procedure and three MCVT-based performance indices for consistent and fair comparison of different soft sensor methods across different datasets. The MCVT procedure and indices are generally applicable for model comparison in other applications. Four case studies are presented to demonstrate the performance of the feature-based soft sensor and to compare it with a full partial least squares (PLS), a least absolute shrinkage and selection operator (Lasso), and a synergy interval PLS (SiPLS) based models following the proposed MCVT procedure. In addition, we examine the potential of kernel PLS (KPLS) based soft sensor approaches, examine their performances, and discuss their pros and cons.

A new monitoring scheme of an air quality network based on the kernel method

Air pollution is classified as one of the most dangerous type on the human health, the environment, and the ecosystem. However, air pollution results in climate change and affects people’s health. For a number of years, monitoring the air quality has become a very urgent and necessary topic. Moreover, safety and health have been attracting attention as one of the important topics to evaluate, firstly, the degree of air pollution and predict pollutant concentrations accurately. Then, it is crucial to establish a more scientific air quality monitoring to ensure the quality of life. In this paper, new reduced air quality monitoring is suggested to enhance the Fault Detection (FD) of an air quality monitoring network. Furthermore, a sensor FD procedure based on Reduced Kernel Partial Least Squares (RKPLS) is proposed to monitor an air quality monitoring network. The main contribution of the suggested procedure is to enhance the FD of an air quality monitoring network in terms of computation time and false alarm rate, using just the important latent components, compared to standard Kernel Partial Least Squares (KPLS).

A novel modelling method for plate heat exchanger to predict the outlet cooling water temperature

AbstractIn this paper, a modelling method for a plate heat exchanger (PHE) is proposed to predict the outlet cooling water temperature. First, the mechanistic model of PHE is developed based on the theory of heat and mass transfer. The unknown parameters in the model are identified by solving an optimization model with the sequential quadratic programming (SQP) method. Then, to improve the output precision, a data‐driven model based on the kernel partial least squares (KPLS) algorithm is developed and added to the mechanistic model in parallel to compensate the deviations between the mechanistic model outputs and the real values of outlet cooling water temperature. With the PHE running, its performance deteriorates due to the aging of the device, fouling on the plates, and other unknown factors. Therefore, a performance assessment for the model is performed, and the model updating is carried out according to the assessment results. Finally, the feasibility and efficiency of the proposed modelling method are validated by application to the PHE in a circulating cooling water system. The established model lays an important foundation for the energy‐saving optimization of the circulating cooling water system.

Multiscale Kernel PLS-Based Exponentially Weighted-GLRT and Its Application to Fault Detection

In this paper, we develop an improved statistical technique in order to enhance monitoring of biological processes. To improve the performance of monitoring, a detection statistic that exploits the advantages of the generalized likelihood ratio test (GLRT) statistic with those of the exponentially weighted moving average filter, kernel partial least square (KPLS) model, and multiscale representation is developed. The advantages of multiscale (MS) KPLS-based exponentially weighted GLRT (EW-GLRT) are threefold: First, the developed EW-GLRT statistic takes into account the information given by the current and previous data by giving high importance to the more recent data; second, the dynamical multiscale representation is proposed to extract accurate deterministic features and decorrelate autocorrelated measurements; third, the MSKPLS model evaluates the KPLS of the wavelet coefficients at each scale. Due to its multiscale nature, MSKPLS is appropriate for modeling of data that contain contributions from events whose behavior changes over time and frequency. The detection performance is studied using Cad System in E. coli model and genomic copy number data for detecting small and moderate shifts. MSKPLS-based EW-GLRT is used to enhance fault detection of the Cad System in E. coli model through monitoring some of the key variables involved in this model, such as enzymes, lysine, and cadaverine. The proposed technique is also applied to detect diseases using genomic copy number data through better detection of aberrations in the genetic information of patients, which can help medical doctors make more accurate diagnosis of diseases.