Kernel Principal Component Regression Research Articles

Research on speed control of high-speed trains based on hybrid modeling

With the continuous improvement of train speed, the automatic driving of trains instead of driver driving has become the development direction of rail transit in order to realize traffic automation. The application of single modeling methods for speed control in the automatic operation of high-speed trains lacks exploration of the com-bination of train operation data information and physical model, resulting in low system modeling accuracy, which impacts the effectiveness of speed control and the operation of high-speed trains. To further increase the dynamic modeling accuracy of high-speed train operation and the high-speed train's speed control effect, a high-speed train speed control method based on hybrid modeling of mechanism and data drive is put forward. Firstly, a model of the high-speed train's mechanism was created by analyzing the train's dynamics. Secondly, the improved kernel-principal component regression algorithm was used to create a data-driven model using the actual opera-tion data of the CRH3 (China Railway High-speed 3) high-speed train from Huashan North Railway Station to Xi'an North Railway Station of "Zhengxi High-speed Railway," completing the mechanism model compensation and the error correction of the speed of the actual operation process of the high-speed train, and realizing the hybrid modeling of mechanism and data-driven. Finally, the prediction Fuzzy PID control algorithm was devel-oped based on the natural line and train characteristics to complete the train speed control simulation under the hybrid model and the mechanism model, respectively. In addition, analysis and comparison analysis were conduct-ed. The results indicate that, compared to the high-speed train speed control based on the mechanism model, the high-speed train speed control based on hybrid modeling is more accurate, with an average speed control error reduced by 69.42%. This can effectively reduce the speed control error, improve the speed control effect and oper-ation efficiency, and demonstrate the efficacy of the hybrid modeling and algorithm. The research results can provide a new ideal of multi-model fusion modeling for the dynamic modeling of high-speed train operation, further improve control objectives such as safety, comfort, and efficiency of high-speed train operation, and pro-vide a reference for automatic driving and intelligent driving of high-speed trains.

An adaptive subspace data-driven method for nonlinear dynamic systems

Quality-related data-driven fault detection methods have been universally recognized by researchers in industry and academia. However, the detection performance of the common data-driven method will degrade if it cannot keep up with the dynamic shifts of the production process. In this paper, an adaptive kernel principal component regression (AKPCR) method is thus proposed to process the nonlinear feature and the time-varying property in the system. The approximate Hilbert space approach is provided to update the projection matrix based on the innovation information from the new data for the nonlinear high-dimensional feature space. Besides, the recursive post-decomposition technique is created from the updated projection matrix to better monitor the quality state. Compared with the fixed threshold, an adaptive one is designed in this study to detect the fault encountered in the time-varying system, which can decrease false alarms in fault detection. Finally, the Tennessee Eastman case (TEC) and the waste water treatment system (WWTS) are leveraged to illustrate the reliability of the proposed approach.

Dynamic nonlinear process monitoring based on dynamic correlation variable selection and kernel principal component regression

In actual industrial processes, data are usually time series. Each process variable may have strong autocorrelation and cross-correlation with other variables with different delays. In addition, there are usually complex nonlinearities among variables. To further improve the monitoring performance for dynamic nonlinear processes, establishing a nonlinear filtering model for each variable is necessary. Therefore, a novel dynamic nonlinear process monitoring method based on dynamic nonlinear feature selection and kernel principal component regression (KPCR) is proposed in this study. First, dynamic nonlinear related variables are selected for each variable through mutual information by considering variables with different time delays. Second, process variables are divided into response and independent variable sets. Third, corresponding KPCR models are established to describe the dynamic relationships with the selected dynamic related variables as input variables and with response variables as output variables. To monitor the dynamic processes, kernel principal component analysis model is constructed on the basis of the residuals, where the residuals are obtained by comparing measured values of instruments with the predicted values of KPCR models. A support vector data description model is established to monitor the independent variables. Three cases are used to verify the performance of the novel approach. Results show that the proposed method is superior to and more effective than other advanced dynamic process monitoring methods.

A distributed principal component regression method for quality-related fault detection and diagnosis

Modern industrial processes are confronted with a large-scale challenge in recent years. In this paper, a novel distributed kernel principal component regression (DKPCR) approach is proposed to study the plant-wide problem of quality-related process monitoring. Along with reducing the scale of abundant measurements, the proposed approach also focuses on the robust issues that arise from the large outliers. In the local phase, every agent of the DKPCR technique processes the data of the partial sections and sends the selected information to the centralized mainframe. The engineer gathers the data from the local agents and then makes a decision based on the Bayesian inference. Additionally, the corresponding weight diagnosis approach is devised to isolate the fault-relevant variables under the smear effect by virtue of the detection results and information. In the end, the Tennessee Eastman case (TEC) and the three-phase flow system (TPFS) are leveraged to demonstrate the detection and diagnosis performance of the proposed approach.

Application of two statistical approaches (Bayesian Kernel Machine Regression and Principal Component Regression) to assess breast cancer risk in association to exposure to mixtures of brominated flame retardants and per- and polyfluorinated alkylated substances in the E3N cohort

BackgroundBrominated flame retardants (BFR) and per- and polyfluorinated alkylated substances (PFAS) are two groups of substances suspected to act as endocrine disruptors. Such substances could therefore be implicated in the occurrence of breast cancer, nevertheless, previous studies have led to inconstant results. Due to the large correlation between these substances, and the possibly non-linear effects they exert, evaluating their joint impact as mixtures on health remains challenging.This exploratory study aimed to generate hypotheses on the relationship between circulating levels of 7 BFR (6 polybrominated diphenyl ethers and 1 polybrominated biphenyls) and 11 PFAS and the risk of breast cancer in a case–control study nested in the E3N French prospective cohort by performing two methods: Principal Component Regression (PCR) models, and Bayesian Kernel Machine Regression (BKMR) models.Methods194 post-menopausal breast cancer cases and 194 controls were included in the present study. Circulating levels of BFR and PFAS were measured by gas chromatography coupled to high-resolution mass spectrometry and liquid chromatography coupled to tandem mass spectrometry, respectively. The first statistical approach was based on Principal Component Analysis (PCA) followed by logistic regression models that included the identified principal components as main exposure variables. The second approach used BKMR models with hierarchical variable selection, this latter being suitable for highly correlated exposures. Both approaches were also run separately for Estrogen Receptor positive (ER +) and Estrogen Receptor negative (ER-) breast cancer cases.ResultsPCA identified four principal components accounting for 67% of the total variance. Component 3 showed a marginal association with ER + breast cancer risk. No clear association between BFR and PFAS mixtures and breast cancer was identified using BKMR models, and the credible intervals obtained were very wide. Finally, the BKMR models suggested a negative cumulative effect of BFR and PFAS on ER- breast cancer risk, and a positive cumulative effect on ER + breast cancer risk.ConclusionAlthough globally no clear association was identified, both approaches suggested a differential effect of BFR and PFAS mixtures on ER + and ER- breast cancer risk. However, the results for ER- breast cancer should be interpreted carefully due to the small number of ER- cases included in the study. Further studies evaluating mixtures of substances on larger study populations are needed.

Mixed kernel principal component weighted regression based on just-in-time learning for soft sensor modeling

Soft sensors have been extensively applied for predicting difficult-to-measure quality variables. However, industrial processes are often characterized with the nonlinearity and time variance, which makes it difficult to accurately predict the quality variables. In this paper, a just-in-time learning (JITL) based mixed kernel principal component weight regression (MKPCWR) is proposed for soft sensing of nonlinear and time-variant processes. Firstly, taking advantages of classical principal component analysis and kernel PCA, a mixed kernel principal component analysis (MKPCA) is proposed to extract the nonlinear feature. The raw input space and high-dimensional nonlinear space form a mixed space, from which the MKPCA gets a better nonlinear feature representation for the raw input data. Then, a weighted regression method based on JITL is proposed to deal with the time-variant problem of processes. When building the JITL-based regression model, relevant training samples are assigned correspond weights in the loss function to increase the prediction accuracy of model. Lastly, the effectiveness of the proposed method for soft sensor modeling is demonstrated on two industrial processes. Results show that JITL-based MKPCWR is more effective to solve the nonlinearity and time variance than JITL-based principal component regression and kernel principal component regression.

A new natural detector for irradiations with blue LED light source in photodynamic therapy measurements via UV-Vis spectroscopy.

Photodynamic therapy has been recently studied, bringing innovations regarding the reduction of exposure time to light by the patient. This work aimed to investigate the feasibility of using Coutarea hexandra (Jacq.) K. Schum (CHS) as a detector in photodynamic therapy measurements. For this, an irradiator containing a blue LED bulb lamp was utilized. The CHS samples were irradiated with ten doses from 0.60 up to 6.0kJ/cm2, and six concentrations were prepared (1, 2, 3, 4, 5, and 6mg/ml) for the CHS detector samples. After irradiation, the detector samples were evaluated using UV-Vis spectrophotometry. The results showed the behavior of the CHS detector with doses and concentrations, its sensitivity, and its linearity was also evaluated both by Wavelength Method (WM) and the Kernel Principal Component Regression (KPCR) Statistical Method. The values obtained indicate that this method can be applied to the CHS sample detector. In conclusion, the CHS is a promising detector in the field of photodynamic therapy.

Genetic robust kernel sample selection for chemometric data analysis

AbstractIn this work, we propose a new algorithm to improve existing techniques used in the field of spectroscopic data regression analysis. In particular, it combines the power of nonlinear kernel regressors (kernel ridge regression [KRR], kernel principal component regression [KPCR], and Gaussian process regression [GPR]) with an optimization based on nondominated sorting multi‐objective genetic algorithm (NSGAII) to filter the residual outliers in the prediction space and leverage points in the features space. The proposed algorithm, contrary to most existing robust algorithms, simultaneously optimizes many complementary objectives for an automatic adaptation and thus a better outliers detection. It is well known that the elimination of outliers greatly improves the regression model. It is thus the aim of this work to develop a new robust regression algorithm. It has been applied on five different datasets, and the results are compared to both classical nonlinear regression methods and the commonly used robust regression methods robust continuum regression (RCR), partial robust M‐regression (PRM), robust principal component regression (RPCR), robust PLSR (RSIMPLS), and locally weighted regression (LWR). They show that the proposed algorithm outperforms the classical nonlinear regression methods and is a promising competitor to the robust methods outperforming most of them. Even though the results obtained are only from five datasets, this algorithm can be considered an interesting contribution for improving data analysis in the field of chemometrics.

Quality-Related Process Monitoring Based on Improved Kernel Principal Component Regression

To date, quality-related multivariate statistical methods are extensively used in process monitoring and have achieved admirable effects. However, most of them contain recursive processes, which result in higher time complexity and are not suitable for increasingly complex industrial processes. Therefore, this paper embeds singular value decomposition (SVD) into the kernel principal component regression (KPCR) to accomplish Quality-related process monitoring with a lower computational cost. Specifically, the kernel technique is devoted to map the original input into the higher dimensional space to boost the nonlinear ability of the principal component regression (PCR), and then the KPCR is employed to capture the correlation between the input kernel matrix and the output matrix. At the same time, the kernelized input space is decomposed into two orthogonal quality-related and quality-unrelated spaces by SVD, and the statistics of the two spaces are calculated to detect the faults respectively. Compared with other multivariate statistical methods, it has the following advantages: 1) A quality-related kernel principal component analysis (QR-KPCR) algorithm is proposed. 2) Compared with partial least squares method, the recursive process is omitted and the training time is shortened. 3) The model is more concise and the fault detection process is faster. 4) By contrast with other multivariate statistical process monitoring, it has a higher fault detection rate. Experimental results on a widespread example and an industry benchmark verify the effectiveness and reliability of the proposed method.

Quality-Related Root Cause Diagnosis Based on Orthogonal Kernel Principal Component Regression and Transfer Entropy

This article is devoted to solving the problem of quality-related root cause diagnosis for nonlinear process. First, an orthogonal kernel principal component regression model is constructed to achieve orthogonal decomposition of feature space, such that quality-related and quality-unrelated faults can be separately detected in the subspaces of opposite correlations to the output, without any effect on each other. Then, in view of the high complexity of traditional nonlinear fault diagnosis methods, an efficient method of kernel sample equivalence replacement is established to replace the partial differential operations of the kernel gradient algorithm, which can convert nonlinear fault detection indicators into the standard quadratic forms of the original variable sample, thereby making it possible to solve the nonlinear fault diagnosis problem by linear manners. Furthermore, a transfer entropy algorithm is utilized to the new model to analyze the causality between the diagnosed candidate faulty variables to find out the accurate root cause of the fault. Finally, comparative studies between the latest result and the proposed one are carried out in the Tennessee Eastman process to verify the effectiveness and superiority of the new method.

Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques

Kernel analog forecasting (KAF), alternatively known as kernel principal component regression, is a kernel method used for nonparametric statistical forecasting of dynamically generated time series data. This paper synthesizes descriptions of kernel methods and Koopman operator theory in order to provide a single consistent account of KAF. The framework presented here illuminates the property of the KAF method that, under measure-preserving and ergodic dynamics, it consistently approximates the conditional expectation of observables that are acted upon by the Koopman operator of the dynamical system and are conditioned on the observed data at forecast initialization. More precisely, KAF yields optimal predictions, in the sense of minimal root mean square error with respect to the invariant measure, in the asymptotic limit of large data. The presented framework facilitates, moreover, the analysis of generalization error and quantification of uncertainty. Extensions of KAF to the construction of conditional variance and conditional probability functions, as well as to non-symmetric kernels, are also shown. Illustrations of various aspects of KAF are provided with applications to simple examples, namely a periodic flow on the circle and the chaotic Lorenz 63 system.

A Study of Complex Industrial Systems Based on Revised Kernel Principal Component Regression Method

As a data-driven process monitoring method, multivariable statistics techniques have special potentials and advantages to handle the increasingly prominent “Big data challenge” in the complex industrial systems. However, the standard partial least square (PLS) method and the principal component regression (PCR) method cannot maintain stable function in the nonlinear operating environment. In order to capture the precise relation of process variables and product variables, an approach called the revised kernel PCR (RKPCR) method is proposed in this thesis to resolve the problems encountered in the traditional PCR method. In addition, a brief and effective diagnosis logic is designed to decrease the difficulty of fault diagnosis. Finally, the effectiveness of the RKPCR algorithm is illustrated utilizing the Tennessee Eastman case (TEC) platform.

Parallel supervised additive and multiplicative faults detection for nonlinear process

In this paper, a novel supervised nonlinear process monitoring method named comprehensive kernel principal component regression (C-KPCR) is proposed to monitor the quality-related/unrelated additive/multiplicative faults. Firstly, mutual information is used to classify the process variables into quality-related part and quality-unrelated part. Secondly, the original variables matrix and the variables variance matrix are constructed and the data is mapped into high-dimensional feature space to deal with the nonlinear problem. Then the quality-related additive and multiplicative faults can be detected based on the regression model using original variables matrix and variables variance matrix, respectively. Afterwards, the monitoring result of quality-unrelated fault is obtained through combining the quality-unrelated information in the regression model and the quality-unrelated process variables. Finally, the effectiveness of the proposed method is demonstrated by a numerical example and the Tennessee Eastman process.

Temperature prediction for roller kiln based on hybrid first-principle model and data-driven MW-DLWKPCR model

In this paper, a hybrid temperature prediction model is developed for an industrial roller kiln of lithium-ion battery cathode materials, which is based on first-principle model and moving window-double locally weighted kernel principal component regression (DLKWKPCR). First, the mechanism model is built for the roller kiln according to the energy conservation law and heat transfer mechanism. Since the first-principle model is based on some simplified assumptions, it often results in large estimation errors. Thus, a data-driven error compensation model is further constructed with real-time process running data. In order to handle the strongly nonlinear, highly redundant and gradually time-varying characteristics, the error compensation model is built with moving window based DLWKPCR. Finally, a hybrid temperature prediction model is obtained by combining the compensation model and the mechanism model. An industrial roller kiln is utilized to test the effectiveness of the hybrid prediction model, in which the modeling results demonstrate that the developed hybrid prediction model can correctly estimate the roller kiln temperature.

A powerful and efficient multivariate approach for voxel-level connectome-wide association studies

We describe an approach to multivariate analysis, termed structured kernel principal component regression (sKPCR), to identify associations in voxel-level connectomes using resting-state functional magnetic resonance imaging (rsfMRI) data. This powerful and computationally efficient multivariate method can identify voxel-phenotype associations based on the whole-brain connectivity pattern of voxels, and it can detect linear and non-linear signals in both volume-based and surface-based rsfMRI data. For each voxel, sKPCR first extracts low-dimensional signals from the spatially smoothed connectivities by structured kernel principal component analysis, and then tests the voxel-phenotype associations by an adaptive regression model. The method's power is derived from appropriately modelling the spatial structure of the data when performing dimension reduction, and then adaptively choosing an optimal dimension for association testing using the adaptive regression strategy. Simulations based on real connectome data have shown that sKPCR can accurately control the false-positive rate and that it is more powerful than many state-of-the-art approaches, such as the connectivity-wise generalized linear model (GLM) approach, multivariate distance matrix regression (MDMR), adaptive sum of powered score (aSPU) test, and least-square kernel machine (LSKM). Moreover, since sKPCR can reduce the computational cost of non-parametric permutation tests, its computation speed is much faster. To demonstrate the utility of sKPCR for real data analysis, we have also compared sKPCR with the above methods based on the identification of voxel-wise differences between schizophrenic patients and healthy controls in four independent rsfMRI datasets. The results showed that sKPCR had better between-sites reproducibility and a larger proportion of overlap with existing schizophrenia meta-analysis findings. Code for our approach can be downloaded from https://github.com/weikanggong/sKPCR.

Temperature Prediction Model for Roller Kiln by ALD-Based Double Locally Weighted Kernel Principal Component Regression

In the roller kiln of cathode materials for lithium batteries, the timely measurement of temperature is very important for effective process control. However, it is sometimes difficult or costly to measure the temperature timely. To handle this problem, a kind of soft sensor modeling framework with double locally weighted kernel principal component regression based on approximate linearity dependence (ALD) is proposed, which simultaneously carries out sample and variable weighted learning in the high-dimensional and nonlinear space to solve the process time-varying and strong nonlinearity problems. Moreover, a kind of just-in-time learning framework based on ALD is adopted for selectively updating the online local models. By setting a reasonable threshold of ALD, the prediction time can be effectively reduced, and the prediction accuracy can be maintained. The effectiveness of the proposed method is demonstrated on an industrial roller kiln. The results show that the proposed method can meet the requirements of prediction accuracy and time efficiency of the roller kiln of cathode materials for lithium batteries.

Hybrid kernel principal component regression and penalty strategy of multiple adaptive genetic algorithms for estimating optimum parameters in abrasive waterjet machining

The quality of machined surface can be improved by choosing the right values of the associated parameters through an optimization process. Recently, researchers have used nonlinear programming models, soft-computing approaches, and hybrid of them to estimate the surface roughness of abrasive waterjet (AWJ) machining. Some researchers have performed a second order polynomial regression (SOPR) as a basis in developing a nonlinear programming in AWJ machining. Nevertheless, the SOPR was developed without considering the existence of multicollinearity which can lead to inappropriate prediction. Besides, it also needs a specific nonlinear regression model in advance. It will be difficult to specify an appropriate nonlinear model of SOPR since it does not usually involve all combination variables in its model. Instead, kernel principal component regression (KPCR) will be employed to overcome the weaknesses of the SOPR. After development and model selection in the KPCR models, the best KPCR is used as an objective function in a nonlinear programming model of AWJ with a certain set of constraints. Single genetic algorithm (GA) and its variants can be conducted to solve the nonlinear programming problem. However, they can yield different decision values due to randomness of their initial population which implies that the best decision values may not converge in a certain optimum solution. Under this circumstance, multiple adaptive probabilities genetic algorithm (MAPGA) combined with a penalty method to solve the optimization problem is proposed. Hybrid of KPCR and MAPGA gives more stable solution compared to hybrid of KPCR with single genetic algorithm and original adaptive probabilities genetic algorithm. Our proposed technique also provides an optimum solution in reasonable time.

Quality Relevant and Independent Two Block Monitoring Based on Mutual Information and KPCA

Traditional process monitoring methods take all the measured variables into account, whereas it will be inappropriate for indicating quality-relevant faults. Some measured variables are independent from the quality variables and these redundancy variables will no doubt degrade the prediction performance of quality variables. This paper proposes a novel quality relevant and independent two block monitoring scheme based on mutual information (MI) and kernel principal component analysis (KPCA). First, all the process variables are divided into two subblocks according to their MI value with quality variables. Then, KPCA monitors the quality-relevant subblock and quality-independent subblock, respectively. When a fault is detected, kernel principal component regression is further utilized to obtain the predicted state of quality variables. Either of the information, whether the current fault disturbs quality-relevant variables or process quality, is necessary and important for engineers. The benefits of MI-KPCA are illustrated through a numerical simulation and the Tennessee Eastman process, and the results reveal the superiority of the proposal compared with some other monitoring methods.

Kernel ridge vs. principal component regression: Minimax bounds and the qualification of regularization operators

Regularization is an essential element of virtually all kernel methods for nonparametric regression problems. A critical factor in the effectiveness of a given kernel method is the type of regularization that is employed. This article compares and contrasts members from a general class of regularization techniques, which notably includes ridge regression and principal component regression. We derive an explicit finite-sample risk bound for regularization-based estimators that simultaneously accounts for (i) the structure of the ambient function space, (ii) the regularity of the true regression function, and (iii) the adaptability (or qualification) of the regularization. A simple consequence of this upper bound is that the risk of the regularization-based estimators matches the minimax rate in a variety of settings. The general bound also illustrates how some regularization techniques are more adaptable than others to favorable regularity properties that the true regression function may possess. This, in particular, demonstrates a striking difference between kernel ridge regression and kernel principal component regression. Our theoretical results are supported by numerical experiments.

Short Term Electric Load Prediction by Incorporation of Kernel into Features Extraction Regression Technique

Accurate load prediction plays an important role in smart power management system, either for planning, facing the increasing of load demand, maintenance issues, or power distribution system. In order to achieve a reasonable prediction, authors have applied and compared two features extraction technique presented by kernel partial least square regression and kernel principal component regression, and both of them are carried out by polynomial and Gaussian kernels to map the original features’ to high dimension features’ space, and then draw new predictor variables known as scores and loadings, while kernel principal component regression draws the predictor features to construct new predictor variables without any consideration to response vector. In contrast, kernel partial least square regression does take the response vector into consideration. Models are simulated by three different cities’ electric load data, which used historical load data in addition to weekends and holidays as common predictor features for all models. On the other hand temperature has been used for only one data as a comparative study to measure its effect. Models’ results evaluated by three statistic measurements, show that Gaussian Kernel Partial Least Square Regression offers the more powerful features and significantly can improve the load prediction performance than other presented models.