Kernel Ridge Regression Research Articles

Augmenting authenticity for non-invasive in vivo detection of random blood glucose with photoacoustic spectroscopy using Kernel-based ridge regression

Photoacoustic Spectroscopy (PAS) is a potential method for the noninvasive detection of blood glucose. However random blood glucose testing can help to diagnose diabetes at an early stage and is crucial for managing and preventing complications with diabetes. In order to improve the diagnosis, control, and treatment of Diabetes Mellitus, an appropriate approach of noninvasive random blood glucose is required for glucose monitoring. A polynomial kernel-based ridge regression is proposed in this paper to detect random blood glucose accurately using PAS. Additionally, we explored the impact of the biological parameter BMI on the regulation of blood glucose, as it serves as the primary source of energy for the body’s cells. The kernel function plays a pivotal role in kernel ridge regression as it enables the algorithm to capture intricate non-linear associations between input and output variables. Using a Pulsed Laser source with a wavelength of 905 nm, a noninvasive portable device has been developed to collect the Photoacoustic (PA) signal from a finger. A collection of 105 individual random blood glucose samples was obtained and their accuracy was assessed using three metrics: Root Mean Square Error (RMSE), Mean Absolute Difference (MAD), and Mean Absolute Relative Difference (MARD). The respective values for these metrics were found to be 10.94 (mg/dl), 10.15 (mg/dl), and 8.86%. The performance of the readings was evaluated through Clarke Error Grid Analysis and Bland Altman Plot, demonstrating that the obtained readings outperformed the previously reported state-of-the-art approaches. To conclude the proposed IoT-based PAS random blood glucose monitoring system using kernel-based ridge regression is reported for the first time with more accuracy.

Hyperdimensional computing with holographic and adaptive encoder.

Brain-inspired computing has become an emerging field, where a growing number of works focus on developing algorithms that bring machine learning closer to human brains at the functional level. As one of the promising directions, Hyperdimensional Computing (HDC) is centered around the idea of having holographic and high-dimensional representation as the neural activities in our brains. Such representation is the fundamental enabler for the efficiency and robustness of HDC. However, existing HDC-based algorithms suffer from limitations within the encoder. To some extent, they all rely on manually selected encoders, meaning that the resulting representation is never adapted to the tasks at hand. In this paper, we propose FLASH, a novel hyperdimensional learning method that incorporates an adaptive and learnable encoder design, aiming at better overall learning performance while maintaining good properties of HDC representation. Current HDC encoders leverage Random Fourier Features (RFF) for kernel correspondence and enable locality-preserving encoding. We propose to learn the encoder matrix distribution via gradient descent and effectively adapt the kernel for a more suitable HDC encoding. Our experiments on various regression datasets show that tuning the HDC encoder can significantly boost the accuracy, surpassing the current HDC-based algorithm and providing faster inference than other baselines, including RFF-based kernel ridge regression. The results indicate the importance of an adaptive encoder and customized high-dimensional representation in HDC.

Ground-Motion Model for Small-to-Moderate Potentially Induced Earthquakes Using an Ensemble Machine Learning Approach for CENA

ABSTRACT Ground-motion models (GMMs) are vital in assessing probabilistic seismic hazards and uncertainties. This study develops new GMMs benefiting from nonparametric machine learning algorithms, including artificial neural networks, kernel ridge, random forest, and gradient boosting regression techniques for small-to-moderate potentially induced earthquakes in central and eastern North America (CENA). As part of this study, we evaluate the performance of different machine learning models in estimating peak ground acceleration (PGA) and 17 spectral accelerations based on the moment magnitude (Mw), hypocentral distance (Rhypo), and the timed-average shear-wave velocity of the upper 30 m of soil (VS30). To train the algorithms, we have utilized a database of nearly 31,000 ground motions with small and moderate moment magnitudes ranging from 3.0 to 5.8, recorded within a hypocentral distance of less than 200 km in CENA. Typically, for GMM development, analysts employ linear regression-based models with predefined functional forms. The requirement for predefined functional forms can restrict the use of complicated and nonlinear equations to improve performance. Although the conventional regression model is more interpretable, machine learning can achieve a better result given sufficient training data. The results of error metrics reveal that gradient-boosting regression provides a better performance. Furthermore, a machine learning ensemble method is used to combine the regression results of four machine learning algorithms. The ensemble method improves the GMM performance and provides smoother results.

PM2.5 concentration forecasting: Development of integrated multivariate variational mode decomposition with kernel Ridge regression and weighted mean of vectors optimization

The accurate prediction of the PM2.5 air quality parameter within industrial and urban settings is the most pressing issue examined by the researchers because it has high health implications. However, accurately forecasting PM2.5 levels is crucial. The traditional machine learning (ML) models are incapable, as the indices fluctuate daily. To successfully manage this problem, a new ML framework is proposed that incorporates various techniques, such as LGBM feature selection (light gradient-boosting machine), MVMD (multivariate variational mode decomposition), KRidge (kernel Ridge regression), and INFO (weighted mean of vectors). The proposed framework is used to estimate PM2.5 pollution at specific stations in China for a one-time prediction. The LGBM feature selection technique is the first step of the pre-processing, which performs to select the most important variables. Next, the MVMD splits the initial signal into intrinsic mode functions (IMFs), accommodating the signal's non-stationary multivariate. Following that, the KRidge approach is applied to each sub-component using the best input feature, and the resulting predictions are summed up to get the PM2.5 levels. To assess the validity of the proposed MVMD-KRidge-INFO model, the categories of gaussian-process-regression (GPR), locally-weighted-linear-regression (LWLR), and multivariate adaptive regression splines (MARS) are analyzed in individual and hybrid moldes. As observed from the research results, MVMD-KRidge-INFO performs optimally at forecasting PM2.5 levels at Huairou and Shunyi, as evidenced by R, RMSE, MAPE, IA, MdAE, and U95% metrics.

Application of machine learning in prediction of Pb2+ adsorption of biochar prepared by tube furnace and fluidized bed.

Data mining by machine learning (ML) has recently come into application in heavy metals purification from wastewater, especially in exploring lead removal by biochar that prepared using tube furnace (TF-C) and fluidized bed (FB-C) pyrolysis methods. In this study, six ML models including Random Forest Regression (RFR), Gradient Boosting Regression (GBR), Support Vector Regression (SVR), Kernel Ridge Regression (KRR), Extreme Gradient Boosting (XGB), and Light Gradient Boosting Machine (LGBM) were employed to predict lead adsorption based on a dataset of 1012 adsorption experiments, comprising 422 TF-C groups from our experiments and 590 FB-C groups from literatures. The XGB model showed superior accuracy and predictive performance for adsorption, achieving R2 values for TF-C (0.992) and FB-C (0.981), respectively. Contrasting inferior results were observed in other models, including RF (0.962 and 0.961), GBR (0.987 and 0.975), SVR (0.839 and 0.763), KRR (0.817 and 0.881), and LGBM (0.975 and 0.868). Additionally, a hybrid dataset combining both biochars in Pb adsorption also indicated high accuracy (0.972) as obtained from XGB model. The investigation revealed that the influence of char characteristics and adsorption conditions on Pb adsorption differs between the two biochar. Specific char characteristics, particularly nitrogen content, significantly influence lead adsorption in both biochar. Interestingly, the influence of pyrolysis temperature (PT) on lead adsorption is found to be greater for TF-C than for FB-C. Consequently, careful consideration of PT is crucial when preparing TF-C biochar. These findings offer practical guidance for optimizing biochar preparation conditions during heavy metal removal from wastewater.

Smooth Nested Simulation: Bridging Cubic and Square Root Convergence Rates in High Dimensions

Nested simulation concerns estimating functionals of a conditional expectation via simulation. In this paper, we propose a new method based on kernel ridge regression to exploit the smoothness of the conditional expectation as a function of the multidimensional conditioning variable. Asymptotic analysis shows that the proposed method can effectively alleviate the curse of dimensionality on the convergence rate as the simulation budget increases, provided that the conditional expectation is sufficiently smooth. The smoothness bridges the gap between the cubic root convergence rate (that is, the optimal rate for the standard nested simulation) and the square root convergence rate (that is, the canonical rate for the standard Monte Carlo simulation). We demonstrate the performance of the proposed method via numerical examples from portfolio risk management and input uncertainty quantification. This paper was accepted by Baris Ata, stochastic models and simulation. Funding: The authors acknowledge financial support from the National Natural Science Foundation of China [Grant NSFC 12101149] and the Hong Kong Research Grants Council [Grants GRF 17201520 and GRF 17206821]. Supplemental Material: The e-companion and data files are available at https://doi.org/10.1287/mnsc.2022.00204 .

Strategies for the fast optimization of the glass transition temperature of sustainable epoxy resin systems via machine learning

AbstractAligned with the prevailing sustainability paradigm, the imperative adoption of bio‐based substitutes for constituents within petroleum‐derived epoxy resin becomes evident. Blending bio‐based and petroleum‐based epoxy resins and curing agents, establishes a synergistic compromise addressing both sustainability imperatives and the mechanical efficacy of thermosets. The conventional approach to discovering optimal compositions for multi‐component mixtures under specific boundary conditions includes empirical trial and error and is seen as a protracted and inefficient endeavor. Conversely, leveraging machine learning might afford a streamlined and confident resolution to this challenge. This investigation elucidates the requisite strategies for maximizing the efficiency of material property optimization through the application of Bayesian optimization and active learning. Illustratively, the study demonstrates the proficient optimization of the glass transition temperature within a four‐component epoxy resin system. This optimization is conducted across varying ranges of bio‐content and cost considerations. The study underscores the utility of machine learning in achieving this task with notable efficiency. The efficacy of least squares, kernel ridge regression, Gaussian process regression, and artificial neural networks, is meticulously evaluated through comprehensive seven‐fold cross‐validation and validated against experimental data.

Functional group analysis and machine learning techniques for MIE prediction

The successful prediction of minimum ignition energies (MIEs) for 55 flammable organic molecules has been accomplished through group contribution and machine learning methods. The applied techniques include least squares regression, Huber regression, and kernel ridge regression, with the Marrero/Gani method applied to determine structurally dependent descriptors to uniquely characterize each molecule. These descriptors were used as predictors for the aforementioned regressions techniques to develop four predictive models. The rudimentary least squares regression resulted in a modest R2 of 0.939 on the at-large data set, but overpredicted the MIEs of several compounds. An improved least squares regression featured a lower R2 of 0.840, but with virtually no overprediction. Outlier analysis was conducted with the Huber and kernel ridge techniques, and these models exhibited reduced outlier influence and considered the non-linear relationship between predictors and MIEs. These improved algorithms also used L2 regularization to reduce sensitivity of MIE predictions on statistically insignificant descriptors. Resulting R2 values for models developed using the Huber and kernel ridge techniques came out to be 0.878 and 0.991, respectively, when applied to the at-large data set, and featured little overprediction. Thus, it is concluded that simple group contribution methods, optimized by Huber and kernel ridge techniques, are potential modeling alternatives for simple and accurate prediction of organic molecule MIEs.

Measurement and analysis of void fraction in horizontal plug flow based on electromagnetic wave sensor

Void fraction measurement is a crucial challenge in two-phase flow, which is related to the measurement of phase flow and the calculation of pressure drop. In this study, a coaxial line phase sensor is designed for void fraction acquisition and the void fraction of the plug measured has a well time-domain and Probability Density Functions (PDF) distribution characteristics. Four typical correlations were selected to compare with the measured values of the sensors, the mean absolute errors (MAPE) are 19.88 %, 18.94 %, 18.25 %, and 17.19 %, respectively. Considering the nonlinear and strong coupling between the flow characteristic parameters and the void fraction, the flow parameters were analyzed and selected by Pearson correlation coefficient. The Kernel ridge regression methods were employed to select effective variables for model training, and the Extreme gradient boosting (XGboost) machine learning model was proposed. The prediction accuracy of the model is 1.65 % (current data) and 4.18 % (literature data), respectively.

Machine learning reveals regime shifts in future ocean carbon dioxide fluxes inter-annual variability

The inter-annual variability of global ocean air-sea CO2 fluxes are non-negligible, modulates the global warming signal, and yet it is poorly represented in Earth System Models (ESMs). ESMs are highly sophisticated and computationally demanding, making it challenging to perform dedicated experiments to investigate the key drivers of the CO2 flux variability across spatial and temporal scales. Machine learning methods can objectively and systematically explore large datasets, ensuring physically meaningful results. Here, we show that a kernel ridge regression can reconstruct the present and future CO2 flux variability in five ESMs. Surface concentration of dissolved inorganic carbon (DIC) and alkalinity emerge as the critical drivers, but the former is projected to play a lesser role in the future due to decreasing vertical gradient. Our results demonstrate a new approach to efficiently interpret the massive datasets produced by ESMs, and offer guidance into future model development to better constrain the CO2 flux.

Nuclear mass predictions of the relativistic continuum Hartree-Bogoliubov theory with the kernel ridge regression

Nuclear mass predictions of the relativistic continuum Hartree-Bogoliubov theory with the kernel ridge regression

PSMA PET/CT as a predictive tool for subregional importance estimates in the parotid gland

Objective. Xerostomia and radiation-induced salivary gland dysfunction remain a common side effect for head-and-neck radiotherapy patients, and attempts have been made to quantify the heterogeneity of the dose response within parotid glands. Prostate Specific Membrane Antigen (PSMA) ligands have demonstrated high uptake in salivary glands, which has been shown to correlate with gland functionality. Here we compare several models of parotid gland subregional relative importance with PSMA positron emission tomography (PET) uptake. We then develop a predictive model for Clark et al's relative importance estimates using PSMA PET and CT radiomic features, and demonstrate a methodology for predicting patient-specific importance deviations from the population. Approach. Intra-parotid gland uptake was compared with four regional importance models using 30 [18F]DCFPyL PSMA PET images. The correlation of uptake and importance was ascertained when numerous non-overlapping subregions were defined, while a paired t-test was used to compare binary region pairs. A radiomics-based predictive model of population importance was developed using a double cross-validation methodology. A model was then devised for supplementing population-level subregional importance estimates for each patient using patient-specific radiomic features. Main Results. Anticorrelative relationships were found to exist between PSMA PET uptake and four independent models of subregional parotid gland importance from the literature. Kernel Ridge Regression with principal component analysis feature selection performed best over test sets (Mean Absolute Error = 0.08), with gray level co-occurrence matrix (GLCM) features being particularly important. Deblurring PSMA PET images with neural blind deconvolution strengthened correlations and improved model performance. Significance. This study suggests that regions of relatively low PSMA PET uptake in parotid glands may exhibit relatively high dose-sensitivity. We’ve demonstrated the utility of PSMA PET radiomic features for predicting relative importance within subregions of parotid glands. PSMA PET appears to be a promising quantitative imaging modality for analyzing salivary gland functionality.

Nuclear charge radius predictions by kernel ridge regression with odd–even effects

Nuclear charge radius predictions by kernel ridge regression with odd–even effects

Development of novel computational models based on artificial intelligence technique to predict the viscosity of ionic liquids-water mixtures

This paper delves into the practical application of K-Nearest Neighbors (KNN), Kernel Ridge Regression (KRR), and Lasso Regression for the prediction of viscosity of ionic liquids in a dataset characterized by categorical variables (Cation, Anion) and numeric variables (T(K), xIL(mol%)). Indeed, mole percentage of ionic liquids and temperature were considered as inputs for the models. The models' effectiveness is rigorously assessed, with K-Nearest Neighbors notably exhibiting exceptional predictive performance. To enhance model accuracy, Tabu Search is employed as an optimization tool for hyperparameter tuning. Numeric results showcase KNN's superiority, supported by a remarkable R2 test score of 0.91628 and the lowest RMSE among the models. Tabu Search optimization further refines model performance, emphasizing the critical role of hyperparameter tuning in achieving robust regression models in predicting the viscosity of ionic liquid-water mixtures. This study contributes valuable insights into the optimization process, demonstrating tangible improvements in predictive accuracy for viscosity predictions in similar contexts.

Predicting the band gap of lead-free inorganic double perovskites using modified parallel residual network

The rapid and precise screening of appropriate inorganic perovskite materials poses a formidable challenge in the field of materials science. Traditional methods for material screening are not only time-consuming but also demand a substantial workforce. In this study, we introduce a modified parallel residual network (PRN) for the purpose of predicting the band gap of lead-free inorganic double perovskite materials, utilizing the atomic composition as the input data. The predictive performance of PRN is assessed using root mean square error (RMSE) and Pearson correlation coefficient ( r) as evaluation metrics. The PRN model yields a band gap prediction with an RMSE of 0.402 eV and an r value of 0.962, respectively. Notably, PRN outperforms various alternative models, including random forest regression (RFR), kernel ridge regression (KRR), support vector regression (SVR), extreme gradient boosting regression (XGBR), one-layer residual variant network (OLRN), and three-layer parallel residual variant network (TLPRN), clearly demonstrating its superior predictive accuracy.

Pore-induced fatigue failure: A prior progressive fatigue life prediction framework of laser-directed energy deposition Ti-6Al-4V based on machine learning

Pores are major cause of fatigue failure in laser-directed energy deposition (L-DED) titanium alloy. For the safe application of L-DED titanium alloys, it is essential to establish a fatigue life prediction method based on pore-induced fatigue. This paper proposes a prior progressive fatigue life prediction framework based on ridge classification and kernel ridge regression algorithms. The fatigue life prediction was carried out on L-DED Ti-6Al-4V alloy in three steps: critical pore identification, fine granular area existence prediction and final fatigue life prediction. The fatigue life prediction method adopted in the current study outperform the others with a correlation coefficient as high as 0.951, followed by a comparison with the results derived from different machine learning algorithms. The results show that the proposed fatigue life prediction framework can predict the fatigue life of L-DED Ti-6Al-4V alloy based on computed tomography tests and microstructure features. Due to its strong generalization ability and effectiveness, the proposed prediction method is expected to be valuable for fatigue-resistant design of L-DED Ti-6Al-4V alloy.

A hybrid machine-learning model for solar irradiance forecasting

Abstract Nowcasting and forecasting solar irradiance are vital for the optimal prediction of grid-connected solar photovoltaic (PV) power plants. These plants face operational challenges and scheduling dispatch difficulties due to the fluctuating nature of their power output. As the generation capacity within the electric grid increases, accurately predicting this output becomes increasingly essential, especially given the random and non-linear characteristics of solar irradiance under variable weather conditions. This study presents a novel prediction method for solar irradiance, which is directly in correlation with PV power output, targeting both short-term and medium-term forecast horizons. Our proposed hybrid framework employs a fast trainable statistical learning technique based on the truncated-regularized kernel ridge regression model. The proposed method excels in forecasting solar irradiance, especially during highly intermittent weather periods. A key strength of our model is the incorporation of multiple historical weather parameters as inputs to generate accurate predictions of future solar irradiance values in its scalable framework. We evaluated the performance of our model using data sets from both cloudy and sunny days in Seattle and Medford, USA and compared it against three forecasting models: persistence, modified 24-hour persistence and least squares. Based on three widely accepted statistical performance metrics (root mean squared error, mean absolute error and coefficient of determination), our hybrid model demonstrated superior predictive accuracy in varying weather conditions and forecast horizons.

Degeneration of kernel regression with Matern kernels into low-order polynomial regression in high dimension.

Kernel methods such as kernel ridge regression and Gaussian process regression with Matern-type kernels have been increasingly used, in particular, to fit potential energy surfaces (PES) and density functionals, and for materials informatics. When the dimensionality of the feature space is high, these methods are used with necessarily sparse data. In this regime, the optimal length parameter of a Matern-type kernel may become so large that the method effectively degenerates into a low-order polynomial regression and, therefore, loses any advantage over such regression. This is demonstrated theoretically as well as numerically in the examples of six- and fifteen-dimensional molecular PES using squared exponential and simple exponential kernels. The results shed additional light on the success of polynomial approximations such as PIP for medium-size molecules and on the importance of orders-of-coupling-based models for preserving the advantages of kernel methods with Matern-type kernels of on the use of physically motivated (reproducing) kernels.

Instance transfer partial least squares for semi-supervised adaptive soft sensor

In order to deal with the unknown data drift problem caused by the change of working conditions in the process of industrial soft sensor modeling, this paper proposes a semi-supervised soft sensor modeling algorithm based on instance transfer learning, named instance transfer partial least squares. Under the framework of nonlinear partial least squares, the algorithm introduces the instance transfer inner model, and realizes the distribution adaptation of the modeling data and the data to be predicted through sample importance weighting. First, a nonlinear partial least squares model is constructed by using labeled samples in the source domain, which consists of a linear outer model and a nonlinear inner model. The linear outer model maps the original data to the hidden variable space, and the nonlinear inner model realizes the nonlinear modeling of hidden variables through kernel ridge regression. Second, a small number of labeled samples from the target domain are used to update the model weights of the nonlinear inner model, enabling adaptation of the latent variable distribution. The effectiveness of the algorithm is verified on a toy dataset, a simulation dataset and a real pulverized coal power plant dataset.

Forest yield prediction under different climate change scenarios using data intelligent models in Pakistan.

This study aimed to develop and evaluate data driven models for prediction of forest yield under different climate change scenarios in the Gallies forest division of district Abbottabad, Pakistan. The Random Forest (RF) and Kernel Ridge Regression (KRR) models were developed and evaluated using yield data of two species (Blue pine and Silver fir) as an objective variable and climate data (temperature, humidity, rainfall and wind speed) as predictive variables. Prediction accuracy of both the models were assessed by means of root mean squared error (RMSE), mean absolute error (MAE), correlation coefficient (r), relative root mean squared error (RRMSE), Legates-McCabe's (LM), Willmott's index (WI) and Nash-Sutcliffe (NSE) metrics. Overall, the RF model outperformed the KRR model due to its higher accuracy in forecasting of forest yield. The study strongly recommends that RF model should be applied in other regions of the country for prediction of forest growth and yield, which may help in the management and future planning of forest productivity in Pakistan.