Kernel Ridge Regression Method Research Articles

Stripping the Swiss discount curve using kernel ridge regression

We analyze and implement the kernel ridge regression (KR) method developed in Filipovic et al. (Stripping the discount curve—a robust machine learning approach. Swiss Finance Institute Research Paper No. 22–24. SSRN. https://ssrn.com/abstract=4058150, 2022) to estimate the risk-free discount curve for the Swiss government bond market. We show that the insurance industry standard Smith–Wilson method is a special case of the KR framework. We recapitulate the curve estimation methods of the Swiss Solvency Test (SST) and the Swiss National Bank (SNB). In an extensive empirical study covering the years 2010–2022 we compare the KR curves with the SST and SNB curves. The KR method proves to be robust, flexible, transparent, reproducible and easy to implement, and outperforms the benchmarks in- and out-of-sample. We show the limitations of all methods for extrapolating the yield curve and propose possible solutions for the extrapolation problem. We conclude that the KR method is the preferred method for estimating the discount curve.

Effect of initial-learning dataset on etching profile optimization using machine learning in plasma etching

Machine learning (ML) was applied to optimize the etching profile for a line and space pattern sample in plasma etching. To investigate the effect of different initial-learning datasets on the optimization of the etching profile, high-, medium-, and low-quality datasets were prepared. The high-quality dataset was composed of etching results relatively close to a target etching profile. The low-quality dataset was composed of etching results relatively far from the target etching profile. The medium-quality dataset was intermediate between the high- and low-quality datasets. For the ML, the kernel ridge regression method was used. After six learning cycles, better etching results were obtained from the medium- and low-quality datasets than from the whole initial-learning dataset. However, the etching results from the high-quality dataset did not exceed those from the whole initial-learning dataset. These results indicate that an initial-learning dataset that has etching results far from the target profile can be useful for optimizing etching profiles.

Ab Initio Molecular Cavity Quantum Electrodynamics Simulations Using Machine Learning Models.

We present a mixed quantum-classical simulation of polariton dynamics for molecule-cavity hybrid systems. In particular, we treat the coupled electronic-photonic degrees of freedom (DOFs) as the quantum subsystem and the nuclear DOFs as the classical subsystem and use the trajectory surface hopping approach to simulate non-adiabatic dynamics among the polariton states due to the coupled motion of nuclei. We use the accurate nuclear gradient expression derived from the Pauli-Fierz quantum electrodynamics Hamiltonian without making further approximations. The energies, gradients, and derivative couplings of the molecular systems are obtained from the on-the-fly simulations at the level of complete active space self-consistent field (CASSCF), which are used to compute the polariton energies and nuclear gradients. The derivatives of dipoles are also necessary ingredients in the polariton nuclear gradient expression but are often not readily available in electronic structure methods. To address this challenge, we use a machine learning model with the Kernel ridge regression method to construct the dipoles and further obtain their derivatives, at the same level as the CASSCF theory. The cavity loss process is modeled with the Lindblad jump superoperator on the reduced density of the electronic-photonic quantum subsystem. We investigate the azomethane molecule and its photoinduced isomerization dynamics inside the cavity. Our results show the accuracy of the machine-learned dipoles and their usage in simulating polariton dynamics. Our polariton dynamics results also demonstrate the isomerization reaction of azomethane can be effectively tuned by coupling to an optical cavity and by changing the light-matter coupling strength and the cavity loss rate.

Examination of machine learning for assessing physical effects: Learning the relativistic continuum mass table with kernel ridge regression* *Supported by the National Natural Science Foundation of China (11875075, 11935003, 11975031, 12141501, 12070131001), the China Postdoctoral Science Foundation under (2021M700256), the State Key Laboratory of Nuclear Physics and Technology, Peking University (NPT2023ZX01, NPT2023KFY02), and the President's Undergraduate Research Fellowship (PURF) of

The kernel ridge regression (KRR) method and its extension with odd-even effects (KRRoe) are used to learn the nuclear mass table obtained by the relativistic continuum Hartree-Bogoliubov theory. With respect to the binding energies of 9035 nuclei, the KRR method achieves a root-mean-square deviation of 0.96 MeV, and the KRRoe method remarkably reduces the deviation to 0.17 MeV. By investigating the shell effects, one-nucleon and two-nucleon separation energies, odd-even mass differences, and empirical proton-neutron interactions extracted from the learned binding energies, the ability of the machine learning tool to grasp the known physics is discussed. It is found that the shell effects, evolutions of nucleon separation energies, and empirical proton-neutron interactions are well reproduced by both the KRR and KRRoe methods, although the odd-even mass differences can only be reproduced by the KRRoe method.

Determination of cohesive parameters for fibre-reinforced composite interfaces based on finite element analysis and machine learning

This paper proposed a method for determining the cohesive parameters of fibre-reinforced composite interfaces based on finite element analysis (FEA) and machine learning. 3D FEA models with different boundary conditions and 2D FEA models were created to simulate the process of microdroplet tests, and to compare their maximum reaction forces and the time costs. The proper FEA model that is accurate and efficient was adopted to establish the data set of machine learning. Machine learning based on the FEA data set was divided into two steps: feature selection and kernel ridge regression (KRR) prediction. Feature selection was carried out to confirm the validity of the features, to obtain the optimal parameter of KRR prediction and to quantitatively illustrate the effect of the cohesive parameters on the maximum reaction force. Interfacial shear strength (IFSS) and interfacial fracture toughness (IFFT) of a poly ( p-phenylene benzobisoxazole) (PBO) fibre-reinforced epoxy composite were successfully predicted by the KRR method without extra mechanical theories or assumptions.

Improved phenomenological nuclear charge radius formulae with kernel ridge regression * *Supported by National Natural Science Foundation of China (11875027, 11775112, 11775026, 11775099, 11975096), Fundamental Research Funds for the Central Universities (2021MS046)

The kernel ridge regression (KRR) method with a Gaussian kernel is used to improve the description of the nuclear charge radius by several phenomenological formulae. The widely used , and formulae, and their improved versions including isospin dependence, are adopted as examples. The parameters in these six formulae are refitted using the Levenberg–Marquardt method, which give better results than the previous versions. The radius for each nucleus is predicted with the KRR network, which is trained with the deviations between experimental and calculated nuclear charge radii. For each formula, the resultant root-mean-square deviations of 884 nuclei with proton number and neutron number can be reduced to about 0.017 fm after considering the modification by the KRR method. The extrapolation ability of the KRR method for the neutron-rich region is examined carefully and compared with the radial basis function method. It is found that the improved nuclear charge radius formulae using the KRR method can avoid the risk of overfitting, and have a good extrapolation ability. The influence of the ridge penalty term on the extrapolation ability of the KRR method is also discussed. Finally, the nuclear charge radii of several recently observed K and Ca isotopes are analyzed.

Machine learning approach for longitudinal spin fluctuation effects in bcc Fe atTcand under Earth-core conditions

We propose a machine learning approach to predict the shapes of the longitudinal spin fluctuation (LSF) energy landscapes for each local magnetic moment. This approach allows the inclusion of the effects of LSFs in, e.g., the simulation of a magnetic material with ab initio molecular dynamics in an effective way. This type of simulation requires knowledge of the reciprocal interaction between atoms and moments, which, in principle, would entail calculating the energy landscape of each atom at every instant in time. The machine learning approach is based on the kernel ridge regression method and developed using bcc Fe at the Curie temperature and ambient pressure as a test case. We apply the trained machine learning models in a combined atomistic spin dynamics and ab initio molecular dynamics (ASD-AIMD) simulation, where they are used to determine the sizes of the magnetic moments of every atom at each time step. In addition to running an ASD-AIMD simulation with the LSF machine learning approach for bcc Fe at the Curie temperature, we also simulate Fe at temperature and pressure comparable to the conditions at the Earth's inner solid core. The latter simulation serves as a critical test of the generality of the method and demonstrates the importance of the magnetic effects in Fe in the Earth's core despite its extreme temperature and pressure.

Mapping leaf chlorophyll content of mangrove forests with Sentinel-2 images of four periods

Chlorophyll is a good indicator of health status and nutritional condition as plant grows. Many studies have investigated the feasibility of retrieving leaf chlorophyll content (LCC) using vegetation indices (VIs) of multiple plant species, yet very few studies have examined the multi-temporal Sentinel-2 images for mapping LCC of mangrove forests. With field collected leaf SPAD values (relative chlorophyll content), this study explored the relationship of leaf SPAD values against five types of newly-developed VIs derived from leaf hyperspectral data and Sentinel-2 data of four periods (May 2018, January 2019, August 2019, and December 2019). Linear regression with best-performing VIs and Kernel Ridge Regression (KRR) were developed to construct the SPAD retrieval model in each period. The leave-one-out cross-validation technique was employed to compare the estimation results of VIs and KRR method, and the four periods of SPAD maps were produced by the best-performing model. The results showed that the newly-developed index (ratio of single-band reflectance to the sum of two bands reflectance, RSSI) achieved the high correlation coefficient with leaf SPAD value at both leaf and canopy level. At canopy level, the linear model using RSSI (B8/(B2 + B5), B8a/(B2 + B4), B8/(B2 + B5), and B8/(B2 + B3)) outperformed than that using traditional broadband indices and KRR model with R2adjust = 0.496, 0.742, 0.681, and 0.801; RMSE = 5.75, 4.29, 4.00, and 3.46; and RE = 7.67%, 5.68%, 4.97%, and 4.63% in each period. We concluded that there are great potentials of newly-developed index of RSSI using Sentinel-2 data for regional retrieving and mapping LCC of mangrove forests across different time periods, which is essential for mangrove ecological conservation and restoration.

Artificial neural networks for predicting charge transfer coupling.

Quantum chemistry calculations have been very useful in providing many key detailed properties and enhancing our understanding of molecular systems. However, such calculation, especially with ab initio models, can be time-consuming. For example, in the prediction of charge-transfer properties, it is often necessary to work with an ensemble of different thermally populated structures. A possible alternative to such calculations is to use a machine-learning based approach. In this work, we show that the general prediction of electronic coupling, a property that is very sensitive to intermolecular degrees of freedom, can be obtained with artificial neural networks, with improved performance as compared to the popular kernel ridge regression method. We propose strategies for optimizing the learning rate and batch size, improving model performance, and further evaluating models to ensure that the physical signatures of charge-transfer coupling are well reproduced. We also address the effect of feature representation as well as statistical insights obtained from the loss function and the data structure. Our results pave the way for designing a general strategy for training such neural-network models for accurate prediction.

Tensor sparse PCA and face recognition: a novel approach

Face recognition is the important field in machine learning and pattern recognition research area. It has a lot of applications in military, finance, public security, to name a few. In this paper, the combination of the tensor sparse PCA with the nearest-neighbor method (and with the kernel ridge regression method) will be proposed and applied to the face dataset. Experimental results show that the combination of the tensor sparse PCA with any classification system does not always reach the best accuracy performance measures. However, the accuracy of the combination of the sparse PCA method and one specific classification system is always better than the accuracy of the combination of the PCA method and one specific classification system and is always better than the accuracy of the classification system itself.

Access to savings and household welfare evidence from a household survey in The Gambia

AbstractIn this study, we provide evidence on the impact of access to formal savings on household well‐being in The Gambia. Specifically, we study how access to formal savings can impact household outcomes such as total spending, ownership of durable assets, health spending, and education spending. Using a representative household survey and kernel ridge regression method, we find that household access to formal savings has a positive and statistically significant impact on all outcomes except health spending. Furthermore, we find that the largest effect size of access to savings is on education spending. However, the impact estimates on education and food expenditure are not very robust to a mild presence of hidden bias. Overall, we find a robust impact for total spending and asset ownership. Hence, increasing household access to formal savings can improve household well‐being in The Gambia.

An Efficient Method to Predict Compressibility Factor of Natural Gas Streams

The gas compressibility factor, also known as the deviation or Z-factor, is one of the most important parameters in the petroleum and chemical industries involving natural gas, as it is directly related to the density of a gas stream, hence its flow rate and isothermal compressibility. Obtaining accurate values of the Z-factor for gas mixtures of hydrocarbons is challenging due to the fact that natural gas is a multicomponent, non-ideal system. Traditionally, the process of estimating the Z-factor involved simple empirical correlations, which often yielded weak results either due to their limited accuracy or due to calculation convergence difficulties. The purpose of this study is to apply a hybrid modeling technique that combines the kernel ridge regression method, in the form of the recently developed Truncated Regularized Kernel Ridge Regression (TR-KRR) algorithm, in conjunction with a simple linear-quadratic interpolation scheme to estimate the Z-factor. The model is developed using a dataset consisting of 5616 data points taken directly from the Standing–Katz chart and validated using the ten-fold cross-validation technique. Results demonstrate an average absolute relative prediction error of 0.04%, whereas the maximum absolute and relative error at near critical conditions are less than 0.01 and 2%, respectively. Most importantly, the obtained results indicate smooth, physically sound predictions of gas compressibility. The developed model can be utilized for the direct calculation of the Z-factor of any hydrocarbon mixture, even in the presence of impurities, such as N 2 , CO 2 , and H 2 S, at a pressure and temperature range that fully covers all upstream operations and most of the downstream ones. The model accuracy combined with the guaranteed continuity of the Z-factor derivatives with respect to pressure and temperature renders it as the perfect tool to predict gas density in all petroleum engineering applications. Such applications include, but are not limited to, hydrocarbon reserves estimation, oil and gas reservoir modeling, fluid flow in the wellbore, the pipeline system, and the surface processing equipment.

Stability Region of Droop-Controlled Distributed Generation in Autonomous Microgrids

This paper proposes a conceptual framework for geometrically characterizing the small-disturbance stable operating area of an autonomous microgrid. A concept of stability region of distributed generation (DGSR) is established on a general differential-algebraic-equation model of autonomous microgrids. It provides a graphical tool to characterize the limits for a microgrid to accommodate the distributed, volatile, and uncertain power generation of renewable energy, inside which the stability of the system can be guaranteed. The Kernel Ridge Regression method is employed to accurately and quickly estimate the boundary of the DGSR, leading to its approximate expression in an analytical form. The DGSR allows for a straightforward and intuitive evaluation of stability and provides guidance for adjustments to enhance it. Case studies on two autonomous microgrids are used to verify the performance of the proposed DGSR and its potential applications.

Predicting sooting tendencies of oxygenated hydrocarbon fuels with machine learning algorithms

The use of oxygenated components is an effective way to suppress soot formation and reduce particulate matter emission in diesel engines. Based on the group additivity method, we proposed a new statistical model with kernel ridge regression (KRR) to predict the sooting tendencies of oxygenated components and interpret the experimental results of fuel sooting tendency. Through a training process, the relationship between fuel molecular structure and sooting tendency was built. In the testing process, the predicted values of the yield sooting index were in good agreement with the measured values, indicating that the KRR model had good predictive performance. The KRR method can be used to compare the sooting tendencies of various components that are important in traditional or new fuels. For example, based on the predicted results of typical five-carbon oxygenated compounds, the inhibiting effects of oxygen functional groups on sooting tendency are in the following order: esters > ketones ≈ aldehydes > ethers > alcohols. In addition, the proposed method provides statistical evidence for the effects of structural features on sooting tendencies. The predictive results for typical ethers indicate that although sooting tendencies increase as carbon chains become more branched, they are not very sensitive to the specific arrangement of the branches compared with the effect of carbon chain length. Moreover, the KRR method is useful for studying non-nearest-neighbor interactions, including functional group interactions. By predicting the sooting tendencies of typical esters, the coupling of ester moiety and CC double bonds shows an overall enhancing effect on sooting tendency, which indicates that CC double bonds play a dominant role in soot formation. This method provides an alternative way to obtain the sooting tendencies of fuel compounds and verifies the selection of low-sooting fuels from a wide range of feedstocks.

An Improved Calibration Technique for Polarization Images

Polarization imaging can be used to improve the image quality by reducing the effects of the unwanted light reflection, enhancing the quality of images taken in non-ideal conditions, such as foggy weather, or reconstructing the 3D form of an object based on the shape from polarization. Reconstructing a 3D form requires the acquisition of multiple images and inference of the polarization parameters, such as the degree of polarization and angle of polarization, from these experimental data. The light polarization extraction process should begin by performing the photometric calibration to improve the performance of the polarize instrument measurements by reducing the measurement errors and improving the consistency between measurements. We propose an algorithm based on the kernel ridge regression method to estimate both the polarization parameters and the measurement (angle) errors in the placement of the polarizer. The algorithm was tested on four different sets of images containing different objects based on their responses to light and polarization. The algorithm that uses the mixed kernel function gives the best results.

Efficient Data-Driven Geologic Feature Characterization from Pre-stack Seismic Measurements using Randomized Machine-Learning Algorithm

Conventional seismic techniques for detecting the subsurface geologic features are challenged by limited data coverage, computational inefficiency, and subjective human factors. We developed a novel data-driven geological feature detection approach based on pre-stack seismic measurements. Our detection method employs an efficient and accurate machine-learning detection approach to extract useful subsurface geologic features automatically. Specifically, our method is based on kernel ridge regression model. The conventional kernel ridge regression can be computationally prohibited because of the large volume of seismic measurements. We employ a data reduction technique in combination with the conventional kernel ridge regression method to improve the computational efficiency and reduce memory usage. In particular, we utilize a randomized numerical linear algebra technique, named Nystr\"om method, to effectively reduce the dimensionality of the feature space without compromising the information content required for accurate detection. We provide thorough computational cost analysis to show efficiency of our new geological feature detection methods. We further validate the performance of our new subsurface geologic feature detection method using synthetic surface seismic data for 2D acoustic and elastic velocity models. Our numerical examples demonstrate that our new detection method significantly improves the computational efficiency while maintaining comparable accuracy. Interestingly, we show that our method yields a speed-up ratio on the order of $\sim10^2$ to $\sim 10^3$ in a multi-core computational environment.

Benchmarking Ensemble Classifiers with Novel Co-Trained Kernel Ridge Regression and Random Vector Functional Link Ensembles [Research Frontier

Studies in machine learning have shown promising classification performance of ensemble methods employing perturb and combine strategies. In particular, the classical random forest algorithm performs the best among 179 classifiers on 121 UCI datasets from different domains. Motivated by this observation, we extend our previous work on oblique decision tree ensemble. We also propose an efficient co-trained kernel ridge regression method. In addition, a random vector functional link network ensemble is also introduced. Our experiments show that our two oblique decision tree ensemble variants and the co-trained kernel ridge regression ensemble are the top three ranked methods among the 183 classifiers. The proposed random vector functional link network ensemble also outperforms all neural network based methods used in the experiments.

機械学習および量子化学計算を用いた気体材料の絶縁破壊電界・沸点予測の精度向上に関する一検討

This study demonstrates that the accuracy of empirical prediction of electric breakdown field of gases can be improved by adopting an appropriate machine learning approach. The performance of the machine learning models for predicting electrical breakdown strengths were evaluated by means of double cross validation technique. It is shown that the coefficient of determination between experimental and predicted electric breakdown strengths can be increased by roughly 30% and the standard deviation can be decreased by roughly 30% by adopting kernel ridge regression (KRR) method and by choosing the best number and combinations of predictors. Electric breakdown strengths and boiling points of (CF3)2CFCN and CF3C(O)CF(CF3)2 molecules that are recently proposed as alternative gases for SF6, are predicted by KRR method with the aid of quantum chemical calculations. Predicted electric breakdown strengths and boiling points were in good agreement with experimental findings; the prediction errors of breakdown strengths and boiling points were within 30 and 10%, respectively.

Applications of kernel ridge estimation to the problem of computing the aerodynamical characteristics of a passenger plane (in comparison with results obtained with artificial neural networks)

We show the results of applying the kernel ridge regression method to the problem of fast computation of aerodynamical characteristics of an airplane. We compare our results with the results found with surrogate models constructed with artificial neural networks.