Application Of Gaussian Process Regression Research Articles

Leveraging Bayesian Quadrature for Accurate and Fast Credit Valuation Adjustment Calculations

Counterparty risk, which combines market and credit risks, gained prominence after the 2008 financial crisis due to its complexity and systemic implications. Traditional management methods, such as netting and collateralization, have become computationally demanding under frameworks like the Fundamental Review of the Trading Book (FRTB). This paper explores the combined application of Gaussian process regression (GPR) and Bayesian quadrature (BQ) to enhance the efficiency and accuracy of counterparty risk metrics, particularly credit valuation adjustment (CVA). This approach balances excellent precision with significant computational performance gains. Focusing on fixed-income derivatives portfolios, such as interest rate swaps and swaptions, within the One-Factor Linear Gaussian Markov (LGM-1F) model framework, we highlight three key contributions. First, we approximate swaption prices using Bachelier’s formula, showing that forward-starting swap rates can be modeled as Gaussian dynamics, enabling efficient CVA computations. Second, we demonstrate the practical relevance of an analytical approximation for the CVA of an interest rate swap portfolio. Finally, the combined use of Gaussian processes and Bayesian quadrature underscores a powerful synergy between precision and computational efficiency, making it a valuable tool for credit risk management.

Application of Gaussian Process Regression for Bench Blasting Rock Fragmentation Prediction and Optimization at Wolongan Open-Pit Mine

Application of Gaussian Process Regression for Bench Blasting Rock Fragmentation Prediction and Optimization at Wolongan Open-Pit Mine

Application of Gaussian process regression and asymmetric least squares baseline algorithm on the determination of electrochemical sensor characteristics: A case study on SARS-CoV-2 glucometer

In this study, it was aimed to determine limit of detection, limit of quantification, and signal to noise ratio by the use of SARS-CoV-2 glucometer response data in "Singh, N·K., Ray, P., Carlin, A.F., Morgan, S·C., Magallanes, C., Laurent, L.C., Aronoff-Spencer, E.S., Hall, D.A., 2021b. Dataset on optimization and development of a point-of-care glucometer-based SARS-CoV-2 detection assay using aptamers. Data Brief 38, 107,278.” and by Bayesian optimization and asymmetric least squares baseline algorithm which was used for prediction of response curve and baseline. It was seen that the predicted limit of detection and limit of quantification reached a certain level after a certain number of training observations and ranged between 0.9-16 pM and 2.8–44 pM respectively.

Molecular dipole moment learning via rotationally equivariant derivative kernels in molecular-orbital-based machine learning.

This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree-Fock computations. A MOB pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of MOs. The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant derivative kernel for Gaussian process regression (GPR) is introduced to learn the dipole moment more efficiently. The proposed problem setup, feature design, and ML algorithm are shown to provide highly accurate models for both dipole moments and energies on water and 14 small molecules. To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset. The application of local scalable GPR with Gaussian mixture model unsupervised clustering GPR scales up MOB-ML to a large-data regime while retaining the prediction accuracy. In addition, compared with the literature results, MOB-ML provides the best test mean absolute errors of 4.21 mD and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110 000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.

Machine Learning-Assisted probabilistic fatigue evaluation of Rib-to-Deck joints in orthotropic steel decks

This study integrates the fatigue test and numerical prediction to derive a comprehensive probability-stress-life (P-S-N) curve for rib-to-deck (RD) welded joints in orthotropic steel decks. Fatigue tests of RD joints are conducted to measure fatigue strength and crack growth data. Based on the test, a probabilistic fatigue crack growth (PFCG) model is established to predict the distribution of fatigue life under various stress ranges. Two machine learning tools are adopted to assist the PFCG model-based prediction, i.e., the Gaussian process regression (GPR) and dynamic Bayesian network (DBN). The GPR is used to train a surrogate model solving stress intensity factors for the PFCG prediction, using 2,000 samples generated from finite element (FE) analyses. The trained model is then validated by a new dataset of 100 FE samples. An adapted DBN model is proposed to update the PFCG model with the fatigue crack growth data measured from ten specimens. According to the result, the application of GPR can reduce the solution cost of the PFCG prediction by approximately 1,875 times. Compared with the prior PFCG model, the updated posterior model shows an improved agreement with the test data, i.e., the maximum difference in fatigue strength between model prediction and test data decreases from 12% to 3%. Based on the posterior PFCG model, the P-S-N curve of RD joints is statistically derived using sufficient numerical samples.

Local Random Feature Approximations of the Gaussian Kernel

A fundamental drawback of kernel-based statistical models is their limited scalability to large data sets, which requires resorting to approximations. In this work, we focus on the popular Gaussian kernel and on techniques to linearize kernel-based models by means of random feature approximations. In particular, we do so by studying a less explored random feature approximation based on Maclaurin expansions and polynomial sketches. We show that such approaches yield poor results when modelling high-frequency data, and we propose a novel localization scheme that improves kernel approximations and downstream performance significantly in this regime. We demonstrate these gains on a number of experiments involving the application of Gaussian process regression to synthetic and real-world data of different data sizes and dimensions.

Application of Gaussian process regression as a surrogate modeling method to assess the dynamics of COVID-19 propagation

In this research, we aimed to assess the possibility of using surrogate modeling methods to replace time-consuming calculations related to modeling of COVID-19 dynamics. The Gaussian process regression (GPR) was used as a surrogate to replace detailed simulations by a COVID-19 multiagent model. Experiments were conducted with various kernels, as a result, in accordance with the quality metrics of the models, kernels were identified in which the surrogate gives the most accurate result (Rational Quadratic kernel and Additive kernel). It was demonstrated that by smoothing the dynamics of COVID-19 propagation, it is possible to achieve greater accuracy in GPR training. The obtained results prove the potential possibility of using surrogate modeling methods to conduct an uncertainty quantification of the multiagent model of COVID-19 propagation.

Application of Gaussian Process Regression (GPR) in Gas Hydrate Mitigation

The production of oil and natural gas contributes to a significant amount of revenue generation in Malaysia thereby strengthening the country’s economy. The flow assurance industry is faced with impediments during smooth operation of the transmission pipeline in which gas hydrate formation is the most important. It affects the normal operation of the pipeline by plugging it. Under high pressure and low temperature conditions, gas hydrate is a crystalline structure consisting of a network of hydrogen bonds between host molecules of water and guest molecules of the incoming gases. Industry uses different types of chemical inhibitors in pipeline to suppress hydrate formation. To overcome this problem, machine learning algorithm has been introduced as part of risk management strategies. The objective of this paper is to utilize Machine Learning (ML) model which is Gaussian Process Regression (GPR). GPR is a new approach being applied to mitigate the growth of gas hydrate. The input parameters used are concentration and pressure of Carbon Dioxide (CO2) and Methane (CH4) gas hydrates whereas the output parameter is the Average Depression Temperature (ADT). The values for the parameter are taken from available data sets that enable GPR to predict the results accurately in terms of Coefficient of Determination, R2 and Mean Squared Error, MSE. The outcome from the research showed that GPR model provided with highest R2 value for training and testing data of 97.25% and 96.71%, respectively. MSE value for GPR was also found to be lowest for training and testing data of 0.019 and 0.023, respectively.

Application of machine learning to predict CO2 trapping performance in deep saline aquifers

Deep saline formations are considered potential sites for geological carbon storage. To better understand the CO2 trapping mechanism in saline aquifers, it is necessary to develop robust tools to evaluate CO2 trapping efficiency. This paper introduces the application of Gaussian process regression (GPR), support vector machine (SVM), and random forest (RF) to predict CO2 trapping efficiency in saline formations. First, the uncertainty variables, including geologic parameters, petrophysical properties, and other physical characteristics data, were utilized to create a training dataset. In total, 101 reservoir simulations were then performed, and residual trapping, solubility trapping, and cumulative CO2 injection were analyzed. The predicted results indicated that three machine learning (ML) models that evaluate performance from high to low (GPR, SVM, and RF) can be selected to predict the CO2 trapping efficiency in deep saline formations. The GPR model had an excellent CO2 trapping prediction efficiency with the highest correlation factor (R2 = 0.992) and the lowest root mean square error (RMSE = 0.00491). Also, the predictive models obtained good agreement between the simulated field and predicted trapping index. These findings indicate that the GPR ML models can support the numerical simulation as a robust predictive tool for estimating the performance of CO2 trapping in the subsurface.

Application of Gaussian process regression to forecast multi-step ahead SPEI drought index

Forecasting of drought can be very useful in preparing to reduce its impacts, especially in the agricultural sector. Three machine learning models of MLP neural network, GRNN neural network, and Gaussian process regression were used to forecast the annual drought index (SPEI12) in intervals of 1 to 3 months ahead in 79 Iranian synoptic stations. For test results, the accuracy of the models was assessed based on RMSE and R2. Synoptic stations were divided into 5 clusters (C1 to C5) based on the drought time series using the K-means algorithm. The findings showed that the accuracy of the models has declined in all three approaches, with the predictive period increasing from one to three months ahead. Across all three forecasting intervals and in all 5 clusters, the Gaussian process regression (GPR) model achieved the lowest RMSE and the highest R2 values. Compared with the other two models used in this analysis, the MLP model had worse results, and the GRNN model had a score between the other two models. The models as mentioned above had the best prediction in cluster 1 (southern and southeastern regions of Iran), and the lowest model accuracy was observed in cluster 5 (Caspian Sea Shores).

Twenty-Four-Hour Ahead Probabilistic Global Horizontal Irradiance Forecasting Using Gaussian Process Regression

Probabilistic solar power forecasting has been critical in Southern Africa because of major shortages of power due to climatic changes and other factors over the past decade. This paper discusses Gaussian process regression (GPR) coupled with core vector regression for short-term hourly global horizontal irradiance (GHI) forecasting. GPR is a powerful Bayesian non-parametric regression method that works well for small data sets and quantifies the uncertainty in the predictions. The choice of a kernel that characterises the covariance function is a crucial issue in Gaussian process regression. In this study, we adopt the minimum enclosing ball (MEB) technique. The MEB improves the forecasting power of GPR because the smaller the ball is, the shorter the training time, hence performance is robust. Forecasting of real-time data was done on two South African radiometric stations, Stellenbosch University (SUN) in a coastal area of the Western Cape Province, and the University of Venda (UNV) station in the Limpopo Province. Variables were selected using the least absolute shrinkage and selection operator via hierarchical interactions. The Bayesian approach using informative priors was used for parameter estimation. Based on the root mean square error, mean absolute error and percentage bias the results showed that the GPR model gives the most accurate predictions compared to those from gradient boosting and support vector regression models, making this study a useful tool for decision-makers and system operators in power utility companies. The main contribution of this paper is in the use of a GPR model coupled with the core vector methodology which is used in forecasting GHI using South African data. This is the first application of GPR coupled with core vector regression in which the minimum enclosing ball is applied on GHI data, to the best of our knowledge.

Gaussian process regression for direct laser absorption spectroscopy in complex combustion environments

Tunable diode laser absorption spectroscopy (TDLAS) has been proved to be a powerful diagnostic tool in combustion research. However, current methods for post-processing a large number of blended spectral lines are often inadequate both in terms of processing speed and accuracy. The present study verifies the application of Gaussian process regression (GPR) on processing direct absorption spectroscopy data in combustion environments to infer gas properties directly from the absorbance spectra. Parallelly-composed generic single-output GPR models and multi-output GPR models based on linear model of coregionalization (LMC) are trained using simulated spectral data at set test matrix to determine multiple unknown thermodynamic properties simultaneously from the absorbance spectra. The results indicate that compared to typical data processing methods by line profile fitting, the GPR models are proved to be feasible for accurate inference of multiple gas properties over a wide spectral range with a manifold of blended lines. While further validation and optimization work can be done, parallelly composed single-output GPR model demonstrates sufficient accuracy and efficiency for the demand of temperature and concentration inference.

Prediction of soil hydraulic properties by Gaussian process regression algorithm in arid and semiarid zones in Iran

The soil water retention curve (SWRC) is one of the principal soil hydraulic properties that is needed as input data in modeling water and solute transport through unsaturated soils. Field or laboratory measurement of SWRC is labor-intensive, expensive and time-consuming. Pedotransfer functions (PTFs) have been developed as an indirect method to predict soil hydraulic properties (e.g. SWRC) from more easily measured soil data by data mining tools. The novelty of the present study is the application of Gaussian process regression (GPR) algorithm as a data mining technique, to predict the SWRC, and comparing its performance with that of the multiple linear regression (MLR) and Rosetta methods, which has not been conducted so far. In this study 15 GPR and MLR-based PTFs were developed to predict the parameters of the van Genuchten model from different combinations of readily available properties of 223 soil samples that were taken from six provinces of Iran. The k-fold (k = 20) cross validation approach was utilized to obtain training and testing data sets for each PTF. The predictions of the GPR and MLR-based PTFs were evaluated by different criteria. The GPR-based PTFs had greater accuracy and reliability than MLR method in predicting SWRC according to integral root mean square error (IRMSE) criterion. However, the differences were not significant (P > 0.05) in the testing step, but the reliability of both methods were significantly (P < 0.05) better than Rosetta-based PTFs. The covariance functions of GPR method can be effectively fitted by kernels with different features for modeling complex relationships. The GPR method can be considered as a competitive alternative to develop parametric-PTFs.

An Automated Approach to Groundwater Quality Monitoring—Geospatial Mapping Based on Combined Application of Gaussian Process Regression and Bayesian Information Criterion

Sustainable management of the environment is based on the preservation of natural resources, first of all, freshwater—both surface and groundwater—from exhaustion and contamination. Thus, development of adequate monitoring solutions, including fast and adaptive modelling approaches, are of high importance. Recent progress in machine learning techniques provide an opportunity to improve the prediction accuracy of the spatial distribution of properties of natural objects and to automate all stages of this process to exclude uncertainties caused by handcrafting. We propose a technique to construct the weighted Water Quality Index (WQI) and the spatial prediction map of the WQI in tested area. In particular, WQI is calculated using dimensionality reduction technique (Principal Component Analysis), and spatial map of WQI is constructed using Gaussian Process Regression with automatic kernel structure selection using Bayesian Information Criterion (BIC). We validate our approach on a new dataset for groundwater quality in the New Moscow region, where groundwater is mostly used for drinking purposes. According to estimated WQI values, groundwater quality across the study region is relatively high, with few points, less than 0.5% of all observations, severely contaminated. Estimated WQIs then were used to construct spatial distribution models, GPR-BIC approach was compared with ordinary Kriging (OK), Universal Kriging (UK) with exponential, Gaussian, polynomial and periodic kernels. Quality of models was assessed using cross-validation scheme, according to which BIC-GPR approach showed better performance on average with 15% higher R2 score comparing to other Kriging models. We show that the proposed geospatial interpolation is a potentially powerful and adaptable tool for predicting the spatial distribution of properties of natural resources.

On the assessment of specific heat capacity of nanofluids for solar energy applications: Application of Gaussian process regression (GPR) approach

To characterize the performance of nanofluids for heat transfer applications in solar systems, an accurate estimation of their specific heat capacity (SHC) is of paramount importance. To this end, having such properties of nanofluids via computational approaches has gained attention as an effective method to eliminate the time-consuming process of experimental investigations. This study focuses on modeling the SHC of different carbon-based and metal oxide-based nanoparticles dispersed in various base fluids. Herein, we propose a novel data-driven dynamic model based on the Gaussian process regression (GPR) technique in comparison with the random forest (RF) approach and generalized regression neural network (GRNN) to predict the SHC of nanofluids. The developed models employ the solid volume fraction (ϕ), temperature (T), mean diameter of nanoparticle (Dp), and SHC of base fluid (CPBase) as the input parameters. The data has been collected from 10 reliable references. The results showed that the GPR model (R=0.99974, RMSE=0.01506 J/K.g) shows superior performance than the results of the RF (R=0.99761, RMSE=0.04598 J/K.g) and GRNN (R=0.99563, RMSE=0.06085 J/K.g). The results proved that the developed model would accurately estimate the SHC of the studied nanofluids. In addition, the sensitivity analysis of the dependence of input variables on the SHC of nanofluids revealed that the mean diameter of nanoparticles and the SHC of base fluid are the major critical factors in the determination of SHC of nanofluids.

Accurate determination of permeability in carbonate reservoirs using Gaussian Process Regression

The knowledge of reservoir permeability determination is increasingly important in petroleum engineering. Understanding the alteration of the abovementioned parameter as a dynamic characteristic of formations helps the characterization of reservoir rock. The existing models available for permeability determination are mainly developed for sandstone reservoir; therefore, a great challenge is in the prediction of permeability in carbonate heterogeneous rock. The current study addresses the application of Gaussian Process Regression (GPR), a state-of-the-art machine learning algorithm, in estimating permeability of carbonate reservoirs. This Bayesian network offers various benefits including non-linearity and multi-dimensionality. A widespread source of data pertinent to four deposits of Vuktyl'skiy, central Asia, Kuybyhev, and Orenburg was adopted from literature. The data set includes water saturation, effective porosity, and pore specific surface area parameters in association with absolute rock permeability as the target. Four different Kernels of Matérn, rational quadratic, exponential, and squared exponential are utilized as the covariance functions of the GPR network. A wide variety of the visualizations and statistical parameters are prepared to assess the potential of the established models in permeability estimation. Finally, it is demonstrated that the GPR (Matérn) gives the highest precision with Mean Magnitude Relative Error (MMRE) and Adjusted R-squared equal to 38% and 0.98, respectively. Besides, the applied sensitivity analysis reveals that porosity and irreducible water saturation have the lowest and the highest absolute impact values on permeability estimation. The applicability and validity of the developed models are also demonstrated by means of the so-called Williams' method for outliers detection. At last, it is worthwhile mentioning that the created new methods in this study can be good nominees for permeability determination in characterizing the heterogeneous carbonate reservoirs.

Application of Gaussian process regression to plasma turbulent transport model validation via integrated modelling

This paper outlines an approach towards improved rigour in tokamak turbulence transport model validation within integrated modelling. Gaussian process regression (GPR) techniques were applied for profile fitting during the preparation of integrated modelling simulations allowing for rigourous sensitivity tests of prescribed initial and boundary conditions as both fit and derivative uncertainties are provided. This was demonstrated by a JETTO integrated modelling simulation of the JET ITER-like-wall H-mode baseline discharge #92436 with the QuaLiKiz quasilinear turbulent transport model, which is the subject of extrapolation towards a deuterium–tritium plasma. The simulation simultaneously evaluates the time evolution of heat, particle, and momentum fluxes over ∼10 confinement times, with a simulation boundary condition at . Routine inclusion of momentum transport prediction in multi-channel flux-driven transport modelling is not standard and is facilitated here by recent developments within the QuaLiKiz model. Excellent agreement was achieved between the fitted and simulated profiles for ne, Te, Ti, and within , but the simulation underpredicts the mid-radius Ti and overpredicts the core ne and Te profiles for this discharge. Despite this, it was shown that this approach is capable of deriving reasonable inputs, including derivative quantities, to tokamak models from experimental data. Furthermore, multiple figures-of-merit were defined to quantitatively assess the agreement of integrated modelling predictions to experimental data within the GPR profile fitting framework.

First-order intrinsic Gaussian Markov random fields for discrete optimisation via simulation

ABSTRACTWe propose first-order intrinsic Gaussian Markov random fields (GMRFs) for Gaussian process (GP) regression of response surfaces, defined on a subset of the integer lattice, in operations research. These GMRFs have several desirable properties, including simple parameter estimation and no mean reversion. We focus on the application of GP regression with GMRFs to discrete optimisation via simulation (DOvS) using the Gaussian Markov improvement algorithm (GMIA). GMIA is a globally convergent GP-based algorithm for DOvS, which models the response surface as the realisation of a GMRF. The particular GMRF used in GMIA is a critical choice and influences the performance of the algorithm. We compare our first-order intrinsic GMRFs to the GMRF used in the original GMIA, provide details for parameter estimation, and discuss the global convergence of GMIA when our first-order intrinsic GMRFs are used. We then present numerical results showing the advantage of using GMIA with our first-order intrinsic GMRFs over the original GMRF.

Dynamic Pricing and Learning: An Application of Gaussian Process Regression

We consider the problem of offering an optimal price when demand is unknown and must be learned by price experimentation - a variant of the multi-armed bandit problem. In each period, the retailer must decide on a price while using knowledge already acquired to weigh the benefits of exploring other prices versus maximizing revenue. Other algorithms proposed for such problems either learn the price-demand relation for each price separately or determine the parameters of an assumed parametric demand function. We instead combine Gaussian process regression with Thompson sampling as a nonparametric learning algorithm that can learn any functional relation between price and demand. This GP-TS algorithm scales significantly better with the number of price vectors than do existing approaches, yet it makes no restrictive assumptions on the price-demand relationship's functional form. We show how to apply the algorithm to finite inventory settings that consider both single and multiple products and also to settings in which exogenous contextual information can affect prices. One advantage of the algorithm is that its performance depends not on the number of price vectors over all products but only on the number of products considered. For each setting considered here, we benchmark our approach to existing algorithms. The GP-TS algorithm's learning performance is far superior to that of its peers, especially when there are increases in the number of products to learn concurrently. Finally, we propose extensions that enable the application of our algorithm when demand follows a Bernoulli or Poisson distribution.

Fast Gaussian Process Regression for Big Data

Abstract Gaussian Processes are widely used for regression tasks. A known limitation in the application of Gaussian Processes to regression tasks is that the computation of the solution requires performing a matrix inversion. The solution also requires the storage of a large matrix in memory. These factors restrict the application of Gaussian Process regression to small and moderate size datasets. We present an algorithm that combines estimates from models developed using subsets of the data obtained in a manner similar to the bootstrap. The sample size is a critical parameter for this algorithm. Guidelines for reasonable choices of algorithm parameters, based on a detailed experimental study, are provided. Various techniques have been proposed to scale Gaussian Processes to large-scale regression tasks. The most appropriate choice depends on the problem context. The proposed method is most appropriate for problems where an additive model works well and the response depends on a small number of features. The minimax rate of convergence for such problems is attractive and we can build effective models with a small subset of the data. The Stochastic Variational Gaussian Process and the Sparse Gaussian Process are also appropriate choices for such problems. Results from experiments conducted as part of this study indicate that the algorithm presented in this work can be as effective as these methods. Unlike these methods, the proposed algorithm requires minimal hyper-parameter tuning and is much simpler to implement. The rate of convergence is also attractive.