Gaussian Process Research Articles

Gaussian Process Regression-Based Predictive Fault Location Algorithm for Three-Terminal Lines Using Two-Terminal PMUs

Gaussian Process Regression-Based Predictive Fault Location Algorithm for Three-Terminal Lines Using Two-Terminal PMUs

Neural Networks for Geospatial Data

Analysis of geospatial data has traditionally been model-based, with a mean model, customarily specified as a linear regression on the covariates, and a Gaussian process covariance model, encoding the spatial dependence. While nonlinear machine learning algorithms like neural networks are increasingly being used for spatial analysis, current approaches depart from the model-based setup and cannot explicitly incorporate spatial covariance. We propose NN-GLS, embedding neural networks directly within the traditional Gaussian process (GP) geostatistical model to accommodate nonlinear mean functions while retaining all other advantages of GP, like explicit modeling of the spatial covariance and predicting at new locations via kriging. In NN-GLS, estimation of the neural network parameters for the nonlinear mean of the Gaussian Process explicitly accounts for the spatial covariance through use of the generalized least squares (GLS) loss, thus, extending the linear case. We show that NN-GLS admits a representation as a special type of graph neural network (GNN). This connection facilitates the use of standard neural network computational techniques for irregular geospatial data, enabling novel and scalable mini-batching, backpropagation, and kriging schemes. We provide methodology to obtain uncertainty bounds for estimation and predictions from NN-GLS. Theoretically, we show that NN-GLS will be consistent for irregularly observed spatially correlated data processes. We also provide a finite sample concentration rate, which quantifies the need to accurately model the spatial covariance in neural networks for dependent data. To our knowledge, these are the first large-sample results for any neural network algorithm for irregular spatial data. We demonstrate the methodology through numerous simulations and an application to air pollution modeling. We develop a software implementation of NN-GLS in the Python package geospaNN. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Fast Calculation of Gaussian Process Multiple-Fold Cross-Validation Residuals and their Covariances

We generalize fast Gaussian process leave-one-out formulas to multiple-fold cross-validation, highlighting in turn the covariance structure of cross-validation residuals in simple and universal kriging frameworks. We illustrate how resulting covariances affect model diagnostics. We further establish in the case of noiseless observations that correcting for covariances between residuals in cross-validation-based estimation of the scale parameter leads back to maximum likelihood estimation. Also, we highlight in broader settings how differences between pseudo-likelihood and likelihood methods boil down to accounting or not for residual covariances. The proposed fast calculation of cross-validation residuals is implemented and benchmarked against a naive implementation, all in R. Numerical experiments highlight the substantial speed-ups that our approach enables. However, as supported by a discussion on main drivers of computational costs and by a numerical benchmark, speed-ups steeply decline as the number of folds (say, all sharing the same size) decreases. An application to a contaminant localization test case illustrates that the way of grouping observations in folds may affect model assessment and parameter fitting compared to leave-one-out. Overall, our results enable fast multiple-fold cross-validation, have consequences in model diagnostics, and pave the way to future work on hyperparameter fitting as well as on goal-oriented fold design. Supplementary materials for this article are available online.

Rational Kriging

This article proposes a new kriging that has a rational form. It is shown that the generalized least squares estimator of the mean from rational kriging is much more well behaved than that of ordinary kriging. Parameter estimation and uncertainty quantification for rational kriging are proposed using a Gaussian process framework. A generalized version of rational kriging is also proposed, which includes ordinary and rational kriging as special cases. Extensive simulations carried out over a wide class of functions show that the generalized rational kriging performs on par or better than both ordinary and rational kriging in terms of prediction and uncertainty quantification. The only extra step needed for generalized rational kriging over ordinary kriging is the computation of Perron eigenvector of an augmented correlation matrix which can be computed in near linear time and therefore, its overall computational complexity is no more than that of ordinary kriging. The potential applications of the new kriging methods in the emulation of computationally expensive models and model calibration problems are illustrated with real and simulated examples. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

SmashGP: Large-Scale Spatial Modeling via Matrix-Free Gaussian Processes

Gaussian processes are essential for spatial data analysis. Not only do they allow the prediction of unknown values, but they also allow for uncertainty quantification. However, in the era of big data, directly using Gaussian processes has become computationally infeasible as cubic run times are required for dense matrix decomposition and inversion. Various alternatives have been proposed to reduce the computational burden of directly fitting Gaussian processes. These alternatives rely on assumptions on the underlying structure of the covariance or precision matrices, such as sparsity or low-rank. In contrast, this article uses hierarchical matrices and matrix-free methods to enable the computation of Gaussian processes for large spatial datasets by exploiting the underlying kernel properties. The proposed framework, smashGP, represents the covariance matrix as an H 2 matrix in O ( n ) time and is able to estimate the unknown parameters of the model and predict the values of spatial observations at unobserved locations in O ( n log n ) time thanks to fast matrix-vector products. Additionally, it can be parallelized to take full advantage of shared-memory computing environments. With simulations and case studies, we illustrate the advantage of smashGP to model large-scale spatial datasets. Supplementary materials for this article are available online.

Pre-owned housing price index forecasts using Gaussian process regressions

Purpose The purpose of this study is to make property price forecasts for the Chinese housing market that has grown rapidly in the last 10 years, which is an important concern for both government and investors. Design/methodology/approach This study examines Gaussian process regressions with different kernels and basis functions for monthly pre-owned housing price index estimates for ten major Chinese cities from March 2012 to May 2020. The authors do this by using Bayesian optimizations and cross-validation. Findings The ten price indices from June 2019 to May 2020 are accurately predicted out-of-sample by the established models, which have relative root mean square errors ranging from 0.0458% to 0.3035% and correlation coefficients ranging from 93.9160% to 99.9653%. Originality/value The results might be applied separately or in conjunction with other forecasts to develop hypotheses regarding the patterns in the pre-owned residential real estate price index and conduct further policy research.

Genetic association mapping leveraging Gaussian processes.

Gaussian processes (GPs) are a powerful and useful approach for modelling nonlinear phenomena in various scientific fields, including genomics and genetics. This review focuses on the application of GPs in genetic association mapping. The aim is to identify genetic variants that alter gene regulation along continuous cellular states at the molecular level, as well as disease susceptibility over time and space at the population level. The challenges and opportunities in this field are also addressed.

A Bayesian optimization approach for data‐driven mixed‐integer nonlinear programming problems

AbstractThe optimization of process systems within the field of Chemical Engineering often confronts uncertainties that can exert significant influences on the performance and dependability of the obtained solutions. This research endeavors to investigate the application of Bayesian optimization in the realm of constrained mixed integer nonlinear problems. A comparative analysis was conducted, exploring different surrogate models, and evaluating diverse kernel functions and acquisition functions. Furthermore, a sampling strategy was devised to assess the enhancement achieved by the acquisition function. The findings of this article reveal the superior performance of sparse Gaussian processes in conjunction with computationally inexpensive acquisition functions, thereby highlighting their suitability for addressing mixed integer nonlinear programming problems characterized by noisy functions and stochastic behavior. Consequently, this article presents a computationally efficient approach to effectively tackle the challenges associated with data‐driven mixed integer nonlinear programming problems within the domain of Process System Engineering.

Impact of Solder Voids on IGBT Thermal Behavior: A Multi-Methodological Approach

This study investigates the thermal behavior of Insulated Gate Bipolar Transistors (IGBTs) with a focus on the influence of solder voids within the device. Utilizing a combination of Finite Element Method (FEM) simulations, X-ray imaging, and SEM-EDX analysis, we accurately modeled the internal structure of IGBTs to assess the impact of void characteristics on thermal resistance. The findings reveal that the presence and characteristics of solder voids—particularly their size, number, and distribution—significantly affect the thermal resistance of IGBT devices. Experimental measurements validate the FEM model’s accuracy, confirming that voids disrupt the heat flow path, which can lead to increased thermal resistance and potential device failure. Five regression models, including Gaussian process regression (GPR) and neural networks, were employed to predict the thermal resistance based on void characteristics, with the GPR models demonstrating superior performance. The optimal GPR RQ model consistently provided accurate predictions with an RMSE of 0.0050 and R2 of 0.9728. Using the void percentage as the only input parameter for the regression models significantly impacted the prediction accuracy, showing the importance of the void extraction method. This study shows the necessity of minimizing solder voids and offers a robust methodological framework for a better prediction of the reliability of IGBTs.

Simulation algorithms for stationary sequences with distributions in the form of a mixture of Gaussian distributions

Abstract In this paper, we present three algorithms for simulation of intervals of stationary vector and scalar sequences with partial distributions of their subsequences of fixed length in the form of a mixture of Gaussian distributions. The first algorithm is based on superposition of two Gaussian vector processes and the second and third ones use the method of conditional distributions and the method of superpositions to simulate the mixtures and to select realizations for approximate construction of conditional realizations. Some properties of these algorithms are presented.

A Physics-based Model Assisted by Machine-Learning for Sodium-ion Batteries with both Liquid and Solid Electrolytes

In the absence of experimental data of fully developed hierarchical 3D sodium solid-state batteries, we developed an improved continuum model by relying on Machine Learning-assisted parameter fitting to uncover the intrinsic material properties that can be transferred into different battery models. The electrochemical system simulated has sodium metal P2-type Na2/3[Ni1/3Fe1/12Mn7/12]O2 (NNFMO) as the cathode material, paired with two types of electrolytes viz, the organic liquid electrolyte and a solid polymer electrolyte. We implemented a 1D continuum model in COMSOL to suit both liquid and solid electrolytes, then used a Gaussian Process Regressor to fit and evaluate the electrochemical parameters in both battery systems. To enhance the generalizability of our model, the liquid cell and solid cell models share the same OCV input for the cathode materials. The resulting parameters are well aligned with their physical meaning and literature values. The continuum model is then used to understand the effect of increasing the thickness of the cathode and current density by analyzing the cathode utilization, and the overpotentials arising from transport and charge transfer. This 1D model and the parameter set are ready to be used in a 3D battery architecture design.

Energy loss and efficiency investigations of a dual clutch transmission based on adaptive experiment and analytical simulation

Energy transfer improvement and energy consumption reduction are getting more crucial in the automotive sector. As one of the essential elements directly influencing energy transfer in the powertrain system, transmission losses have been widely analyzed with both simulation and experimental approaches. For a better characterization of individual losses, this paper investigates and compares most of the available analytical models for further system implementation. Regarding laboratory experiments, an adaptive sampling strategy is presented and analyzed for various operating conditions. Compared to the traditional factorial design, the applied Gaussian Process Regression (GPR) delivers both estimations and their predictive variances in the entire experiment process. Besides, new measurement candidates can be determined with the prior information from GPR model. The analytical simulation and adaptive measurement are employed to investigate a seven-speed transmission. With both approaches, a detailed discussion about total transmission losses and individual loss mechanisms are performed under various operating conditions.

Pressure Interpolation in Water Distribution Networks by Using Gaussian Processes: Application to Leak Diagnosis

This work presents the reconstruction of the pressure head map of a water distribution system (WDS). This approach relies on historical data collected from a reduced number of sensors placed at some nodes of the WDS. Thus, a Gaussian regression process is then applied to estimate the pressure head at those nodes without a sensor, which allows the reconstruction of the pressure map for the whole network. Then, for leak diagnosis purposes, a dataset of pressure head maps of the WDN is created considering leaky scenarios, and a correlation method is applied to estimate the leak location. Then, for clarity, the Hanoi network is used to evaluate the performance of this leak diagnosis strategy in a simulation environment, assuming the availability of only three sensors. The results showed the potential for pressure head map reconstruction and leak localization.

Adaptive Robust Control of Uncertain Euler-Lagrange Systems Using Gaussian Processes.

This article proposes a novel adaptive robust control approach based on Gaussian processes (GPs) for the high-precision tracking problem of uncertain Euler-Lagrange (EL) systems with time-varying external disturbances. Given a prior dynamic model, the GP regression (GPR) technique is employed to obtain a nonparametric data-based uncertainty model, including its probabilistic confidence intervals. Based on the adaptive sliding mode control (ASMC) framework, the posterior means of GPs are utilized for dynamic compensation, whereas the posterior variances are applied to adjust the feedback gains. This proposed control strategy is robust against significant system uncertainty with low feedback gains. A novel adaptive law for updating hyperparameters based on tracking error feedback is presented, thereby improving the performance of both tracking control and GP modeling simultaneously. Compared to existing likelihood-based optimization methods, this hyperparameter adaptive law enables data-efficient and fast uncertainty learning for control applications. The proposed control strategy guarantees the semiglobal asymptotic convergence to zero tracking error with a specified probability. Simulations using an underwater robot model demonstrate that the utilization of GPs and hyperparameter adaptive law significantly improves the performance of tracking control and uncertainty learning.

High Performance Model Predictive Control for PMSM System Using Bayesian Ascent and Gaussian Process

High Performance Model Predictive Control for PMSM System Using Bayesian Ascent and Gaussian Process

Attitude Takeover Control for Noncooperative Space Targets Based on Gaussian Processes With Online Model Learning

Attitude Takeover Control for Noncooperative Space Targets Based on Gaussian Processes With Online Model Learning

Compressing spectral kernels in Gaussian Process: Enhanced generalization and interpretability

The modeling capabilities of a Gaussian Process (GP), such as generalization, nonlinearity, and smoothness, are largely determined by the choice of its kernel. A popular family of kernels for GPs, the spectral mixture (SM) kernels, have the desirable property that with a large number of spectral components they can approximate any stationary kernel. However, using a large number of SM components increases the risk of overfitting and hinders interpretability. To overcome these challenges, we propose a compression algorithm incorporating component pruning and component merging for GPs. Here SM components with small signal variance are removed, and a moment-matching merge method is proposed to further reduce the number of SM components. The main novelty of the proposed method is a similarity measure between SM components based on their normalized cross-correlation, which is related to the Bhattacharyya coefficient. We derive a greedy GP compression algorithm and perform a comparative evaluation over various learning tasks in terms of forecasting performance and compression capability. Results substantiate the beneficial effect of the method, both in terms of generalization and interpretability.11Source code:https://github.com/ck2019ML/Gaussian-Process-Model-Compression.

Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling.

In this paper, we propose FGPR: a Federated Gaussian process ( GP) regression framework that uses an averaging strategy for model aggregation and stochastic gradient descent for local computations. Notably, the resulting global model excels in personalization as FGPR jointly learns a shared prior across all devices. The predictive posterior is then obtained by exploiting this shared prior and conditioning on local data, which encodes personalized features from a specific dataset. Theoretically, we show that FGPR converges to a critical point of the full log-marginal likelihood function, subject to statistical errors. This result offers standalone value as it brings federated learning theoretical results to correlated paradigms. Through extensive case studies on several regression tasks, we show that FGPR excels in a wide range of applications and is a promising approach for privacy-preserving multi-fidelity data modeling.

An efficient machine learning-based model for predicting the stress-strain relationships of thermoplastic polymers with limited testing data

Thermoplastic polymers used in aeronautical structures such as poly-ether-ether-ketone (PEEK) usually exhibit nonlinear stress-strain relationships, which can be usually predicted by the physical and phenomenological models. For different thermoplastic polymers, however, existing models may encounter difficulties in reasonably predicting the stress-strain relationships. By reasonably using experimental data, the machine learning-based model can accurately predict the stress-strain relationships. In this paper, an efficient machine learning-based model is built to predict the stress-strain relationships of thermoplastic polymers by using the Kriging model, where limited testing data are merely used. Experiments of 69 specimens for PEEK are firstly performed under uniaxial tensile, where the temperatures range from 23 °C to 140 °C and the strain rates range from 10−3 s−1 to 10−1 s−1. Genetic algorithm is employed to train the proposed model based on the stress-strain relationships obtained from experiments. Moreover, the results predicted by the proposed Kriging-based model are compared with those obtained from the DSGZ (Duan-Saigal-Greif-Zimmerman) model. The results indicate that the Kriging-based model possesses better accuracy, lower complexity, and better robustness than the selected model, because the prior estimation is introduced by the assumption of Gaussian stochastic processes. In addition, the proposed model is also applied to Poly-Propylene (PP), PEEK at high temperatures, and polycarbonate (PC), it can be found that the Kriging-based model can also predict the stress-strain relationships of other amorphous and semi-crystalline thermoplastic polymers.

Robust clustering-based hybrid technique enabling reliable reservoir water quality prediction with uncertainty quantification and spatial analysis

Machine learning methodology has recently been considered a smart and reliable way to monitor water quality parameters in aquatic environments like reservoirs and lakes. This study employs both individual and hybrid-based techniques to boost the accuracy of dissolved oxygen (DO) and chlorophyll-a (Chl-a) predictions in the Wadi Dayqah Dam located in Oman. At first, an AAQ-RINKO device (CTD+ sensor) was used to collect water quality parameters from different locations and varying depths in the reservoir. Second, the dataset is segmented into homogeneous clusters based on DO and Chl-a parameters by leveraging an optimized K-means algorithm, facilitating precise estimations. Third, ten sophisticated variational-individual data-driven models, namely generalized regression neural network (GRNN), random forest (RF), gaussian process regression (GPR), decision tree (DT), least-squares boosting (LSB), bayesian ridge (BR), support vector regression (SVR), K-nearest neighbors (KNN), multilayer perceptron (MLP), and group method of data handling (GMDH) are employed to estimate DO and Chl-a concentrations. Fourth, to improve prediction accuracy, bayesian model averaging (BMA), entropy weighted (EW), and a new enhanced clustering-based hybrid technique called Entropy-ORNESS are employed to combine model outputs. The Entropy-ORNESS method incorporates a Genetic Algorithm (GA) to determine optimal weights and then combine them with EW weights. Finally, the inclusion of bootstrapping techniques introduces a stochastic assessment of model uncertainty, resulting in a robust estimator model. In the validation phase, the Entropy-ORNESS technique outperforms the independent models among the three fusion-based methods, yielding R2 values of 0.92 and 0.89 for DO and Chl-a clusters, respectively. The proposed hybrid-based methodology demonstrates reduced uncertainty compared to single data-driven models and two combination frameworks, with uncertainty levels of 0.24% and 1.16% for cluster 1 of DO and cluster 2 of Chl-a parameters. As a highlight point, the spatial analysis of DO and Chl-a concentrations exhibit similar pattern variations across varying depths of the dam according to the comparison of field measurements with the best hybrid technique, in which DO concentration values notably decrease during warmer seasons. These findings collectively underscore the potential of the upgraded weighted-based hybrid approach to provide more accurate estimations of DO and Chl-a concentrations in dynamic aquatic environments.