Gaussian Process Model Research Articles

Bayes_Opt-SWMM: A Gaussian process-based Bayesian optimization tool for real-time flood modeling with SWMM

Real-time flood model plays a pivotal role in averting urban flood damage, particularly when there is minimal lead time for preparatory measures. However, urban flood modeling in real-time often contends with inherent uncertainties arising from input data uncertainty and parameter ambiguities. This study introduces a real-time calibration (RTC) tool called Bayes_Opt-SWMM , specifically tailored for real-time urban flood modeling and uncertainty optimization. This tool leverages the Gaussian process-based Bayesian optimization algorithm and interfaces seamlessly with the Stormwater Management Model (SWMM). It integrates real-time model forcing data and flood monitoring collected through sensors and gauges which are strategically placed within critical locations of urban drainage systems. Our approach hinges on the Surrogate Model based Uncertainty Optimization (SMUO) concept, providing an avenue for enhancing real-time flood modeling. Bayes_Opt-SWMM runs the optimization process using a surrogate model called Gaussian Process emulator with two inference methods: (1) the Gaussian Process (GP) model and (2) Markov Chain Monte Carlo (MCMC) algorithm in GP model (GP_MCMC). Furthermore, three acquisition functions, namely Expected Improvement (EI), Maximum Probability of Improvement (MPI), and Lower Confidence Bound (LCB), facilitate optimal parameter fitting within the surrogate models. The efficiency of GP-based surrogate models in learning SWMM model parameters, leads to an improved uncertainty quantification and accelerated real-time flood modeling in urban areas. Overall, Bayes_Opt-SWMM emerges as a cost-effective and valuable tool for real-time flood modeling and monitoring, with significant potential for managing intelligent storm water systems in urban environments.

Research on Quantification of Structural Natural Frequency Uncertainty and Finite Element Model Updating Based on Gaussian Processes

During bridge service, material degradation and aging occur, affecting bridge functionality. Bridge health monitoring, crucial for detecting structural damage, includes finite element model modification as a key aspect. Current finite element-based model updating techniques are computationally intensive and lack practicality. Additionally, changes in loading and material property deterioration lead to parameter uncertainty in engineering structures. To enhance computational efficiency and accommodate parameter uncertainty, this study proposes a Gaussian process model-based approach for predicting structural natural frequencies and correcting finite element models. Taking a simply supported beam structure as an example, the elastic modulus and mass density of the structure are sampled by the Sobol sequence. Then, we map the collected samples to the corresponding physical space, substitute them into the finite element model, and calculate the first three natural frequencies of the model. A Gaussian surrogate model was established for the natural frequency of the structure. By analyzing the first three natural frequencies of the simply supported beam, the elastic modulus and mass density of the structure are corrected. The error between the corrected values of elastic modulus and mass density and the calculated values of the finite element model is very small. This study demonstrates that Gaussian process models can improve calculation efficiency, fulfilling the dual objectives of predicting structural natural frequencies and adjusting model parameters.

GP+: A Python library for kernel-based learning via Gaussian processes

In this paper we introduce GP+, an open-source library for kernel-based learning via Gaussian processes (GPs) which are powerful statistical models that are completely characterized by their parametric covariance and mean functions. GP+ is built on PyTorch and provides a user-friendly and object-oriented tool for probabilistic learning and inference. As we demonstrate with a host of examples, GP+ has a few unique advantages over other GP modeling libraries. We achieve these advantages primarily by integrating nonlinear manifold learning techniques with GPs’ covariance and mean functions. As part of introducing GP+, in this paper we also make methodological contributions that (1) enable probabilistic data fusion and inverse parameter estimation, and (2) equip GPs with parsimonious parametric mean functions which span mixed feature spaces that have both categorical and quantitative variables. We demonstrate the impact of these contributions in the context of Bayesian optimization, multi-fidelity modeling, sensitivity analysis, and calibration of computer models.

A Bayesian approach to assessing uncertainty in the effect of fertilization strategies on paddy rice yield via multiple on-farm experiments

ABSTRACT The surge in global agricultural input prices, particularly for nitrogen (N) fertilizers, has presented challenges for farmers facing stagnant crop prices and rising production costs. This study explored a system designed to sustain or enhance farmers’ income while reducing production costs, with a specific focus on implementing practices that involve cultivating crops with lower rates of fertilizer than conventional rates, particularly in smaller land settings, through on-farm experimentation (OFE). OFE was conducted over three years across 11 fields in Gifu, Japan, utilizing a strip trial design to investigate the effect of fertilizer input rates on rice yield. To collect yield data, unmanned aerial vehicle (UAV)-mounted multispectral cameras were deployed. A Gaussian process model was used to address the impact of fertilizer input rates on yield while accounting for spatial yield variation. The study further assessed the number of field trials achieving positive profits at certain probability levels by integrating outcomes from multiple OFEs. The results highlighted little rice yield response to fertilizer input in most of the fields, while several fields showed significant positive yield responses. Notably, the use of lower fertilizer input rates in certain fields proved to be more profitable whereas higher fertilizer input rates were less profitable in most fields. The results further emphasized the economic viability of lower rates, especially when fertilizer prices are high, thus contributing valuable insights for sustainable and profitable rice production in Japanese cultivation.

Probabilistic rutting model using NGBoost and SHAP: Incorporating other performance indicators

This study focuses on accurately predicting rutting depth (RD) through a probabilistic prediction model, which is crucial for timely road maintenance and traffic safety. Natural Gradient Boosting (NGBoost) technology is employed to achieve this goal as it provides accurate predictions and estimates the uncertainty of predictions. Furthermore, to explain the decision-making process of the proposed prediction model, Shapley Additive Explanations (SHAP) analysis is applied to identify and quantify significant factors affecting RD. Research results show that the NGBoost model surpasses the traditional Gaussian Process and Random Forest model regarding prediction accuracy and performs better regarding certainty of prediction intervals. Meanwhile, SHAP analysis improves the transparency of model decision-making and further validates the model. Explanatory ability elucidates the impact of other characteristics on RD. The two-stage probabilistic prediction model not only enhances the reliability of predictions but also provides an in-depth understanding of the key influencing factors of pavement performance, improving the efficiency and effectiveness of maintenance measures.

High-precision Lamb wave evaluation for corrosion damage with model-based Bayesian Optimization and Gaussian meta-modeling strategy

The presence of corrosion damage in metallic structures is a critical problem affecting system safety, which poses challenges for structural maintenance. Ultrasonic Lamb wave testing has shown many benefits, including high sensitivity and a large coverage area. However, multiple external factors are causing a mismatch between the imaging result and the actual damage. This study proposes a Bayesian Optimization-based model calibration framework, incorporating a Gaussian process (GP) measurement model, for accurate corrosion damage quantification. In this method, one critical problem is obtaining the high-fidelity Finite Element Method (FEM) model. The initial FEM model usually has inevitable modeling errors due to necessary simplifications and idealized constraints. Here we introduced Bayesian Optimization by which the numerical model could be calibrated. The calibrated FEM model, which had fewer discrepancies with actual experiments, could generate high-fidelity signals corresponding to various damage severity. Then, the GP measurement model outputted the mean and variance of the corrosion width and depth corresponding to the differences between experiment signals and model outputs. The inversion of damage information from the established GP model was accomplished by the genetic algorithm. Hence, both the damage size and depth could be evaluated with high accuracy. Once the corrosion information along different propagation paths was obtained, it can be combined with the location of the corrosion damage center for locating the edge points of the corrosion damage. In the end, the edge points were fitted using the least square algorithm. Validations were experimentally performed, and the detection result showed good consistency with the actual damage.

Measuring wake deflection from SCADA data during wake steering using machine learning

As wake steering is implemented on large wind plants, methods for evaluating how well these systems optimize plant performance are needed. The data analysis from experiments conducted so far generally focuses on evaluating the improvement in power or energy production of the plant and comparing that to the predicted improvement. In this study, we explore a new approach to provide additional insight and validation of optimization tools by measuring the wake deflection experimentally using only the downstream turbine as a sensor. This approach is demonstrated using the SMARTEOLE wind plant wake steering experimental data. Light gradient boosting machines (LGBM) and Gaussian process (GP) machine learning models are trained and then interrogated by making a set of predictions under defined conditions to determine the wake deflection observed at the downstream turbine. The data set is sparse, particularly for larger yaw angles, and noisy due to factors not captured in the SCADA leading to relatively high uncertainty in the predictions. The GP model is better suited to smoothly fit this data. Using the GP model, a wake deflection of 0.35 rotor diameters at 3.7 rotor diameters downwind is estimated for a yaw error of 20 degrees on the upwind turbine.

Treed Gaussian processes for animal movement modeling.

Wildlife telemetry data may be used to answer a diverse range of questions relevant to wildlife ecology and management. One challenge to modeling telemetry data is that animal movement often varies greatly in pattern over time, and current continuous-time modeling approaches to handle such nonstationarity require bespoke and often complex models that may pose barriers to practitioner implementation. We demonstrate a novel application of treed Gaussian process (TGP) modeling, a Bayesian machine learning approach that automatically captures the nonstationarity and abrupt transitions present in animal movement. The machine learning formulation of TGPs enables modeling to be nearly automated, while their Bayesian formulation allows for the derivation of movement descriptors with associated uncertainty measures. We demonstrate the use of an existing R package to implement TGPs using the familiar Markov chain Monte Carlo algorithm. We then use estimated movement trajectories to derive movement descriptors that can be compared across individuals and populations. We applied the TGP model to a case study of lesser prairie-chickens (Tympanuchus pallidicinctus) to demonstrate the benefits of TGP modeling and compared distance traveled and residence times across lesser prairie-chicken individuals and populations. For broad usability, we outline all steps necessary for practitioners to specify relevant movement descriptors (e.g., turn angles, speed, contact points) and apply TGP modeling and trajectory comparison to their own telemetry datasets. Combining the predictive power of machine learning and the statistical inference of Bayesian methods to model movement trajectories allows for the estimation of statistically comparable movement descriptors from telemetry studies. Our use of an accessible R package allows practitioners to model trajectories and estimate movement descriptors, facilitating the use of telemetry data to answer applied management questions.

Long-period modulation of the classical T Tauri star CI Tau

Context. Detecting planets within protoplanetary disks around young stars is essential for understanding planet formation and evolution. However, planet detection using the radial velocity method faces challenges due to the strong stellar activity in these early stages. Aims. We detect long-term periodicities in photometric and spectroscopic time series of the classical T Tauri star (CTTS) CI Tau, and retrieve evidence for inner embedded planets in its disk. Methods. The study conducted photometric and spectroscopic analyses using K2 and Las Cumbres Observatory Global Network light curves, and high-resolution spectra from ESPaDOnS and SPIRou. We focused our radial velocity analysis on a wavelength domain less affected by spot activity. To account for spot effects, a quasi-periodic Gaussian process model was applied to K2 light curve, ESPaDOnS, and SPIRou radial velocity data. Additionally, a detailed bisector analysis on cross-correlation functions was carried out to understand the cause of long-term periodicity. Results. We detect coherent periods at ~6.6d, 9d, ~11.5d, ~14.2d, and ~25.2d, the last of which is seen consistently across all datasets. Bisector analysis of the cross-correlation functions provides strong hints for combined activity-induced and Doppler reflex signals in the radial velocities at a period of 25.2 d. Our analysis suggests that this periodicity is best explained by the presence of a 3.6 ± 0.3 MJup eccentric (e ~ 0.58) planet at a semi-major axis of 0.17 au. Conclusions. We report the detection of a massive inner planet in CI Tau. Our study outlines the difficulty of searching for disk-embedded planets in the inner 0.1 au of young and active systems. When searching for planets in actively accreting stars such as CI Tau, we demonstrate that the primary limitation is stellar activity rather than the precision of RV measurements provided by the instrument.

A Global Paleosecular Variation Database for the Paleogene: Stationary Secular Variation Behavior Since the Triassic?

AbstractPaleosecular variation analysis is a primary tool for characterizing ancient geomagnetic behavior and its evolution through time. This study presents a new high‐quality directional data set, paleosecular variation of the Paleogene (PSVP), with and without correction for serial correlation, compiled from 1,667 sites from 45 different localities from the Paleogene and late Cretaceous (84–23 Ma). The data set is used to study the variability, structure, and latitude dependence of the geomagnetic field during that period by varying selection criteria and PSV models. Modeled values for the equatorial virtual geomagnetic pole (VGP) dispersion have over‐lapping uncertainty intervals within their uncertainty bounds between 8.3° and 18.6° for the past 250 Ma. We investigate the suitability of two descriptive models of PSV, Model G‐style quadratic fits and covariant Giant Gaussian Process models, and find that both styles of model fail to satisfactorily reproduce the latitude dependent morphology of PSV, but suggest that estimates of the equatorial VGP dispersion may still robustly characterize aspects of Earth's long‐term field morphology. During this time where the PSV behavior has not changed substantially, the reversal frequency has varied widely. The lack of a clear relationship between PSV behavior and reversal frequency is not trivially explained in the context of published findings regarding numerical geodynamo simulations.

Adaptive Robust Control of Uncertain Euler-Lagrange Systems Using Gaussian Processes.

This article proposes a novel adaptive robust control approach based on Gaussian processes (GPs) for the high-precision tracking problem of uncertain Euler-Lagrange (EL) systems with time-varying external disturbances. Given a prior dynamic model, the GP regression (GPR) technique is employed to obtain a nonparametric data-based uncertainty model, including its probabilistic confidence intervals. Based on the adaptive sliding mode control (ASMC) framework, the posterior means of GPs are utilized for dynamic compensation, whereas the posterior variances are applied to adjust the feedback gains. This proposed control strategy is robust against significant system uncertainty with low feedback gains. A novel adaptive law for updating hyperparameters based on tracking error feedback is presented, thereby improving the performance of both tracking control and GP modeling simultaneously. Compared to existing likelihood-based optimization methods, this hyperparameter adaptive law enables data-efficient and fast uncertainty learning for control applications. The proposed control strategy guarantees the semiglobal asymptotic convergence to zero tracking error with a specified probability. Simulations using an underwater robot model demonstrate that the utilization of GPs and hyperparameter adaptive law significantly improves the performance of tracking control and uncertainty learning.

Conglomerate Multi-fidelity Gaussian Process Modeling, with Application to Heavy-Ion Collisions

Conglomerate Multi-fidelity Gaussian Process Modeling, with Application to Heavy-Ion Collisions

Category tree Gaussian process for computer experiments with many-category qualitative factors and application to cooling system design

In computer experiments, Gaussian process (GP) models are commonly used for emulation. However, when both qualitative and quantitative factors are in the experiments, emulation using GP models becomes challenging. In particular, when the qualitative factors contain many categories in the experiments, existing methods in the literature become cumbersome due to the curse of dimensionality. Motivated by the computer experiments for the design of a cooling system, a new tree-based GP is proposed that emulates computer models with many-category qualitative factors, which we call category tree GP. The proposed method incorporates a tree structure to split the categories of the qualitative factors, and GP or mixed-input GP models are employed for modeling the simulation outputs in the leaf nodes. The splitting rule takes into account the cross-correlations between the categories of the qualitative factors, which have been shown by a recent theoretical study to be a crucial element for improving the prediction accuracy. In addition, a pruning procedure based on the cross-validation error is proposed to ensure the prediction accuracy. The application to the design of a cooling system indicates that the proposed method not only enjoys marked computational advantages and produces accurate predictions, but also provides valuable insights into the cooling system by discovering the tree structure.

Multifidelity-based Gaussian process for quasi-site-specific probabilistic prediction of soil properties

Conventional empirical equations for soil properties prediction tend to be site-specific, exhibiting poor reliability and accuracy. Meanwhile, alternative data-driven methods require large datasets for training. To address these issues, this study proposed a novel multifidelity residual neural-network-based Gaussian process (MR-NNGP) modelling framework. A soil property low-fidelity (LF) prediction model is first trained using abundant LF data collected from worldwide sites for generating preliminary estimation. A high-fidelity (HF) model is subsequently trained on sparse HF data from the specific site of interest for calibrating the LF model to make quasi-site-specific predictions. An infinitely wide NN-inspired NNGP is adopted as the baseline algorithm for training LF and HF models. The compression index of clays is selected as an example to examine the capability of the proposed MR-NNGP. The results indicate that the compression index of clays can be well captured by MR-NNGP, exhibiting superior accuracy and reliability compared with one-shot training without using MR modelling and other baseline algorithms such as GP. The MR-NNGP framework alleviates data dependency and improves model performance through hierarchical modelling on relatively simple correlations using a superior algorithm. Unified LF data and efficient hyper-parameter optimization indicate the flexibility for broader applications in various sites worldwide.

Gaussian processes modeling for the prediction of polymeric nanoparticle formulation design to enhance encapsulation efficiency and therapeutic efficacy.

Conventional drugs have been facing various drug delivery obstacles, including first-pass metabolism for oral medications, drug degradation by cellular enzymes, off-target effects, and cytotoxicity of healthy cells. Nanoparticles (NP) application in drug delivery can compensate for these drawbacks to a great extent. NPs can be fabricated using different materials and structures to achieve desired therapeutic effects. For each type of NP material, its physicochemical properties determine compatibility with specific drugs and other supplemental compositions. The optimized material selection becomes prominent in NP development to improve NP performances. Due to the nature of NP fabrication, the process is long and expensive. To accelerate NP composition optimization, machine learning (ML) techniques are among the most promising methods for efficient data predictions and optimizations.As a proof-of concept, we created Gaussian Process (GP) models to make predictions for drug encapsulation efficiency (EE%) and therapeutic efficacy of 32 poly (lactic-co-glycolic acid) (PLGA) NPs that are formed with materials with different physicochemical properties. Two model drugs, doxorubicin (DOX) and docetaxel (DTX) were loaded separately. The IC50 values for the various NPs formulations were evaluated using the OVCAR3 epithelial ovarian cancer cell line. EE% GP model has the highest prediction accuracy with the lowest normalized root-mean-squared-error (RMSE) of 0.187. The DOX and DTX IC50 GP models have normalized RMSEs of 0.296 and 0.206, respectively, which are higher than that of the EE% GP model.

Construction of orthogonal-MaxPro Latin hypercube designs

Orthogonal Latin hypercube designs (LHDs) and maximum projection (MaxPro) LHDs are widely used in computer experiments. They are efficient for estimating the trend part and the Gaussian process part of the universal Kriging (i.e., the Gaussian process) model, respectively, especially when only some of the factors are active. Yet, the orthogonality and the MaxPro criteria often do not agree with each other. In this work, we propose a new class of optimal designs, called orthogonal-MaxPro LHDs, optimizing a well-defined multi-objective criterion combining the correlation and the MaxPro metrics. An efficient parallel algorithm via level permutations and expansions is developed, whose efficiency is guaranteed by theories. Numerical results are presented to show that the construction is fast and the obtained designs are attractive, especially for large computer experiments.

Parametric Dynamic Simulation and Bayesian Design Optimization of a Front-Loading Washing Machine

Abstract Objective This paper presents the dynamic modeling and design optimization of a fully parametrized front-loading washing machine. Methods A thorough mathematical analysis is performed to capture the effect of nine design variables (including rotation speed, spring stiffness, weight balancer mass density, damping coefficient, spring and damper angles) on vibrational characteristics of the washing machine. The parametric simulation reveals a complex relationship between the vibrational dynamics and design variables, entailing a computationally efficient optimization approach. A Bayesian Optimization (BO) framework is developed, in which a Gaussian process model and Genetic Algorithm (GA) are utilized along with carefully selected acquisition functions to enable adaptive sampling and search for optimal design values. The objective function is to minimize the maximum amplitude of the tub displacement inside the washing machine body. Two case studies are performed to consider a different number of design variables and choices of various infill methods. Results Results show that the fully parametrized front-loading washing machine model is applicable for parametric simulation and design optimization. The proposed BO is able to successfully find a set of design variable values corresponding to the lowest maximum displacement for vibration reduction. A comparison analysis is also carried out to exhibit the difference in optimization convergence and computational cost caused by the choice of acquisition functions. Conclusion It is shown that the fully parametrized washing machine model yields a lower maximum displacement than the model with fewer design variables. Of the two acquisition functions studied for optimization, the Expected Improvement function converges more rapidly than the Lower Bound criterion.

Online Bayesian Optimization of Polynomial-Multigrid Cycles for Flux Reconstruction

In this study, we present a novel strategy for dynamically optimizing polynomial multigrid cycles to accelerate convergence within the dual-time-stepping formulation of the artificial compressibility method. To accomplish this, a Gaussian process model is developed using Bayesian optimization to efficiently sample possible cycles to minimize run-time. To allow the use of conventional optimization methods, we developed fractional smoothing steps, moving the optimization from a discrete space to a continuous space. Initially, a static, offline, approach was developed, and optimal cycles were found for two flow past cylinder test cases with Re=200 and Re=500; however, when exchanging optimal cycles between the different test cases, there was significant degradation in speedup. Toward this, a dynamic, online, approach was developed where cycles are optimized during a simulation. The performance of the resulting optimal cycles gave a similar speedup to the offline approach while achieving a net reduction in run-time. Again testing the optimization strategy on the flow past a cylinder, this yielded candidates with mean speedups of ∼3.0× and ∼2.1×, respectively. Finally, testing online optimization on a turbulent flow past a cylinder at Re=3900 resulted in an overall speedup of ∼1.9×.

Model based data-driven optimisation of automated fibre placement set up

The composites manufacturing industry relies heavily on manual hand layup, unsuitable for increasing demands. Automated Fiber Placement (AFP) offers a solution, but setup complexity and extensive trials inflate costs. Digital tools promise to expedite development, but CPU-intensive simulations limit large-scale parameter optimisation. This paper introduces Gaussian Process (GP) models for understanding AFP parameters’ relationship with nip-point temperature. The GP emulator streamlines optimisation, offering accurate power curves with minimal simulations. An efficient method to enhance emulator performance through sequential design is also proposed, providing a cost-effective decision tool for AFP optimisation.

Predicting the fundamental fluxes of an eddy-covariance station using machine learning methods

Monitoring tools are needed to maximise living systems' ability to mitigate emissions and adapt to changing environmental conditions. Therefore, it is important to be able to predict the fundamental fluxes in crops, in this case vineyards, such as sensible heat flux (H), latent heat flux (LE) and carbon dioxide flux (CO2), in order to know their capacity to adapt to the environmental effects of climate change. In this study, Linear Regression (LR), Elastic Net (EN) regression, K-Nearest Neighbours (KNN), Gaussian-Process (GP), Decision Tree (TREE) Regression, Random Forest (RF) Regression, XGBoost (XGB) Regression, Support Vector Regression (SVR) and Multi-layer Perceptron (MLP) models have been applied to predict fundamental fluxes of an eddy-covariance station from conventional meteorological parameters. These models reproduced well the estimations of three output parameters from the eddy-covariance station. The performance of each predictive model was evaluated using Root-Mean-Squared Error (RMSE), Mean Absolute Error (MAE), Mean Square Error (MSE), Mean Absolute Percentage Error (MAPE), Mean Absolute Scaled Error (MASE) and the coefficient of determination (R2). The findings indicate that for the variable H, the GP model outperformed the SVR and all other models, achieving an R2 value of 0.99. Conversely, the SVR demonstrated superior performance for the variables LE and CO2, with R2 values of 0.96 for both. In summary, these findings suggest that the three models proposed show a robust performance in the prediction of the studied fluxes, underlining their versatility and adaptability to the various environmental conditions of the vineyard.