Gaussian Process Model Research Articles

A study of the role of data and model uncertainty in active learning

Uncertainty-based active learning strategies have demonstrated significant superiority in small data research of materials domain. This study explores the effects of model uncertainty and data uncertainty separately on the performance of active learning strategies, specifically focusing on the number of iterations required to identify the optimal samples. For model uncertainty, three kinds of acquisition functions are compared, including predicted value strategy (PV), ranking of predicted value strategy (PR) and expected improvement strategy (EI). Among these, the active learning model utilizing PR requires the fewest average iterations (1.75). For data uncertainty, we evaluate the iterations of active learning by Gaussian process models that incorporate the uncertainty of the observations and noise samples that takes account into the uncertainty of the input features respectively. The results indicate that the active learning iterations of the three strategies converge to similar at the optimal weighting when the uncertainty of the observations is considered in the model (EI for 1.75, PV for 1.21 and PR for 1.18). In contrast, incorporating noise samples into the augmented dataset after the original samples would severely deteriorate the efficiency of active learning recommendations. Our findings aim to offer guidance for exploring more favorable acquisition functions and methods for active learning strategies.

Robust Health Condition Prediction of Helicopter Turboshaft Engines Using Ensemble Machine Learning Models

This paper presents a novel ensemble approach that combines multiple machine-learning algorithms to deliver robust predictions of helicopter turboshaft engine health status (nominal or faulty) using operational data. Engine health is evaluated through the torque margin, defined as the percentage difference between the measured and target torque values. A Gaussian process model is used to estimate the torque margin as a probability distribution function (PDF), and these predictions are incorporated as features into various machine-learning models. These models are then employed to perform binary classification, determining the engine's health state. To enhance performance, a reference set is defined for each unseen data point, allowing a comparison of the relative performances of the models, with the best performer selected for the final prediction. Our ensemble method achieves high accuracy in health classification while providing precise torque margin estimates. The results demonstrate that ensemble models offer superior generalization and reliability compared to individual machine-learning algorithms, especially when applied to complex, multivariate datasets like those from helicopter turboshaft engines.

LiDAR-Based Modeling of Individual Tree Height to Crown Base in Picea crassifolia Kom. in Northern China: Comparing Bayesian, Gaussian Process, and Random Forest Approaches

This study compared hierarchical Bayesian, mixed-effects Gaussian process regression, and random forest models for predicting height to crown base (HCB) in Qinghai spruce (Picea crassifolia Kom.) forests using LiDAR-derived data. Both modeling approaches were applied to a dataset of 510 trees from 16 plots in northern China. The models incorporated tree-level variables (height, diameter at breast height, crown projection area) and plot-level spatial competition indices. Model performance was evaluated using leave-one-plot-out cross-validation. The Gaussian mixed-effects process model (with an RMSE of 1.59 and MAE of 1.25) slightly outperformed the hierarchical Bayesian model and the random forest model. Both models identified LiDAR-derived tree height, DBH, and LiDAR-derived crown projection area as primary factors influencing HCB. The spatial competition index (SCI) emerged as the most effective random effect, with the lowest AIC and BIC values, highlighting the importance of local competition dynamics in HCB formation. Uncertainty analysis revealed consistent patterns across the predicted values, with an average relative uncertainty of 33.89% for the Gaussian process model. These findings provide valuable insights for forest management and suggest that incorporating spatial competition indices can enhance HCB predictions.

Trajectory tracking for autonomous surface ships using Gaussian process regression and model predictive control with BVS strategy

Autonomous navigation is critical to the development of next-generation shipping systems. The proposal of intelligent ships enables innovation in the shipping and shipbuilding industry and increases the safety and efficiency of ship operations. Autonomous control of surface ships to follow a prescribed trajectory is critical in a variety of maritime applications. This paper proposes a trajectory tracking control strategy for autonomous surface ships that combines nonparametric modelling using Gaussian Process Regression (GPR) with the Model Predictive Control (MPC) framework. A Bézier curve-based Virtual Ship(BVS) guidance strategy is proposed to convert dynamic trajectory points into reference heading angles and speeds, such that the trajectory tracking problem can be decomposed into heading control and speed control problems. Gaussian process regression is utilised to identify the correlation between propeller revolution speed and ship speed, as well as the correlation between rudder angle and heading angle based on experimental data. Two GPR models are therefore constructed as the prediction models for designing MPC controllers for heading control and speed control, respectively. Nonlinear optimisation algorithms are utilised to search for optimal control commands in each sampling interval to solve the optimisation problem in MPC with GPR models and input constraints. Simulations are carried out to evaluate the effectiveness of the proposed method.

Artificial Neural Network and Gaussian Approach to Predict Rotor-Airframe Acoustic Waveforms

An artificial neural network-based surrogate model and Gaussian process model were developed to predict the acoustic interaction for a fixed-pitch rotor in proximity to a downstream cylindrical airframe typical of small unmanned aerial system platforms. The models were trained to predict the acoustic waveform under representative hover conditions as a function of rotational speed, airframe proximity, and observer angle. Training data were acquired in an anechoic chamber on both isolated rotors and rotor–airframe configurations. The acoustic amplitude and phase of the revolution-averaged interaction were predicted, which required up to 25 harmonics to capture the impulse event caused by the blade’s approach and departure from the airframe. Prediction performance showed, on average, that the artificial neural network models could estimate the acoustic amplitude and phase over the relevant harmonics for unseen conditions with 86% and 75% accuracy, respectively. This enables a time-domain reconstruction of the waveform for the range of geometric and flow parameters tested. In contrast, the Gaussian process matched the amplitude but underpredicted the phase for unseen conditions at 86% and 45% accuracy, respectively.

Evaluation method of carbon-fiber-reinforced polymer material anti-radiation performance based on synergistic effect model

Aiming at the high computational costs of the current multi-source X-ray radiation numerical simulations, a multi-source X-ray evaluation method based on a synergistic effect model is studied. First, based on the advantage of the low computational cost of single-source X-ray radiation numerical simulations, a hierarchical Gaussian process model is used to construct a superimposed single-source numerical simulation data fusion model. Second, by constructing composite correlation functions, the data fusion ability of the Gaussian process model is improved. The unknown parameters in the data fusion evaluation model are solved with the help of the Markov chain Monte Carlo method and a nonlinear optimization algorithm. Based on the obtained sampling values of the unknown parameters, an approximate solution method for the estimated value of the superimposed single-source X-ray radiation numerical simulations with unknown input conditions is given. Then a synergistic effect model is constructed based on the established linear regression surrogate model. The synergistic evaluation of multi-source X-ray radiation tests based on single-source X-ray radiation numerical simulation superimposed test data is studied. Finally, carbon-fiber-reinforced polymer experiments show the proposed method in better than existing Kriging methods, for prediction of multi-source X-ray radiation data.

Detection of E. coli using bacteriophage T7 and analysis of excitation‑emission matrix fluorescence spectroscopy

Conventional detection methods require the isolation and enrichment of bacteria, followed by molecular, biochemical, or culture-based analysis. To address some of the limitations of conventional methods, this study develops a machine learning (ML) approach to analyze the excitation-emission matrix (EEM) fluorescence data generated based on bacteriophage T7 and E. coli interactions for in-situ detection of live bacteria in the presence of fresh produce homogenate. We trained classification models using various ML algorithms based on the 3-D EEM data generated with bacteria and their interactions with a T7 phage. These ML algorithms, including linear Support Vector Classifier and Random Forest, demonstrate high accuracy (>0.85) for detecting E. coli at 102 CFU/ml concentration within 6 hours. Additionally, these ML models can differentiate among different E. coli concentration levels. For example, the Gaussian Process model achieved an accuracy of 92% in detecting different concentration levels of live E. coli. Application of these ML methods to detect E. coli in spinach homogenate yielded an accuracy of 89% using the linear-SVC model. Furthermore, feature selection techniques were employed to reduce the dimensionality of the data, revealing that only six features were necessary for achieving classification accuracy (>0.85) of spinach homogenate samples containing 102 CFU/ml of E. coli. These findings highlight the potential of this novel bacterial detection methodologies, offering rapid, specific, and efficient solutions for applications in food safety and environmental monitoring.

Optimizing Thermoplastic Starch Film with Heteroscedastic Gaussian Processes in Bayesian Experimental Design Framework

The development of thermoplastic starch (TPS) films is crucial for fabricating sustainable and compostable plastics with desirable mechanical properties. However, traditional design of experiments (DOE) methods used in TPS development are often inefficient. They require extensive time and resources while frequently failing to identify optimal material formulations. As an alternative, adaptive experimental design methods based on Bayesian optimization (BO) principles have been recently proposed to streamline material development by iteratively refining experiments based on prior results. However, most implementations are not suited to manage the heteroscedastic noise inherently present in physical experiments. This work introduces a heteroscedastic Gaussian process (HGP) model within the BO framework to account for varying levels of uncertainty in the data, improve the accuracy of the predictions, and increase the overall experimental efficiency. The aim is to find the optimal TPS film composition that maximizes its elongation at break and tensile strength. To demonstrate the effectiveness of this approach, TPS films were prepared by mixing potato starch, distilled water, glycerol as a plasticizer, and acetic acid as a catalyst. After gelation, the mixture was degassed via centrifugation and molded into films, which were dried at room temperature. Tensile tests were conducted according to ASTM D638 standards. After five iterations and 30 experiments, the films containing 4.5 wt% plasticizer and 2.0 wt% starch exhibited the highest elongation at break (M = 96.7%, SD = 5.6%), while the films with 0.5 wt% plasticizer and 7.0 wt% starch demonstrated the highest tensile strength (M = 2.77 MPa, SD = 1.54 MPa). These results demonstrate the potential of the HGP model within a BO framework to improve material development efficiency and performance in TPS film and other potential material formulations.

Optimizing cervical cancer classification using transfer learning with deep gaussian processes and support vector machines

BackgroundCervical cancer is the fourth most frequent cancer in women worldwide. Even though cervical cancer deaths have decreased significantly in Western countries, low and middle-income countries account for nearly 90% of cervical cancer deaths. While Western countries are leveraging the powers of artificial intelligence (AI) in the health sector, most countries in sub-Saharan Africa are still lagging. In Uganda, cytologists manually analyze Pap smear images for the detection of cervical cancer, a process that is highly subjective, slow, and tedious. Machine learning (ML) algorithms have been used in the automated classification of cervical cancer. However, most of the MLs have overfitting limitations which limits their deployment, especially in the health sector where accurate predictions are needed.MethodsIn this study, we propose two kernel-based algorithms for automated detection of cervical cancer. These algorithms are (1) an optimized support vector machine (SVM), and (2) a deep Gaussian Process (DGP) model. The SVM model proposed uses an optimized radial basis kernel while the DGP model uses a hybrid kernel of periodic and local periodic kernel.ResultsExperimental results revealed accuracy of 100% and 99.48% for an optimized SVM model and DGP model respectively. Results on precision, recall, and F1 score were also reported.ConclusionsThe proposed models performed well on cervical cancer detection and classification, and therefore suitable for deployment. We plan to deploy our proposed models in a mobile application-based tool. The limitation of the study was the lack of access to high-performance computational resources.

A general Bayesian algorithm for the autonomous alignment of beamlines.

Autonomous methods to align beamlines can decrease the amount of time spent on diagnostics, and also uncover better global optima leading to better beam quality. The alignment of these beamlines is a high-dimensional expensive-to-sample optimization problem involving the simultaneous treatment of many optical elements with correlated and nonlinear dynamics. Bayesian optimization is a strategy of efficient global optimization that has proved successful in similar regimes in a wide variety of beamline alignment applications, though it has typically been implemented for particular beamlines and optimization tasks. In this paper, we present a basic formulation of Bayesian inference and Gaussian process models as they relate to multi-objective Bayesian optimization, as well as the practical challenges presented by beamline alignment. We show that the same general implementation of Bayesian optimization with special consideration for beamline alignment can quickly learn the dynamics of particular beamlines in an online fashion through hyperparameter fitting with no prior information. We present the implementation of a concise software framework for beamline alignment and test it on four different optimization problems for experiments on X-ray beamlines at the National Synchrotron Light Source II and the Advanced Light Source, and an electron beam at the Accelerator Test Facility, along with benchmarking on a simulated digital twin. We discuss new applications of the framework, and the potential for a unified approach to beamline alignment at synchrotron facilities.

Analyzing and forecasting air pollution concentration in the capital and Southern Thailand using a lag-dependent Gaussian process model.

The air pollution problem has now amassed worldwide attention due to its multifaceted harm to human health. Exploring the concentration of air pollution and improving forecast have important consideration worldwide. In this research, we analyze the air pollution concentration of Southern Thailand and compare it with the central region. Also, we proposed a methodology based on the lag-dependent Gaussian process (LDGP), a Bayesian non-parametric machine learning model, with a stable optimization approach, which is a cluster-based multi-starter technique based on the Nelder-Mead optimizer. This model also provides the confidence band for forecasted values. We also used autoregressive deep neural network (AR-DNN), autoregressive random forest (AR-RF), gradient boosting (GB), and K-nearest neighbors (KNN) models. A comparison of the proposed methodology was performed on the daily air pollution data collected from the southern provinces and also from the capital of Thailand from 1 January 2018 to 31 December 2022. We used well-established performance evaluation measures to compare the performance of the models. To evaluate the bias due to overfit, we performed a tenfold cross-validation for all the pollutants in each region and compared the models to choose the best one. Moreover, we explored the concentration of air pollution in these regions. Results of descriptive analysis revealed that Bangkok had a much higher concentration of air pollution as compared to the southern region. However, the southern region had higher exposure to PM air pollutants as per WHO recommendations and also had higher exposure to O3 and CO levels. The proposed LDGP model outperformed the other machine learning models for forecasting all air pollutants. Hence, it is recommended to be used by experts for further research and studies with different kernel functions. This research is also expected to contribute to local government planning and prevention and worldwide use of the same methodology for the sustainability of public health.

Advancing Vehicle Trajectory Prediction: A Probabilistic Approach Using Combined Sequential Models

Abstract This paper addresses the critical need to quantify vehicle trajectory uncertainty in autonomous driving under environmental variability. We focus on predicting the posterior distribution of vehicle trajectories over a fixed horizon, given an initial state and a sequence of actions. We propose and compare three approaches: a probabilistic seq2seq model based on stochastic variational Gaussian processes, sequential Monte Carlo simulation with a single-step Gaussian process model, and a hybrid model that leverages the strengths of both methods. Each approach incorporates a baseline vehicle kinematics model to enhance stability and convergence. We evaluate these methods using a dataset generated from the CARLA simulator, assessing both point error metrics and probabilistic prediction metrics. This research introduces novel approaches to quantifying vehicle trajectory uncertainty through various uncertainty quantification techniques, with the goal of improving the safety and reliability of autonomous vehicle control systems.

Analysis of differential scanning calorimetry data for aged plutonium.

Differential scanning calorimetry data for samples of a 52 year old plutonium alloy with 3.3 at. % Ga that were heated beyond the melting point is analyzed using transition state theory to find activation energies for the δ to ɛ and ɛ to liquid phase transitions. A Bayesian statistical method involving a Gaussian process model is used to find mean values and confidence intervals for the activation energies. The activation energy for the δ to ɛ phase transition increases by 3.3 ± 3.8% per decade, relative to the case when all age related plutonium lattice point defects have been removed through annealing. The corresponding increase in activation energy for the ɛ to liquid transition is shown to be 7.1 ± 1.8% per decade. It is postulated that the change in activation energy with age for both phase transitions is caused, in part, by the accumulation of the same type of lattice point defects associated with the observed increase in elastic bulk modulus over time.

A Gaussian process model for stellar activity in 2D line profile time-series

ABSTRACT Stellar active regions like spots and faculae can distort the shapes of spectral lines, inducing variations in the radial velocities that are often orders of magnitude larger than the signals from Earth-like planets. Efforts to mitigate these activity signals have hitherto focused on either the time or the velocity (wavelength) domains. We present a physics-driven Gaussian process (GP) framework to model activity signals directly in time series of line profiles or cross-correlation functions (CCFs). Unlike existing methods that correct activity signals in line profile time series, our approach exploits the time correlation between velocity (wavelength) bins in the line profile variations, and is based on a simplified but physically motivated model for the origin of these variations. When tested on both synthetic and real data sets with signal-to-noise ratios down to ∼100, our method was able to separate the planetary signal from the activity signal, even when their periods were identical. We also conducted injection/recovery tests using two years of realistically sampled HARPS-N solar data, demonstrating the ability of the method to accurately recover a signal induced by a 1.5-Earth mass planet with a semi-amplitude of 0.3 m s−1 and a period of 33 d during high solar activity.

Resilient Coastal Protection Infrastructures: Probabilistic Sensitivity Analysis of Wave Overtopping Using Gaussian Process Surrogate Models

This paper presents a novel mathematical framework for assessing and predicting the resilience of critical coastal infrastructures against wave overtopping hazards and extreme climatic events. A probabilistic sensitivity analysis model is developed to evaluate the relative influence of hydrodynamic, geomorphological, and structural factors contributing to wave overtopping dynamics. Additionally, a stochastic Gaussian process (GP) model is introduced to predict the mean overtopping discharge from coastal defences. Both the sensitivity analysis and the predictive models are validated using a large homogeneous dataset comprising 163 laboratory and field-scale tests. Statistical evaluations demonstrate the superior performance of the GPs in identifying key parameters driving wave overtopping and predicting mean discharge rates, outperforming existing regression-based formulae. The proposed model offers a robust predictive tool for assessing the performance of critical coastal protection infrastructures under various climate scenarios.

Development of electromagnetic pollution maps utilizing Gaussian process spatial models

The rapid proliferation of wireless technologies in everyday environments demands the quick and precise estimation of electromagnetic field distribution. This distribution is commonly depicted through the electric field strength across various geographical areas. The objective of this research is to determine the most effective geospatial model for generating a national-level electric field strength map within the 30 MHz–6 GHz frequency range. To achieve this, we employed five different methodologies for constructing the electric field strength map. Four of these methodologies are based on Gaussian process regression, while the fifth utilizes the classical weighted-average method of the nearest neighbor. Our study focused on a country with a total area of 9251 km2, using a dataset comprising 3621 measurements. The findings reveal that Gaussian process spatial models, also known as Kriging models, generally outperform other methods when applied to spatial data. However, it was observed that, after excluding some outlier data points, the performance of the classical nearest neighbor models becomes comparable to that of the Gaussian process models. This indicates the potential for both approaches to be effective, depending on the data quality and the presence of outliers.

Continuous planning for inertial-aided systems

Inertial-aided systems require continuous motion excitation among other reasons to characterize the measurement biases that will enable accurate integration required for localization frameworks. This paper proposes the use of informative path planning to find the best trajectory for minimizing the uncertainty of IMU biases and an adaptive traces method to guide the planner towards trajectories that aid convergence. The key contribution is a novel regression method based on Gaussian Process (GP) to enforce continuity and differentiability between waypoints from a variant of the RRT∗\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\hbox {RRT}^*$$\\end{document} planning algorithm. We employ linear operators applied to the GP kernel function to infer not only continuous position trajectories, but also velocities and accelerations. The use of linear functionals enable velocity and acceleration constraints given by the IMU measurements to be imposed on the position GP model. The results from both simulation and real-world experiments show that planning for IMU bias convergence helps minimize localization errors in state estimation frameworks.

Iterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models

Latent Gaussian process (GP) models are flexible probabilistic non-parametric function models. Vecchia approximations are accurate approximations for GPs to overcome computational bottlenecks for large data, and the Laplace approximation is a fast method with asymptotic convergence guarantees to approximate marginal likelihoods and posterior predictive distributions for non-Gaussian likelihoods. Unfortunately, the computational complexity of combined Vecchia-Laplace approximations grows faster than linearly in the sample size when used in combination with direct solver methods such as the Cholesky decomposition. Computations with Vecchia-Laplace approximations can thus become prohibitively slow precisely when the approximations are usually the most accurate, i.e., on large data sets. In this article, we present iterative methods to overcome this drawback. Among other things, we introduce and analyze several preconditioners, derive new convergence results, and propose novel methods for accurately approximating predictive variances. We analyze our proposed methods theoretically and in experiments with simulated and real-world data. In particular, we obtain a speed-up of an order of magnitude compared to Cholesky-based calculations and a threefold increase in prediction accuracy in terms of the continuous ranked probability score compared to a state-of-the-art method on a large satellite data set. All methods are implemented in a free C++ software library with high-level Python and R packages. 4

Distance preserving machine learning for uncertainty aware accelerator capacitance predictions

Abstract Accurate uncertainty estimations are essential for producing reliable machine learning models, especially in safety-critical applications such as accelerator systems. Gaussian process models are generally regarded as the gold standard for this task; however, they can struggle with large, high-dimensional datasets. Combining deep neural networks with Gaussian process approximation techniques has shown promising results, but dimensionality reduction through standard deep neural network layers is not guaranteed to maintain the distance information necessary for Gaussian process models. We build on previous work by comparing the use of the singular value decomposition against a spectral-normalized dense layer as a feature extractor for a deep neural Gaussian process approximation model and apply it to a capacitance prediction problem for the High Voltage Converter Modulators in the Oak Ridge Spallation Neutron Source. Our model shows improved distance preservation and predicts in-distribution capacitance values with less than 1% error.

Bayesian Earthquake Forecasting Using Gaussian Process Modeling: GP-ETAS Applications

Abstract Numerous seismicity models are known to simulate different observed characteristics of earthquake occurrence successfully. However, their ability of prospective forecasting future events is a priori not always known. The recently proposed semiparametric model, Gaussian process epidemic-type aftershock sequence (GP-ETAS) model, which combines the ETAS model with GP modeling of the background activity, has led to promising results when applied to synthetic seismicity. In this study, we focus on the ability of GP-ETAS for different forecasting experiments in two case studies: first, the Amatrice, Italy, sequence during 2016 and 2017, and second, long-term seismicity in Southern California. The results indicate that GP-ETAS performs well compared with selected benchmark models. The advantages become particularly visible in cases with sparse data, in which GP-ETAS shows in general a more robust behavior compared to other approaches.