Gaussian Process Model Research Articles

Prediction of decay heat using non-destructive assay

This research paper introduces a novel approach to predict the decay heat of spent nuclear fuel assemblies (SNFs) using data from non-destructive gamma and neutron measurements, addressing the challenge of ensuring safety in geological repositories. Because calorimetric measurements are time-consuming, it is envisioned that gamma and neutron measurements can be used for decay heat prediction before encapsulation. This paper analyses gamma and neutron data to extract key features, specifically the activities of Cs-137, Eu-154, and the total neutron count rate. A Gaussian process model is then employed to estimate SNF decay heat. The methodology involves training a prediction model on a calibrated simulated dataset designed to mimic real experimental conditions closely. The model is then successfully used to predict the decay heat for unseen experimental data. The results highlight the potential of using gamma and neutron measurements for reliable decay heat prediction. It is shown that the magnitude of the relative deviation obtained is 2-4 %. Furthermore, the study explores the impact of removing certain input features or adjusting their uncertainty levels on the decay heat prediction model precision, in particular for the Eu-154 activity and neutron count rate. This comprehensive methodology paves the way for applying these techniques to a larger experimental scale offering a significant advancement in the safety assessment of SNFs prior to encapsulation and long-term storage.

MLAPI: A framework for developing machine learning-guided drug particle syntheses in automated continuous flow platforms

Recently, machine learning (ML) models are increasingly being used in process analytical technology (PAT) frameworks for pharmaceutical manufacturing. Yet, the applications of ML-integrated PAT frameworks are limited by big data requirements. This work introduces a computational framework to develop data-efficient ML models to guide drug particle synthesis in an automated continuous flow precipitation platform. The framework incorporates classification algorithms to identify feasible (fouling-free) operating regions of the precipitation platform, a multiple-output Gaussian process (GP) regression model to relate key process parameters to the drug particle size, and active learning to optimally generate new data for training and validation of the GP model. The usefulness of the proposed framework is demonstrated on the synthesis of ibuprofen microparticles in an automated flow precipitation platform. We envision that properly trained GP models developed using the proposed framework can be employed to fine tune the drug particle size, targeting desired particle bioavailability and processability.

Physics-informed machine learning for system reliability analysis and design with partially observed information

Constructing a high-fidelity predictive model is crucial for analyzing complex systems, optimizing system design, and enhancing system reliability. Although Gaussian Process (GP) models are well-known for their capability to quantify uncertainty, they rely heavily on data and necessitate a large representative dataset to establish a high-fidelity predictive model. Physics-informed Machine Learning (PIML) has emerged as a way to integrate prior physics knowledge and machine learning models. However, current PIML methods are generally based on fully observed datasets and mainly suffer from two challenges: (1) effectively utilize partially available information from multiple sources of varying dimensions and fidelity; (2) incorporate physics knowledge while maintaining the mathematical properties of the GP-based model and uncertainty quantification capability of the predictive model. To overcome these limitations, this paper proposes a novel physics-informed machine learning method that incorporates physics prior knowledge and multi-source data by leveraging latent variables through the Bayesian framework. This method effectively utilizes partially available limited information, significantly reduces the need for costly fully observed data, and improves prediction accuracy while maintaining the model property of uncertainty quantification. The developed approach has been demonstrated with two case studies: the vehicle design problem and the battery capacity loss prediction. The case study results demonstrate the effectiveness of the proposed model in complex system design and optimization while propagating uncertainty with limited fully observed data.

Optimization design of composite cap-shape pillar structure based on gaussian process of combined kernel function

Abstract To quickly realize the optimal design of the cap-shaped structure, an optimization design method of composite cap-shaped pillar structure based on combined kernel function Gaussian process was proposed. First, the shortcomings of two kernel functions representing different linear characteristics of data in model construction were analyzed, on which a new combined kernel function was constructed. Based on the finite element numerical simulation results, a database with the web thickness, web height, fiber volume, and ultimate bearing capacity of the cap-shaped structure was constructed. Then, a Gaussian process model based on the combined kernel function was established and the model parameters were trained by the maximum likelihood method. Finally, taking the ultimate bearing capacity per unit volume as the optimization objective, the artificial fish swarm algorithm was used to quickly obtain the structure design parameters under the optimal objective function, and the simulation results were compared with the initial design parameters. The results showed that the method proposed in this paper can optimize the structure performance, improve the design efficiency, and meet the engineering design requirements.

A sub-Earth-mass planet orbiting Barnard’s star

Context. ESPRESSO guaranteed time observations (GTOs) at the 8.2m VLT telescope were performed to look for Earth-like exoplanets in the habitable zone of nearby stars. Barnard’s star is a primary target within the ESPRESSO GTO as it is the second closest neighbour to our Sun after the α Centauri stellar system. Aims. We present here a large set of 156 ESPRESSO observations of Barnard’s star carried out over four years with the goal of exploring periods of shorter than 50 days, thus including the habitable zone (HZ). Methods. Our analysis of ESPRESSO data using Gaussian process (GP) to model stellar activity suggests a long-term activity cycle at 3200 d and confirms stellar activity due to rotation at 140 d as the dominant source of radial velocity (RV) variations. These results are in agreement with findings based on publicly available HARPS, HARPS-N, and CARMENES data. ESPRESSO RVs do not support the existence of the previously reported candidate planet at 233 d. Results. After subtracting the GP model, ESPRESSO RVs reveal several short-period candidate planet signals at periods of 3.15 d, 4.12 d, 2.34 d, and 6.74 d. We confirm the 3.15 d signal as a sub-Earth mass planet, with a semi-amplitude of 55 ± 7 cm s−1, leading to a planet minimum mass mp sin i of 0.37 ± 0.05 M⊕, which is about three times the mass of Mars. ESPRESSO RVs suggest the possible existence of a candidate system with four sub-Earth mass planets in circular orbits with semi-amplitudes from 20 to 47 cm s−1, thus corresponding to minimum masses in the range of 0.17–0.32 M⊕. Conclusions. The sub-Earth mass planet at 3.1533 ± 0.0006 d is in a close-to circular orbit with a semi-major axis of 0.0229 ± 0.0003 AU, thus located inwards from the HZ of Barnard’s star, with an equilibrium temperature of 400 K. Additional ESPRESSO observations would be required to confirm that the other three candidate signals originate from a compact short-period planet system orbiting Barnard’s star inwards from its HZ.

Spatio-temporal Prediction of Air Quality Using Spatio-temporal Clustering and Hierarchical Bayesian Model

Air pollution can cause many negative impacts, so it is meaningful to establish an air quality prediction model with high accuracy for air pollution prevention and control. In previous studies, most models did not consider the spatio-temporal distribution characteristics of air quality. So this study proposes a novel spatio-temporal prediction approach STC-HBM, which combines spatio-temporal clustering (STC) and hierarchical Bayesian model (HBM) to build a spatio-temporal prediction model. In this study, we first perform spatio-temporal clustering on the Beijing-Tianjin-Hebei region in China and then apply hierarchical Bayesian models to different clusters separately. We consider three hierarchical Bayesian models: the separable spatio-temporal (SST) model, the Gaussian process (GP) model, and the autoregressive (AR) model. Since the prior distribution affects the prediction accuracy of the HBM, the resultant output of the Bayesian linear regression (BLR) model is used as the prior input of the HBM, which improves the flexibility of the model. The experimental results show that 13 cities in the BTH region are clustered into two clusters according to their spatio-temporal characteristics. Based on MAE, RMSE, MAPE, and coverage (CVG), cluster 1, and cluster 2 are better in both temporal and spatial prediction compared to the overall prediction model. In addition, for cluster 1, the models with the best prediction in time and space are AR and GP, respectively; for cluster 2, the models with the best prediction in time and space are SST and GP, respectively.

GPMelt: A hierarchical Gaussian process framework to explore the dark meltome of thermal proteome profiling experiments.

Thermal proteome profiling (TPP) is a proteome wide technology that enables unbiased detection of protein drug interactions as well as changes in post-translational state of proteins between different biological conditions. Statistical analysis of temperature range TPP (TPP-TR) datasets relies on comparing protein melting curves, describing the amount of non-denatured proteins as a function of temperature, between different conditions (e.g. presence or absence of a drug). However, state-of-the-art models are restricted to sigmoidal melting behaviours while unconventional melting curves, representing up to 50% of TPP-TR datasets, have recently been shown to carry important biological information. We present a novel statistical framework, based on hierarchical Gaussian process models and named GPMelt, to make TPP-TR datasets analysis unbiased with respect to the melting profiles of proteins. GPMelt scales to multiple conditions, and extension of the model to deeper hierarchies (i.e. with additional sub-levels) allows to deal with complex TPP-TR protocols. Collectively, our statistical framework extends the analysis of TPP-TR datasets for both protein and peptide level melting curves, offering access to thousands of previously excluded melting curves and thus substantially increasing the coverage and the ability of TPP to uncover new biology.

Adaptive safe reinforcement learning‐enabled optimization of battery fast‐charging protocols

AbstractOptimizing charging protocols is critical for reducing battery charging time and decelerating battery degradation in applications such as electric vehicles. Recently, reinforcement learning (RL) methods have been adopted for such purposes. However, RL‐based methods may not ensure system (safety) constraints, which can cause irreversible damages to batteries and reduce their lifetime. To this end, this article proposes an adaptive and safe RL framework to optimize fast charging strategies while respecting safety constraints with a high probability. In our method, any unsafe action that the RL agent decides will be projected into a safety region by solving a constrained optimization problem. The safety region is constructed using adaptive Gaussian process (GP) models, consisting of static and dynamic GPs, that learn from online experience to adaptively account for any changes in battery dynamics. Simulation results show that our method can charge the batteries rapidly with constraint satisfaction under varying operating conditions.

Bayesian Sequential Design of Computer Experiments for Quantile Set Inversion

We consider an unknown multivariate function representing a system—such as a complex numerical simulator—taking both deterministic and uncertain inputs. Our objective is to estimate the set of deterministic inputs leading to outputs whose probability (with respect to the distribution of the uncertain inputs) of belonging to a given set is less than a given threshold. This problem, which we call Quantile Set Inversion (QSI), occurs for instance in the context of robust (reliability-based) optimization problems, when looking for the set of solutions that satisfy the constraints with sufficiently large probability. To solve the QSI problem we propose a Bayesian strategy, based on Gaussian process modeling and the Stepwise Uncertainty Reduction (SUR) principle, to sequentially choose the points at which the function should be evaluated to efficiently approximate the set of interest. We illustrate the performance and interest of the proposed SUR strategy through several numerical experiments.

Multi-output prediction of dose–response curves enables drug repositioning and biomarker discovery

Drug response prediction is hampered by uncertainty in the measures of response and selection of doses. In this study, we propose a probabilistic multi-output model to simultaneously predict all dose–responses and uncover their biomarkers. By describing the relationship between genomic features and chemical properties to every response at every dose, our multi-output Gaussian Process (MOGP) models enable assessment of drug efficacy using any dose–response metric. This approach was tested across two drug screening studies and ten cancer types. Kullback-leibler divergence measured the importance of each feature and identified EZH2 gene as a novel biomarker of BRAF inhibitor response. We demonstrate the effectiveness of our MOGP models in accurately predicting dose–responses in different cancer types and when there is a limited number of drug screening experiments for training. Our findings highlight the potential of MOGP models in enhancing drug development pipelines by reducing data requirements and improving precision in dose–response predictions.

Active learning inspired multi-fidelity probabilistic modelling of geomaterial property

The identification of geomaterial properties is costly but pivotal for engineering design. A wide range of approaches perform well with sufficiently measured data but their performance is problematic for sparse data. To address this issue, this study proposes an active learning based multi-fidelity residual Gaussian process (AL-MR-GP) modelling framework. A low-fidelity (LF) prediction model is first trained using extensive LF data collected from worldwide sites to generate preliminary estimations. Subsequent training employs active learning to prioritize high-fidelity data from the specific site of interest with larger information gain for calibrating the LF model to make ultimate predictions. The compression index of clays is selected as an example to examine the capability of the proposed framework. The results indicate that using the same number of site-specific datasets, the compression index of clays can be well captured by AL-MR-GP, exhibiting superior accuracy and reliability than models without incorporating multi-fidelity data or active learning. Based on unified LF data, the proposed framework becomes data-efficient for the model development of three sites and is significantly competitive in extrapolation, compared with site-specific models even with active learning. These promising characteristics indicate substantial potential to be extended to broader applications in geotechnical engineering.

Non-smooth Bayesian optimization in tuning scientific applications

Tuning algorithmic parameters to optimize the performance of large, complicated computational codes is an important problem involving finding the optima and identifying regimes defined by non-smooth boundaries in black-box functions. Within the Bayesian optimization framework, the Gaussian process surrogate model produces smooth mean functions, but functions in the tuning problem are often non-smooth, which is exacerbated by the fact that we usually have limited sequential samples from the black-box function. Motivated by these issues encountered in tuning, we propose a novel Gaussian process model called a clustered Gaussian process (cGP), where the components are dynamically updated by clustering. In our studies, the performance of cGP can be better than stationary GPs in nearly 90% of the experiments and better than non-stationary GPs in nearly 70% of the repeated experiments while requiring less computational cost. cGP provides a novel approach for dynamic GP, computes more efficiently than recursive partitioning, and discovers non-smoothness regimes. We provide extensive experiments including high-performance computing (HPC) and industrial simulation functions to show the effectiveness of our methods.

A Data-Driven Approach for Forecasting Current Direction With a Hybrid Model of Empirical Mode Decomposition and Warped Gaussian Process

Abstract Ocean current forecasting is essential for tidal renewable energy generation and operation. However, forecasting the current direction at multiple points along the water depth are still lacking of comprehensive studies. In this study, a data-driven approach was developed to attain short-term prediction in the current direction with reasonable uncertainty quantification. The developed approach employed empirical mode decomposition (EMD) and the warped Gaussian process (WGP) in the forecasting process. The ocean current data, which were measured by a seabed-mounted acoustic Doppler current profiler in the Haitian Strait, were used to illustrate the developed approach. The measured current direction data were preprocessed with the average shifting method to obtain the principal and random components for the improvement of the forecasting accuracy. The random components were decomposed into intrinsic mode functions (IMFs) and residuals. The principal components, IMFs, and residuals of the current direction were then forecasted by the WGP approach. The forecasting performance of the developed approach was investigated through comparisons with those of single standard GP, single WGP, and EMD + GP models. The effects of the kernel function and training input on the forecasting efficiency and precision were investigated. The extrapolation performances of the proposed model for a 1-step prediction and multistep-ahead prediction were also examined.

Soybean prediction using computationally efficient Bayesian spatial regression models and satellite imagery

AbstractPreharvest yield estimates can be used for harvest planning, marketing, and prescribing in‐season fertilizer and pesticide applications. One approach that is being widely tested is the use of machine learning (ML) or artificial intelligence (AI) algorithms to estimate yields. However, one barrier to the adoption of this approach is that ML/AI algorithms behave as a black block. An alternative approach is to create an algorithm using Bayesian statistics. In Bayesian statistics, prior information is used to help create the algorithm. However, algorithms based on Bayesian statistics are not often computationally efficient. The objective of the current study was to compare the accuracy and computational efficiency of four Bayesian models that used different assumptions to reduce the execution time. In this paper, the Bayesian multiple linear regression (BLR), Bayesian spatial, Bayesian skewed spatial regression, and the Bayesian nearest neighbor Gaussian process (NNGP) models were compared with ML non‐Bayesian random forest model. In this analysis, soybean (Glycine max) yields were the response variable (y), and spaced‐based blue, green, red, and near‐infrared reflectance that was measured with the PlanetScope satellite were the predictor (x). Among the models tested, the Bayesian (NNGP; R2‐testing = 0.485) model, which captures the short‐range correlation, outperformed the (BLR; R2‐testing = 0.02), Bayesian spatial regression (SRM; R2‐testing = 0.087), and Bayesian skewed spatial regression (sSRM; R2‐testing = 0.236) models. However, associated with improved accuracy was an increase in run time from 534 s for the BLR model to 2047 s for the NNGP model. These data show that relatively accurate within‐field yield estimates can be obtained without sacrificing computational efficiency and that the coefficients have biological meaning. However, all Bayesian models had lower R2 values and higher execution times than the random forest model.

Multispecies initial numerical validation of an efficient algorithm prototype for auditory brainstem response hearing threshold estimation.

The auditory brainstem response (ABR) can be used to evaluate hearing sensitivity of animals. However, typical measurement protocols are time-consuming. Here, an adaptive algorithm is proposed for efficient ABR threshold estimation. The algorithm relies on the update of the predicted hearing threshold from a Gaussian process model as ABR data are collected using iteratively optimized stimuli. To validate the algorithm, ABR threshold estimation is simulated by adaptively subsampling pre-collected ABR datasets. The simulated experiment is performed on 5 datasets of mouse, budgerigar, gerbil, and guinea pig ABRs (27 ears). The datasets contain 68-106 stimuli conditions, and the adaptive algorithm is configured to terminate after 20 stimuli conditions. The algorithm threshold estimate is compared against human rater estimates who visually inspected the full waveform stacks. The algorithm threshold matches the human estimates within 10 dB, averaged over frequency, for 15 of the 27 ears while reducing the number of stimuli conditions by a factor of 3-5 compared to standard practice. The intraclass correlation coefficient is 0.81 with 95% upper and lower bounds at 0.74 and 0.86, indicating moderate to good reliability between human and algorithm threshold estimates. The results demonstrate the feasibility of a Bayesian adaptive procedure for rapid ABR threshold estimation.

A novel class of non-Gaussian system performance assessment and controller parameter tuning methods

Traditional variance-based control performance assessment (CPA) and controller parameter tuning (CPT) methods tend to ignore non-Gaussian external disturbances. To address this limitation, this study proposes a novel class of CPA and CPT methods for non-Gaussian single-input single-output systems, denoted as data Gaussianization (inverse) transformation methods. The idea of quantile transformation is used to transform the non-Gaussian data with the goal of maximizing mutual information into virtual Gaussian data. In addition, optimal system data for the virtual loop are mapped back to the actual non-Gaussian system using quantile inverse transformation. Furthermore, a CARMA model-based recursive extended least square algorithm and a CARMA model-based least absolute deviation iterative algorithm are used to identify virtual Gaussian and non-Gaussian system process models, respectively, while implementing the CPT. Finally, a unified framework is proposed for the CPA and CPT of a non-Gaussian control system. The simulation results demonstrate that the proposed strategy can provide a consistent benchmark judgment criterion (threshold) for different non-Gaussian noises, and the tuned controller parameters have good performance.

Benchmarking of quantum fidelity kernels for Gaussian process regression

Abstract Quantum computing algorithms have been shown to produce performant quantum kernels for machine-learning classification problems. Here, we examine the performance of quantum kernels for regression problems of practical interest. For an unbiased benchmarking of quantum kernels, it is necessary to construct the most optimal functional form of the classical kernels and the most optimal quantum kernels for each given data set. We develop an algorithm that uses an analog of the Bayesian information criterion to optimize the sequence of quantum gates used to estimate quantum kernels for Gaussian process models. The algorithm increases the complexity of the quantum circuits incrementally, while improving the performance of the resulting kernels, and is shown to yield much higher model accuracy with fewer quantum gates than a fixed quantum circuit ansatz. We demonstrate that quantum kernels thus obtained can be used to build accurate models of global potential energy surfaces (PES) for polyatomic molecules. The average interpolation error of the six-dimensional PES obtained with a random distribution of 2000 energy points is 16 cm−1 for H3O+, 15 cm−1 for H2CO and 88 cm−1 for HNO2. We show that a compositional optimization of classical kernels for Gaussian process regression converges to the same errors. This indicates that quantum kernels can achieve the same, though not better, expressivity as classical kernels for regression problems.

Implementing Gaussian process modelling in predictive maintenance of mining machineries

Implementing Gaussian process modelling in predictive maintenance of mining machineries

Long-term memory induced correction to Arrhenius law

The Kramers escape problem is a paradigmatic model for the kinetics of rare events, which are usually characterized by Arrhenius law. So far, analytical approaches have failed to capture the kinetics of rare events in the important case of non-Markovian processes with long-term memory, as occurs in the context of reactions involving proteins, long polymers, or strongly viscoelastic fluids. Here, based on a minimal model of non-Markovian Gaussian process with long-term memory, we determine quantitatively the mean FPT to a rare configuration and provide its asymptotics in the limit of a large energy barrier E. Our analysis unveils a correction to Arrhenius law, induced by long-term memory, which we determine analytically. This correction, which we show can be quantitatively significant, takes the form of a second effective energy barrier E′<E\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$E^{\\prime} \\, < \\, E$$\\end{document} and captures the dependence of rare event kinetics on initial conditions, which is a hallmark of long-term memory. Altogether, our results quantify the impact of long-term memory on rare event kinetics, beyond Arrhenius law.

Bayesian modelling of multiple plasma diagnostics at Wendelstein 7-X

Abstract Inference of electron density and temperature has been performed using multiple, diverse sets of plasma diagnostic data at Wendelstein 7-X. Predictive models for the interferometer, Thomson scattering and helium beam emission spectroscopy (He-BES) systems have been developed within the Minerva framework and integrated into a unified model. Electron density and temperature profiles are modelled using Gaussian processes. Calibration factors for the Thomson scattering system and predictive uncertainties are considered as additional unknown parameters. The joint posterior probability distribution for the electron density and temperature profiles as well as Gaussian process hyperparameters and model parameters is explored through a Markov chain Monte Carlo algorithm. Samples from this distribution are numerically marginalised over the hyperparameters and model parameters to yield marginal posterior distributions for the electron density and temperature profiles. The profile inferences incorporate various data combinations from the interferometer and Thomson scattering as well as constraints at the limiter/divertor positions through virtual observations or edge data from He-BES. Additionally, the integration of x-ray imaging crystal spectrometer data into the model for ion temperature profiles is presented. All profiles presented in this study are inferred with optimally selected hyperparameters and model parameters by exploring the joint posterior distribution, inherently applying Bayesian Occam’s razor.