Gaussian Process Surrogate Model Research Articles

A Novel Gaussian Process Surrogate Model with Expected Prediction Error for Optimization under Constraints

Optimization, particularly constrained optimization problems (COPs), is fundamental in engineering, influencing various sectors with its critical role in enhancing design efficiency, reducing experimental costs, and shortening testing cycles. This study explores the challenges inherent in COPs, with a focus on developing efficient solution methodologies under stringent constraints. Surrogate models, especially Gaussian Process Regression (GPR), are pivotal in our approach, enabling the approximation of complex systems with reduced computational demand. We evaluate the efficacy of the Efficient Global Optimization (EGO) algorithm, which synergizes GPR with the Expected Improvement (EI) function, and further extend this framework to Constrained Expected Improvement (CEI) and our novel methodology Constrained Expected Prediction Error (CEPE). We demonstrate the effectiveness of these methodologies by numerical benchmark simulations and the real-world application of optimizing a Three-Bar Truss Design. In essence, the innovative CEPE approach promises a potent balance between solution accuracy and computational prowess, offering significant potential in the broader engineering field.

Accelerated Bayesian Inference for Molecular Simulations using Local Gaussian Process Surrogate Models.

While Bayesian inference is the gold standard for uncertainty quantification and propagation, its use within physical chemistry encounters formidable computational barriers. These bottlenecks are magnified for modeling data with many independent variables, such as X-ray/neutron scattering patterns and electromagnetic spectra. To address this challenge, we employ local Gaussian process (LGP) surrogate models to accelerate Bayesian optimization over these complex thermophysical properties. The time-complexity of the LGPs scales linearly in the number of independent variables, in stark contrast to the computationally expensive cubic scaling of conventional Gaussian processes. To illustrate the method, we trained a LGP surrogate model on the radial distribution function of liquid neon and observed a 1,760,000-fold speed-up compared to molecular dynamics simulation, beating a conventional GP by three orders-of-magnitude. We conclude that LGPs are robust and efficient surrogate models poised to expand the application of Bayesian inference in molecular simulations to a broad spectrum of experimental data.

Robotic compliant assembly for complex-shaped composite aircraft frame based on Gaussian process considering uncertainties

Robots are finding increasing application in aircraft composite structure assembly due to their flexibility and the growing demand of aircraft manufacturers for high production rates. The contact force of the composite frame in a robotic assembly of the aircraft composite fuselage panel can hardly be controlled due to the multi-surface variable contact stiffness caused by compliance and complex shape with multiple mating surfaces. The paper proposes a robotic assembly system for the aircraft composite fuselage frame with a compliant contact force control strategy using the Gaussian process surrogate model. First, a robotic assembly system is introduced, and the global coordinate system transformation model is built. Then, a compliant force control architecture is designed to generate the desired output force. Subsequently, a Gaussian process surrogate model with uncertainties is utilized to model the complicated relationship between the robot’s output force and the normal contact force acting on the mating surface of the composite frame. Furthermore, an optimal contact force control strategy is implemented to improve the contact quality. Finally, an experiment demonstrates that the proposed methodology can ensure that the contact force on each surface is within the limit of the engineering specification and uniformly distributed, improving the quality compared to the traditional assembly process.

AMFGP: An active learning reliability analysis method based on multi-fidelity Gaussian process surrogate model

Multi-fidelity modeling is widely available in theoretical research and engineering practice. Although high-fidelity models often necessitate substantial computational resources, they yield more accurate and reliable results. Low-fidelity models are less computationally demanding, while their results may be inaccurate or unreliable. For the reliability analysis based on complex limit state functions, a method based on active learning multi-fidelity Gaussian process model, called AMFGP, is proposed by combining surrogate model with adaptive strategy, ensuring a balance between prediction accuracy and computational cost in terms of both surrogate modeling and active learning: A dependent Gaussian process surrogate model using complete statistical characteristics is developed under the multi-fidelity framework, and the surrogate performances of different single-fidelity and multi-fidelity models with different learning functions are investigated; based on the proposed model, an adaptive strategy considering the dependence between predictions, the model correlation, and the sample density is designed, and the adaptive performance of different learning functions in different models is explored. The proposed method is validated for effectiveness and adaptability in three mathematical examples with different dimensions and demonstrated for efficiency and practicality in an engineering application to aero engine gear.

Reducing the Environmental Impact of Wireless Communication via Probabilistic Machine Learning

Machine learning methods are increasingly adopted in communications problems, particularly those arising in next generation wireless settings. Though seen as a key climate mitigation and societal adaptation enabler, communications related energy consumption is high and is expected to grow in future networks in spite of anticipated efficiency gains in 6G due to exponential communications traffic growth. To make meaningful climate mitigation impact in the communications sector, a mindset shift away from maximizing throughput at all cost and towards prioritizing energy efficiency is needed. Moreover, this must be adopted in both existing (without incurring further embodied carbon costs through equipment replacement) and future network infrastructure, given the long development time of mobile generations. To that end, we present summaries of two such problems, from both current and next generation network specifications, where probabilistic inference methods were used to great effect: using Bayesian parameter tuning we are able to safely reduce the energy consumption of existing hardware on a live communications network by 11% whilst maintaining operator specified performance envelopes; through spatiotemporal Gaussian process surrogate modeling we reduce the overhead in a next generation hybrid beamforming system by over 60%, greatly improving the networks' ability to target highly mobile users such as autonomous vehicles. The Bayesian paradigm is itself helpful in terms of energy usage, since training a Bayesian optimization model can require much less computation than, say, training a deep neural network.

Optimal drive cycle current supply of a wound field automotive electrical machine using surrogate models

Surrogate models have become a widely used solution for reducing computation times along design processes. In this work, a Gaussian Process surrogate model is built and used to predict the performance and losses of a wound field electrical machine in a fast manner. This approach is relevant, especially for drive cycle calculations that rapidly generate rising computation costs if they are computed using physical models, especially finite elements analysis. We present in detail the established method and a comparison of the obtained results with finite elements results. In addition, a detailed analysis of the optimized current supply is presented, and the advantages of variable excitation current are highlighted.

Geometry of adipocyte packing in subcutaneous tissue contributes to nonlinear tissue properties captured through a Gaussian process surrogate model.

Subcutaneous tissue mechanical response is governed by the geometry and mechanical properties at the microscale and drives physiological and clinical processes such as drug delivery. Even though adipocyte packing is known to change with age, disease, and from one individual to another, the link between the geometry of the packing and the overall mechanical response of adipose tissue remains poorly understood. Here we create 1200 periodic representative volume elements (RVEs) that sample the possible space of Laguerre packings describing adipose tissue. RVE mechanics are modeled under tri-axial loading. Equilibrium configuration of RVEs is solved by minimizing an energetic potential that includes volume change contributions from adipocyte expansion, and area change contributions from collagen foam stretching. The resulting mechanical response across all RVE samples is interpolated with the aid of a Gaussian process (GP), revealing how the microscale geometry dictates the overall RVE mechanics. For example, increase in adipocyte size and increase in sphericity lead to adipose tissue softening. We showcase the use of the homogenized model in finite element simulations of drug injection by implementing a Blatz-Ko model, informed by the GP, as a custom material in the popular open-source package FEBio. These simulations show how microscale geometry can lead to vastly different injection dynamics even if the constituent parameters are held constant, highlighting the importance of characterizing individual's adipose tissue structure in the development of personalized therapies.

UNCERTAINTY QUANTIFICATION AND GLOBAL SENSITIVITY ANALYSIS OF SEISMIC FRAGILITY CURVES USING KRIGING

Seismic fragility curves have been introduced as key components of seismic probabilistic risk assessment studies. They express the probability of failure of mechanical structures conditional to a seismic intensity measure and must take into account various sources of uncertainties, the so-called epistemic uncertainties (i.e., coming from the uncertainty on the mechanical parameters of the structure) and the aleatory uncertainties (i.e., coming from the randomness of the seismic ground motions). For simulation-based approaches we propose a methodology to build and calibrate a Gaussian process surrogate model to estimate a family of nonparametric seismic fragility curves for a mechanical structure by propagating both the surrogate model uncertainty and the epistemic ones. Gaussian processes have indeed the main advantage to propose both a predictor and an assessment of the uncertainty of its predictions. In addition, we extend this methodology to sensitivity analysis. Global sensitivity indices such as aggregated Sobol' indices and kernel-based indices are proposed to know how the uncertainty on the seismic fragility curves is apportioned according to each uncertain mechanical parameter. This comprehensive uncertainty quantification framework is finally applied to an industrial test case consisting of a part of a piping system of a pressurized water reactor.

Gaussian process-based storage location assignments with risk assessments for progressive zone picking systems

E-commerce warehouses are under constant pressure to adapt their order picking systems and reassign product storage locations to meet fluctuating customer demands. Most existing approaches optimize storage location reassignments based on customer orders and operational configurations to maintain high order picking performance. This paper presents a Gaussian process surrogate model (GPSM) approach to predict the performance metrics for storage location reassignments. The GPSM estimates the expected flow time of orders from the historical data on previous storage location assignments and aids in identifying the new assignments that yield the minimum estimated average flow times. Management can also take advantage of the GPSM’s uncertainty quantification capability to assess the probability of improvement for a given storage reassignment and its implementation. The proposed model and assignment policy are validated using discrete-event simulations and industrial data. Experimental results demonstrate that the GPSM can improve expected flow time by 7.51% and reduce unnecessary reassignment operations by 43.25%.

Fast and robust Bayesian inference using Gaussian processes with GPry

We present the GPry algorithm for fast Bayesian inference of general (non-Gaussian) posteriors with a moderate number of parameters. GPry does not need any pre-training, special hardware such as GPUs, and is intended as a drop-in replacement for traditional Monte Carlo methods for Bayesian inference. Our algorithm is based on generating a Gaussian Process surrogate model of the log-posterior, aided by a Support Vector Machine classifier that excludes extreme or non-finite values. An active learning scheme allows us to reduce the number of required posterior evaluations by two orders of magnitude compared to traditional Monte Carlo inference. Our algorithm allows for parallel evaluations of the posterior at optimal locations, further reducing wall-clock times. We significantly improve performance using properties of the posterior in our active learning scheme and for the definition of the GP prior. In particular we account for the expected dynamical range of the posterior in different dimensionalities. We test our model against a number of synthetic and cosmological examples. GPry outperforms traditional Monte Carlo methods when the evaluation time of the likelihood (or the calculation of theoretical observables) is of the order of seconds; for evaluation times of over a minute it can perform inference in days that would take months using traditional methods. GPry is distributed as an open source Python package (pip install gpry) and can also be found at https://github.com/jonaselgammal/GPry.

Bayesian optimization with Gaussian process surrogate model for source localization.

Source localization with a geoacoustic model requires optimizing the model over a parameter space of range and depth with the objective of matching a predicted sound field to a field measured on an array. We propose a sample-efficient sequential Bayesian optimization strategy that models the objective function as a Gaussian process (GP) surrogate model conditioned on observed data. Using the mean and covariance functions of the GP, a heuristic acquisition function proposes a candidate in parameter space to sample, balancing exploitation (sampling around the best observed objective function value) and exploration (sampling in regions of high variance in the GP). The candidate sample is evaluated, and the GP conditioned on the updated data. Optimization proceeds sequentially until a fixed budget of evaluations is expended. We demonstrate source localization for a shallow-water waveguide using Monte Carlo simulations and experimental data from an acoustic source tow. Compared to grid search and quasi-random sampling strategies, simulations and experimental results indicate the Bayesian optimization strategy converges on optimal solutions rapidly.

Machine learning informed visco-plastic model for the cyclic relaxation of 316H stainless steel at 550 °C

Among the structural alloys for this fast reactor, 316H stainless steel has emerged as a promising candidate. Because the operating temperature of Sodium-cooled reactor is specifically designed to be 550 °C, this operating temperature triggers material inelastic behavior depends more on the coupling of fatigue and creep, which complicates the constitutive model. By introducing static recovery terms, previous studies could capture some experimental features, but failed to describe the interaction by fatigue and creep. In this work, in order to describe the fatigue and creep during cyclic relaxation of 316H stainless steel at 550 °C, we propose a modified visco-plastic constitutive model within the framework of unified Chaboche model. In the proposed model, the parameters related to the static recovery items are coupled, and thus cannot be identified from experiments using the traditional trial and error. To address this issue, we employed the Bayesian approach to identify these parameters. The parameter identification involves two steps: (i) constructing a Gaussian Process surrogate model using data generated from the finite element method, and (ii) obtaining the value of parameters through Markov Chain Monte Carlo sampling under the Bayesian framework. The proposed procedure, is demonstrated by the using experimental results of 316H stainless steel at 550 °C. Under the coupling of fatigue-creep, the material exhibits a cyclic-dependent accelerated stress relaxation before reaching the saturated stage and a steady state of relaxed stress after a long holding time. These mechanical responses are well predicted by the proposed model. Further, we conducted two kinds of multi-axial cyclic test, tensile test of notched bar and coupled tensile-torsion test, to validate the proposed constitutive model for the cyclic behavior under the multi-axial stress state.

Faster variational quantum algorithms with quantum kernel-based surrogate models

Abstract We present a new optimization strategy for small-to-intermediate scale variational quantum algorithms (VQAs) on noisy near-term quantum processors which uses a Gaussian process surrogate model equipped with a classically-evaluated quantum kernel. VQAs are typically optimized using gradient-based approaches however these are difficult to implement on current noisy devices, requiring large numbers of objective function evaluations. Our approach shifts this computational burden onto the classical optimizer component of these hybrid algorithms, greatly reducing the number of quantum circuit evaluations required from the quantum processor. We focus on the variational quantum eigensolver (VQE) algorithm and demonstrate numerically that these surrogate models are particularly well suited to the algorithm’s objective function. Next, we apply these models to both noiseless and noisy VQE simulations and show that they exhibit better performance than widely-used classical kernels in terms of final accuracy and convergence speed. Compared to the typically-used stochastic gradient-descent approach to VQAs, our quantum kernel-based approach is found to consistently achieve significantly higher accuracy while requiring less than an order of magnitude fewer quantum circuit executions. We analyze the performance of the quantum kernel-based models in terms of the kernels’ induced feature spaces and explicitly construct their feature maps. Finally, we describe a scheme for approximating the best-performing quantum kernel using a classically-efficient tensor network representation of its input state and so provide a pathway for scaling this strategy to larger systems.

Gaussian Process-Based Random Search for Continuous Optimization via Simulation

A gaussian process-based random search framework for continuous optimization via simulation Stochastic optimization via simulation (OvS) is widely used for optimizing the performances of complex systems with continuous decision variables. Because of the existence of simulation noise and infinite feasible solutions, it is challenging to design an efficient mechanism to do the searching and estimation simultaneously to find the optimal solutions. In “Gaussian process-based random search for continuous optimization via simulation,” Wang et al. propose a Gaussian process-based random search (GPRS) framework for the design of single-observation and adaptive continuous OvS algorithms. This framework builds a Gaussian process surrogate model to estimate the objective function value of every solution based on a single observation of each sampled solution in each iteration and allow for a wide range of sampling distributions. They prove the global convergence and analyze the rate of convergence for algorithms under the GPRS framework. They also give a specific example of GPRS algorithms and validate its theoretical properties and practical efficiency using numerical experiments.

Full long-term extreme buffeting response calculations using sequential Gaussian process surrogate modeling

The full long-term method (FLM) provides the most accurate estimates of the long-term (i.e., 10–100 years) extreme buffeting responses in the design of wind-sensitive structures. However, it suffers from high computational demand if the number of uncertain turbulence parameters becomes high. In this paper, we propose a new algorithm for efficient solution of the full long-term problem through sequential Gaussian Process (GP) surrogate modeling. The algorithm efficiently trains a GP surrogate model that replaces the buffeting response calculation while exploiting the specific properties of the long-term problem. The performance of the algorithm is demonstrated on a practical design problem of estimating extreme buffeting-induced stresses of a long-span suspension bridge, where the results are compared with the ones obtained from other viable methods. The results show that the algorithm achieves computational efficiency comparable to the highly efficient IFORM, while also significantly enhancing the accuracy of the response estimate. The algorithm also has the added benefit of converging to the true solution of the full long-term problem, in contrast to the IFORM solution, which converges to an approximate solution.

A dynamic Gaussian process surrogate model based on the grasshopper optimization algorithm for non-probabilistic reliability analysis of complex structures

In practical complex engineering structures, the performance function (PF) for reliability analysis is commonly implicit and highly nonlinear. Commonly used surrogate models are appropriate for structural reliability analysis with an implicit PF. However, these methods require hundreds of PF values, which are time-consuming to obtain by adopting numerical analysis, such as finite element analysis (FEA). Therefore, since non-probabilistic reliability analysis does not require a large number of samples, it has great development potential. In this paper, a dynamic Gaussian process (GP) surrogate model based on the grasshopper optimization algorithm (DGP-GOA) is proposed for the non-probabilistic reliability analysis. First, with the help of the scale factor of the convex set model, the non-probabilistic reliability analysis problem is transformed into an unconstrained optimization problem. Second, the DGP-GOA fits the PF by constructing a GP surrogate model with a small dataset. Third, the GOA is used to search for the global optimal solution to obtain a non-probabilistic reliability index. Then, a dynamic retraining strategy is proposed to improve the fitting accuracy and efficiency. The results demonstrate that the proposed method is highly applicable to the non-probabilistic structural reliability analysis of complex engineering structures.

Machine Learning-Enabled Development of Accurate Force Fields for Refrigerants.

Hydrofluorocarbon (HFC) refrigerants with zero ozone-depleting potential have replaced chlorofluorocarbons and are now ubiquitous. However, some HFCs have high global warming potential, which has led to calls by governments to phase out these HFCs. Technologies to recycle and repurpose these HFCs need to be developed. Therefore, thermophysical properties of HFCs are needed over a wide range of conditions. Molecular simulations can help understand and predict the thermophysical properties of HFCs. The prediction capability of a molecular simulation is directly tied to the accuracy of the force field. In this work, we applied and refined a machine learning-based workflow to optimize the Lennard-Jones parameters of classical HFC force fields for HFC-143a (CF3CH3), HFC-134a (CH2FCF3), R-50 (CH4), R-170 (C2H6), and R-14 (CF4). Our workflow involves liquid density iterations with molecular dynamics simulations and vapor-liquid equilibrium (VLE) iterations with Gibbs ensemble Monte Carlo simulations. Support vector machine classifiers and Gaussian process surrogate models save months of simulation time and can efficiently select optimal parameters from half a million distinct parameter sets. Excellent agreement as evidenced by low mean absolute percent errors (MAPEs) of simulated liquid density (ranging from 0.3% to 3.4%), vapor density (ranging from 1.4% to 2.6%), vapor pressure (ranging from 1.3% to 2.8%), and enthalpy of vaporization (ranging from 0.5% to 2.7%) relative to experiments was obtained for the recommended parameter set of each refrigerant. The performance of each new parameter set was superior or similar to the best force field in the literature.

Fast optimisation of the formability of dry fabric preforms: A Bayesian approach

A new framework for optimising the process of forming dry textile materials using finite element (FE) analysis and Gaussian Process (GP) regression is explored in this work. FE models were generated to simulate the double diaphragm forming process of non-crimp fabric over a hemisphere tool. A GP emulator was developed to regress the dataset generated by FE model, then used to optimise the forming process. Importantly the FE simulations can capture the formation of wrinkles during the process under different forming configurations and boundary conditions. Rigid blocks (risers) were introduced to the forming process to affect the defects generation by controlling the block positions. Several indices abstracted from FE output files were used to assess the wrinkle level of the forming simulations and compared, as the model output. A small dataset was generated by Latin hypercubic sampling (LHS) to train an initial GP surrogate model. Then, the prediction error of the model was reduced to an acceptable level (<10 %) through a Bayesian active learning method. The trained surrogate model was then used to optimise a forming process using only tens of simulations, rather than hundreds or even thousands, as required by traditional optimisation methods.

Barycentric Kernel for Bayesian Optimization of Chemical Mixture

Chemical-reaction optimization not only increases the yield of chemical processes but also reduces impurities and improves the performance of the resulting products, contributing to important innovations in various industries. This paper presents a novel barycentric kernel for chemical-reaction optimization using Bayesian optimization (BO), a powerful machine-learning method designed to optimize costly black-box functions. The barycentric kernel is specifically tailored as a positive definite kernel for Gaussian-process surrogate models in BO, ensuring stability in logarithmic and differential operations while effectively mapping concentration space for solving optimization problems. We conducted comprehensive experiments comparing the proposed barycentric kernel with other widely used kernels, such as the radial basis function (RBF) kernel, across six benchmark functions in concentration space and three Hartmann functions in Euclidean space. The results demonstrated the barycentric kernel’s stable convergence and superior performance in these optimization scenarios. Furthermore, the paper highlights the importance of accurately parameterizing chemical concentrations to prevent BO from searching for infeasible solutions. Initially designed for chemical reactions, the versatile barycentric kernel shows promising potential for a wide range of optimization problems, including those requiring a meaningful distance metric between mixtures.

A new framework for the assessment of model probabilities of the different crystal plasticity models for lamellar grains in Titanium alloys

This paper presents a novel framework for assessing the relative accuracy of the different slip transfer criterion in modeling the constitutive response of the lamellar morphology grains in Titanium alloys. The main steps involved in the proposed framework include: (i) rigorous statistical evaluation of the salient morphological characteristics of the lamellar grains using image processing techniques, (ii) modeling the effects of dislocation slip transfer as additive penalties to the critical resolved shear strengths of the different slip systems in each phase, (iii) consideration of multiple geometric slip transfer criteria for the numerical evaluation of the different penalty terms, and (iv) the Bayesian calibration of the material parameters in the constitutive descriptions using spherical nanoindentation experimental data on a polycrystalline Ti-6Al-4V sample. Step (iv) described above involves the construction and refinement of a Gaussian process (GP) surrogate model for the indentation measurements, which is trained on crystal plasticity finite element simulations incorporating the constitutive descriptions selected in Steps (ii) and (iii). The GP surrogates are then combined with Markov Chain Monte Carlo sampling techniques to calibrate the parameters in the various constitutive models studied in this work. Additionally, we have computed the posterior model probabilities for all the constitutive models considered and identified the most plausible slip transfer criterion. This specific constitutive model is then utilized to present a homogenized crystal plasticity model for the lamellar morphology. The proposed framework presents a robust scale-bridging protocol for incorporating the uncertain information obtained from lower scales.