Multi-fidelity Gaussian Process Research Articles

Parametric shape optimization of pin fin arrays using a multi-fidelity surrogate model based Bayesian method

The current work aims to develop a multi-fidelity Bayesian optimization methodology to reduce computation time in CFD based design optimization of an internal cooling channel of a gas turbine blade. To build energy efficient gas turbine engines, it is important to increase the turbine inlet temperature, while preventing material failure of hot gas path components. Power lost due to pumping compressed air in turbine blade internal cooling channels can affect the overall efficiency of a power cycle. While designing such a cooling channel, it is important to ensure high heat transfer within a given pressure drop budget. Design optimization using CFD typically needs a sufficient number of computationally expensive high fidelity simulations to search for an optimum design. In the current work, a multi-fidelity Gaussian process surrogate based Bayesian optimization framework has been used to design a pin fin array channel, typically used for gas turbine blade trailing edge cooling, with the goal of maximizing Nusselt number at a constrained pressure loss. Data from a cheaper, low fidelity computation model has been leveraged, along with an expensive high fidelity one, in order to search for a design configuration that maximizes heat transfer at a given pressure drop constraint. The multi-fidelity approach improves upon single fidelity optimization results, at a low additional computational cost due to usage of simulation data from the cheap lowfidelity CFD model. A multifidelity approach outperformed the previous single fidelity case, with 6% higher objective while satisfying the pressure constraint, with 2% lower computational expense and 8 fewer high fidelity data point computations.

Multifidelity Gaussian Processes for Predicting Shear Viscosity over Wide Ranges of Liquid State Points Based on Molecular Dynamics Simulations

Multifidelity Gaussian Processes for Predicting Shear Viscosity over Wide Ranges of Liquid State Points Based on Molecular Dynamics Simulations

AMFGP: An active learning reliability analysis method based on multi-fidelity Gaussian process surrogate model

Multi-fidelity modeling is widely available in theoretical research and engineering practice. Although high-fidelity models often necessitate substantial computational resources, they yield more accurate and reliable results. Low-fidelity models are less computationally demanding, while their results may be inaccurate or unreliable. For the reliability analysis based on complex limit state functions, a method based on active learning multi-fidelity Gaussian process model, called AMFGP, is proposed by combining surrogate model with adaptive strategy, ensuring a balance between prediction accuracy and computational cost in terms of both surrogate modeling and active learning: A dependent Gaussian process surrogate model using complete statistical characteristics is developed under the multi-fidelity framework, and the surrogate performances of different single-fidelity and multi-fidelity models with different learning functions are investigated; based on the proposed model, an adaptive strategy considering the dependence between predictions, the model correlation, and the sample density is designed, and the adaptive performance of different learning functions in different models is explored. The proposed method is validated for effectiveness and adaptability in three mathematical examples with different dimensions and demonstrated for efficiency and practicality in an engineering application to aero engine gear.

Machine learning framework for the real-time reconstruction of regional 4D ocean temperature fields from historical reanalysis data and real-time satellite and buoy surface measurements

The ability to model 4D properties of near-coast ocean regions in real time is valuable for both general environmental monitoring and for specialized industry applications. We present a computationally parsimonious framework to build improved spatiotemporal models that combine dynamics inferred from historical high-fidelity model outputs and real-time sensor measurements. We are motivated by a temperature data set of the Massachusetts Bay from sensors that can only obtain measurements at the ocean’s surface. We first apply standard principal component analysis (PCA) to a historical reanalysis data set of the time-evolving 3D (x,y,z) temperature field. Next, we train a temporal convolutional neural network (TCN) to predict the time coefficients of the vertical PCA modes, and their variance, as a function of surface temperature. Simultaneously, we estimate the time-evolving 2D (x,y) surface temperature field from the satellite and buoy measurements with multi-fidelity Gaussian process regression (GPR). Finally, the surface temperature is used as input to the neural network to probabilistically predict the PCA coefficients and reconstruct the full 4D temperature field. The results are compared to in-situ measurements at all depths, and the median absolute error is found to be 0.97°C. Overall, the proposed framework is able to produce more accurate models in less time, and the uncertainty quantification can be leveraged for future decision-making.

A Graphical Multi-Fidelity Gaussian Process Model, with Application to Emulation of Heavy-Ion Collisions

With advances in scientific computing and mathematical modeling, complex scientific phenomena such as galaxy formations and rocket propulsion can now be reliably simulated. Such simulations can however be very time-intensive, requiring millions of CPU hours to perform. One solution is multi-fidelity emulation, which uses data of different fidelities to train an efficient predictive model which emulates the expensive simulator. For complex scientific problems and with careful elicitation from scientists, such multi-fidelity data may often be linked by a directed acyclic graph (DAG) representing its scientific model dependencies. We thus propose a new Graphical Multi-fidelity Gaussian Process (GMGP) model, which embeds this DAG structure (capturing scientific dependencies) within a Gaussian process framework. We show that the GMGP has desirable modeling traits via two Markov properties, and admits a scalable algorithm for recursive computation of the posterior mean and variance along at each depth level of the DAG. We also present a novel experimental design methodology over the DAG given an experimental budget, and propose a nonlinear extension of the GMGP via deep Gaussian processes. The advantages of the GMGP are then demonstrated via a suite of numerical experiments and an application to emulation of heavy-ion collisions, which can be used to study the conditions of matter in the Universe shortly after the Big Bang. The proposed model has broader uses in data fusion applications with graphical structure, which we further discuss.

Machine learning modeling of lung mechanics: Assessing the variability and propagation of uncertainty in respiratory-system compliance and airway resistance

Background and ObjectiveTraditional assessment of patient response in mechanical ventilation relies on respiratory-system compliance and airway resistance. Clinical evidence has shown high variability in these parameters, highlighting the difficulty of predicting them before the start of ventilation therapy. This motivates the creation of computational models that can connect structural and tissue features with lung mechanics. In this work, we leverage machine learning (ML) techniques to construct predictive lung function models informed by non-linear finite element simulations, and use them to investigate the propagation of uncertainty in the lung mechanical response. MethodsWe revisit a continuum poromechanical formulation of the lungs suitable for determining patient response. Based on this framework, we create high-fidelity finite element models of human lungs from medical images. We also develop a low-fidelity model based on an idealized sphere geometry. We then use these models to train and validate three ML architectures: single-fidelity and multi-fidelity Gaussian process regression, and artificial neural networks. We use the best predictive ML model to further study the sensitivity of lung response to variations in tissue structural parameters and boundary conditions via sensitivity analysis and forward uncertainty quantification. Codes are available for download at https://github.com/comp-medicine-uc/ML-lung-mechanics-UQ ResultsThe low-fidelity model delivers a lung response very close to that predicted by high-fidelity simulations and at a fraction of the computational time. Regarding the trained ML models, the multi-fidelity GP model consistently delivers better accuracy than the single-fidelity GP and neural network models in estimating respiratory-system compliance and resistance (R2∼0.99). In terms of computational efficiency, our ML model delivers a massive speed-up of ∼970,000× with respect to high-fidelity simulations. Regarding lung function, we observed an almost matched and non-linear behavior between specific structural parameters and chest wall stiffness with compliance. Also, we observed a strong modulation of airways resistance with tissue permeability. ConclusionsOur findings unveil the relevance of specific lung tissue parameters and boundary conditions in the respiratory-system response. Furthermore, we highlight the advantages of adopting a multi-fidelity ML approach that combines data from different levels to yield accurate and efficient estimates of clinical mechanical markers. We envision that the methods presented here can open the way to the development of predictive ML models of the lung response that can inform clinical decisions.

Multiobjective Bayesian Optimization for Aeroengine Using Multiple Information Sources

Aero-engine performance optimization remains significant for both efficiency and safety during specific operating conditions. Previous works usually solve this optimization problem under a single-objective optimization framework, while multiple objectives need to be optimized simultaneously. Besides, the underlying optimization process requires a variety of function evaluations, and the evaluation cost for an aero-engine is expensive. In reality, the aero-engine model has multiple information sources with different costs and accuracy. The different costs and accuracy of the multiple information sources should be traded off to guide the search for the optimal in a cost-efficient way. Therefore, we propose a multi-information source framework for enabling efficient multi-objective Bayesian optimization. We construct the surrogate model with a multi-fidelity Gaussian process and choose the location-source pair with a modified acquisition function. Finally, we apply the proposed method to improve the performance indexes of the aero-engine, which confirms the efficiency of the proposed algorithm.

Prediction of melt pool geometry by fusing experimental and simulation data

Accurate prediction of the melt pool geometry in laser-directed energy deposition (L-DED) is crucial for ensuring product quality. Although machine learning methods have offered aid in addressing this challenge, existing surrogate models are typically based on either simulation or experimental data. Surrogates trained using numerical simulation data reflect modeling assumptions in their accuracy. Conversely, surrogates developed using experimental data would demand a substantial investment of effort and resources to generate adequate data for training. As a possible solution, in this paper, a multi-fidelity Gaussian process (MFGP) framework is developed to fuse experimental and simulation data. Experimental melt pool dimensions are obtained from single-layer, single-track deposits fabricated via powder-fed L-DED. These constitute the high-fidelity/ground truth data for the MFGP surrogate. An analytical Eagar-Tsai model is calibrated and queried at sampled input points within a predefined process parameter window to yield near-accurate estimates of low-fidelity melt pool data at a significantly lower effort. Thereafter, the MFGP surrogate is designed on the combined sources of data using an appropriate kernel and adequate calibration. An elaborate assessment of the trained surrogate on unseen experimental data is shown to yield results with high accuracy and low uncertainty. The proposed method of blending experimental data with simulation data has the potential to reduce the volume of experimental data required for creating surrogates without compromising accuracy. While the current work focuses on the L-DED of a popular alloy, SS316L, it can be easily extended to other advanced manufacturing processes to design reliable surrogates.

Estimation of hydraulic conductivity in a watershed using sparse multi-source data via Gaussian process regression and Bayesian experimental design

Enhanced water management systems depend on accurate estimation of subsurface hydraulic properties. However, geologic formations can vary significantly, so information from a single source (e.g., widely spaced boreholes) is insufficient in characterizing subsurface aquifer properties. Therefore, multiple sources of information are needed to complement the hydrogeology understanding of a region. This study presents a numerical framework in which information from different measurement sources is combined to characterize the 3D random field in a multi-fidelity prediction model. Coupled with the model, a Bayesian experimental design was used to determine the best future sampling locations. The Upper Sangamon watershed in east-central Illinois was selected as the case study site, where the multi-fidelity Gaussian process model was used to estimate the hydraulic conductivity in the region of interest. Multi-source observation data were obtained from electrical resistivity and borehole pumping tests. The accuracy of the model prediction is dependent on the locations and the distribution of both high- and low-fidelity data. Furthermore, the multi-fidelity model was compared with the single-fidelity model. The uncertainties and confidence in the measurements and parameter estimates were quantified and used to design future cycles of data collection to further improve the confidence intervals.

Multifidelity Algorithm for the Sensitivity Analysis of Multidisciplinary Problems

AbstractThe present article proposes an algorithm for the sensitivity analysis of a multidisciplinary problem, in which the derivative-based global sensitivity indices are computed with multifidelity Gaussian process models. Two levels of fidelity are used to estimate the indices, where the low-fidelity samples are obtained by stopping the multidisciplinary analysis solver before convergence. A dedicated refinement strategy for the multifidelity Gaussian process is proposed to ensure the accuracy of the sensitivity index estimation. This algorithm is tested on three multidisciplinary problems of increasing complexity (one analytical and two representative engineering design problems), and proved to be both reliable in detecting the noninfluential variables and computationally efficient, compared to classical Monte Carlo integration and to three other candidate algorithms.

A multi-fidelity Gaussian process for efficient frequency sweeps in the acoustic design of a vehicle cabin

Highly accurate predictions from large-scale numerical simulations are associated with increased computational resources and time expense. Consequently, the data generation process can only be performed for a small sample size, limiting a detailed investigation of the underlying system. The concept of multi-fidelity modeling allows the combination of data from different models of varying costs and complexities. This study introduces a multi-fidelity model for the acoustic design of a vehicle cabin. Therefore, two models with different fidelity levels are used to solve the Helmholtz equation at specified frequencies with the boundary element method. Gaussian processes (GPs) are trained on each fidelity level with the simulation results to predict the unknown system response. In this way, the multi-fidelity model enables an efficient approximation of the frequency sweep for acoustics in the frequency domain. Additionally, the proposed method inherently considers uncertainties due to the data generation process. To demonstrate the effectiveness of our framework, the multifrequency solution is validated with the high-fidelity (HF) solution at each frequency. The results show that the frequency sweep is efficiently approximated by using only a limited number of HF simulations. Thus, these findings indicate that multi-fidelity GPs can be adopted for fast and, simultaneously, accurate predictions.

Multi-fidelity Bayesian optimization to solve the inverse Stefan problem

In this work, we propose an efficient solution of the inverse Stefan problem by multi-fidelity Bayesian optimization. We construct a multi-fidelity Gaussian process surrogate model by combining many low-fidelity estimates of a solidification problem with only a few high-fidelity measurements. To solve the inverse problem, we employ the Gaussian process model in a Bayesian optimization approach based on a multi-fidelity knowledge gradient acquisition function. To account for the specific structure of the target function, we reformulate it as a composite function and thus significantly improve the stability of the optimization procedure. Target values can be switched easily, and previously obtained samples and surrogate models can be reused. The proposed method iteratively improves the recommended solution of the inverse problem. Explicitly adding recommended points of previous iterations to the solution procedure enhances the convergence properties of the algorithm.We demonstrate the applicability of the algorithm by solving the inverse problem for a planar solidification front in a single-fidelity setting. Process parameters are identified for targeted crystal-growth velocities during directional dendritic solidification. The relation between these velocities and process parameters, such as undercooling, thermal diffusivity, capillarity, and capillary anisotropy, defines the thermo-mechanical properties of the solidified part in metal-based additive manufacturing. Material design is based on the corresponding inverse problem. Cost-efficiency of solving the inverse problem is improved by introducing a fidelity hierarchy based on coarse-grid approximations of high-fidelity numerical simulations. The open-source simulation framework ALPACA for multiphase flows allows to generate data at all fidelities. We demonstrate the superior convergence properties of the presented multi-fidelity approach by comparison with an approach solely based on high-fidelity measurements of the tip velocity.

A Batched Bayesian Optimization Approach for Analog Circuit Synthesis via Multi-Fidelity Modeling

Device sizing is a challenging problem for analog circuit design. Traditional methods depend on domain knowledge and intensive simulations to search for feasible parameters. Recent studies apply the Bayesian optimization (BO) and a Gaussian process (GP) model in analog circuit synthesis to improve efficiency. The BO framework automatically selects the parameter candidates by inferring the surrogate GP model. However, naive BO employs a sequential updating strategy which is inefficient in a multicore environment. Besides, the widely used GP model requires costly high fidelity data, which are obtained from fine simulations. In this article, we propose a constrained batch BO approach with a multifidelity (MF) model to solve the above difficulties. The batch BO exploits parallel computing and selects promising parameters by multiple acquisition function ensemble. In addition, the MF GP model adapts the low fidelity data obtained from coarse simulations. Specifically, the proposed method incorporates information gain in a weighted clustering algorithm to refine the parameter candidates. As a result, the proposed method maintains the candidates’ quality and diversity, which speeds up the optimization convergence. In the experiments, we demonstrate the efficiency of the proposed approach on three real-world circuits. The results show that our approach reduces the simulation costs by at least 54.6% compared to the state-of-the-art baselines.

Multi-fidelity hydrodynamic analysis of an autonomous surface vehicle at surveying speed in deep water subject to variable payload

Autonomous surface vehicles (ASV) allow the investigation of coastal areas, ports, and harbors as well as harsh and dangerous environments such as the arctic regions. In the past decade, several studies addressed the shape optimization, control and stability, and seakeeping of ASVs. Nevertheless, the hydrodynamic analysis of ASV performance subject to variable operational parameters lacks a thorough investigation. In this context, this paper presents a multi-fidelity (MF) hydrodynamic analysis of a catamaran ASV, namely the shallow water autonomous multipurpose platform (SWAMP), at surveying speed in calm water and subject to variable payload and location of the center of mass, accounting for the variety of equipment that the vehicle can carry. The analysis is conducted in deep water, which is the condition mostly encountered by the ASV during surveys of coastal and harbor areas. The objective of the analysis is the investigation of the effects of the payload and its arrangement on the variations of the resistance, the vehicle attitude, and the wave generated in the region between the catamaran hulls. These are assessed using an unsteady Reynolds–averaged Navier–Stokes equation (URANSE) code and a linear potential flow (PF) solver. The latter also includes variable grid refinement and coupling between hydrodynamic loads and rigid body equations of motion. Furthermore, the limitation of a PF analysis in the current context is investigated. Finally, a multi-fidelity Gaussian process (MF-GP) model is obtained combining high- (URANSE) and low-fidelity (PF) solutions following an additive correction approach of the latter. The MF-GP model is iteratively refined using an active learning approach based on the maximum prediction uncertainty and the computational cost of the numerical simulations. Numerical results show that the MF-GP is effective in producing response surfaces of the SWAMP performance with a limited computational cost. It is highlighted how the SWAMP performance is significantly affected not only by the payload, but also by the location of the center of mass. The latter can be therefore properly calibrated to minimize the resistance and allow for longer-range operations.

Multifidelity Gaussian Process Emulation for Atmospheric Radiative Transfer Models

Multifidelity Gaussian Process Emulation for Atmospheric Radiative Transfer Models

AN ADAPTIVE STRATEGY FOR SEQUENTIAL DESIGNS OF MULTILEVEL COMPUTER EXPERIMENTS

Investigating uncertainties in computer simulations can be prohibitive in terms of computational costs, since the simulator needs to be run over a large number of input values. Building an emulator, i.e., a statistical surrogate model of the simulator constructed using a design of experiments made of a comparatively small number of evaluations of the forward solver, greatly alleviates the computational burden to carry out such investigations. Nevertheless, this can still be above the computational budget for many studies. Two major approaches have been used to reduce the budget needed to build the emulator: efficient design of experiments, such as sequential designs, and combining training data of different degrees of sophistication in a so-called multifidelity method, or multilevel in case these fidelities are ordered typically for increasing resolutions. We present here a novel method that combines both approaches, the multilevel adaptive sequential design of computer experiments in the framework of Gaussian process (GP) emulators. We make use of reproducing kernel Hilbert spaces as a tool for our GP approximations of the differences between two consecutive levels. This dual strategy allows us to allocate efficiently limited computational resources over simulations of different levels of fidelity and build the GP emulator. The allocation of computational resources is shown to be the solution of a simple optimization problem in a special case where we theoretically prove the validity of our approach. Our proposed method is compared to other existing models of multifidelity Gaussian process emulation. Gains in orders of magnitudes in accuracy or computing budgets are demonstrated in some numerical examples for some settings.

CAMERA: A method for cost-aware, adaptive, multifidelity, efficient reliability analysis

Estimating probability of failure in aerospace systems is a critical requirement for flight certification and qualification. Failure probability estimation involves resolving tails of probability distributions, and Monte Carlo sampling methods are intractable when expensive high-fidelity simulations have to be queried. We propose a method to use models of multiple fidelities that trade accuracy for computational efficiency. Specifically, we propose the use of multifidelity Gaussian process models to efficiently fuse models at multiple fidelity, thereby offering a cheap surrogate model that emulates the original model at all fidelities. Furthermore, we propose a novel sequential acquisition function based experiment design framework that can automatically select samples from appropriate fidelity models to make predictions about quantities of interest at the highest fidelity. We use our proposed approach in an importance sampling setting and demonstrate our method on the failure level set and probability estimation on synthetic test functions and two real-world applications, namely, the reliability analysis of a gas turbine engine blade using a finite element method and a transonic aerodynamic wing test case using Reynolds-averaged Navier-Stokes equations. We show that our method predicts the failure boundary and probability more accurately and at a fraction of the computational cost compared with using just a single expensive high-fidelity model. Finally, we show that our sequential approach is guaranteed to asymptotically converge to the true failure boundary with high probability.

A random forest with multi-fidelity Gaussian process leaves for modeling multi-fidelity data with heterogeneity

Modeling multi-fidelity datasets has been widely used recently. High-fidelity data often suffer from scarcity. Low-fidelity models have abundant observations where information from low-fidelity models can be transferred to high-fidelity models. However, the modeling performance for the multi-fidelity models is below par in most cases due to the heterogeneity of the data. Modeling time is also a critical issue for MF datasets due to high dimension of the data. We propose to frame a multi-fidelity Gaussian process model into a random forest framework to incorporate its flexibility and improve the prediction performance when there are a limited amount of high-fidelity data and the data exhibit heterogeneity in the space of interest. Information extracted from the low-fidelity model can be borrowed for the high-fidelity model by capturing cross-level data correlations. The multi-fidelity model is extended to a tree ensemble structure with an efficient partitioning criterion to tackle data heterogeneity. The proposed method is able to provide uncertainty quantification for predicted values. Numerical examples and case studies are conducted to show the efficacy of our method for the heterogeneous behaviors of the responses across the input space.

Additive Manufacturing Melt Pool Prediction and Classification via Multifidelity Gaussian Process Surrogates

Additive Manufacturing Melt Pool Prediction and Classification via Multifidelity Gaussian Process Surrogates

Multi-Fidelity Gaussian Process Surrogate Modeling of Pediatric Tissue Expansion.

Growth of skin in response to stretch is the basis for tissue expansion (TE), a procedure to gain new skin area for reconstruction of large defects. Unfortunately, complications and suboptimal outcomes persist because TE is planned and executed based on physician's experience and trial and error instead of predictive quantitative tools. Recently, we calibrated computational models of TE to a porcine animal model of tissue expansion, showing that skin growth is proportional to stretch with a characteristic time constant. Here, we use our calibrated model to predict skin growth in cases of pediatric reconstruction. Available from the clinical setting are the expander shapes and inflation protocols. We create low fidelity semi-analytical models and finite element models for each of the clinical cases. To account for uncertainty in the response expected from translating the models from the animal experiments to the pediatric population, we create multifidelity Gaussian process surrogates to propagate uncertainty in the mechanical properties and the biological response. Predictions with uncertainty for the clinical setting are essential to bridge our knowledge from the large animal experiments to guide and improve the treatment of pediatric patients. Future calibration of the model with patient-specific data-such as estimation of mechanical properties and area growth in the operating room-will change the standard for planning and execution of TE protocols.