Multi-fidelity Gaussian Process Regression Research Articles

Machine learning framework for the real-time reconstruction of regional 4D ocean temperature fields from historical reanalysis data and real-time satellite and buoy surface measurements

The ability to model 4D properties of near-coast ocean regions in real time is valuable for both general environmental monitoring and for specialized industry applications. We present a computationally parsimonious framework to build improved spatiotemporal models that combine dynamics inferred from historical high-fidelity model outputs and real-time sensor measurements. We are motivated by a temperature data set of the Massachusetts Bay from sensors that can only obtain measurements at the ocean’s surface. We first apply standard principal component analysis (PCA) to a historical reanalysis data set of the time-evolving 3D (x,y,z) temperature field. Next, we train a temporal convolutional neural network (TCN) to predict the time coefficients of the vertical PCA modes, and their variance, as a function of surface temperature. Simultaneously, we estimate the time-evolving 2D (x,y) surface temperature field from the satellite and buoy measurements with multi-fidelity Gaussian process regression (GPR). Finally, the surface temperature is used as input to the neural network to probabilistically predict the PCA coefficients and reconstruct the full 4D temperature field. The results are compared to in-situ measurements at all depths, and the median absolute error is found to be 0.97°C. Overall, the proposed framework is able to produce more accurate models in less time, and the uncertainty quantification can be leveraged for future decision-making.

Machine learning modeling of lung mechanics: Assessing the variability and propagation of uncertainty in respiratory-system compliance and airway resistance

Background and ObjectiveTraditional assessment of patient response in mechanical ventilation relies on respiratory-system compliance and airway resistance. Clinical evidence has shown high variability in these parameters, highlighting the difficulty of predicting them before the start of ventilation therapy. This motivates the creation of computational models that can connect structural and tissue features with lung mechanics. In this work, we leverage machine learning (ML) techniques to construct predictive lung function models informed by non-linear finite element simulations, and use them to investigate the propagation of uncertainty in the lung mechanical response. MethodsWe revisit a continuum poromechanical formulation of the lungs suitable for determining patient response. Based on this framework, we create high-fidelity finite element models of human lungs from medical images. We also develop a low-fidelity model based on an idealized sphere geometry. We then use these models to train and validate three ML architectures: single-fidelity and multi-fidelity Gaussian process regression, and artificial neural networks. We use the best predictive ML model to further study the sensitivity of lung response to variations in tissue structural parameters and boundary conditions via sensitivity analysis and forward uncertainty quantification. Codes are available for download at https://github.com/comp-medicine-uc/ML-lung-mechanics-UQ ResultsThe low-fidelity model delivers a lung response very close to that predicted by high-fidelity simulations and at a fraction of the computational time. Regarding the trained ML models, the multi-fidelity GP model consistently delivers better accuracy than the single-fidelity GP and neural network models in estimating respiratory-system compliance and resistance (R2∼0.99). In terms of computational efficiency, our ML model delivers a massive speed-up of ∼970,000× with respect to high-fidelity simulations. Regarding lung function, we observed an almost matched and non-linear behavior between specific structural parameters and chest wall stiffness with compliance. Also, we observed a strong modulation of airways resistance with tissue permeability. ConclusionsOur findings unveil the relevance of specific lung tissue parameters and boundary conditions in the respiratory-system response. Furthermore, we highlight the advantages of adopting a multi-fidelity ML approach that combines data from different levels to yield accurate and efficient estimates of clinical mechanical markers. We envision that the methods presented here can open the way to the development of predictive ML models of the lung response that can inform clinical decisions.

The fusion of flow field data with multiple fidelities

We propose a spatial-temporal multi-fidelity Gaussian process regression framework for the fusion of flow field data with various availabilities and fidelities but not sufficiently large to train neural networks commonly encountered in fluid mechanics studies. For example, fluid experiments lead to data with high fidelity but sparse in time and space, while most of the numerical data are generally regarded as less accurate but are spatially temporally continuous. The proposed framework aims at generating a new set of fused data by combining the merits of those in the spatial-temporal space. Numerical simulations [e.g., direct numerical simulation (DNS), large eddy simulation, Reynolds-averaged Navier–Stokes] of flow around a National Advisory Committee for Aeronautics 0012 airfoil are performed to collect the original raw data with various fidelities, and a fraction of the DNS result is used to mimic the high-fidelity but sparse experimental data. It is found that the accuracy of the fused data increases with the density of high-fidelity points until reaching a threshold, above which the fusion accuracy becomes insensitive. This limit can be overcome by introducing extra dimensions, such as the gradients of the low-fidelity data field. By examining the error fields, it is found that the high-fidelity points can tune low-fidelity fields but only within a limited local region. The accuracy can be firmly improved by introducing more high-fidelity points or higher levels of spatial gradients if the data set captures the temporal development.

Active learning of tandem flapping wings at optimizing propulsion performance

In the present work, we propose an optimization framework based on the active learning method, which aims to quickly determine the conditions of tandem flapping wings for optimal performance in terms of thrust or efficiency. Especially, multi-fidelity Gaussian process regression is used to establish the surrogate model correlating the kinematic parameters of tandem flapping wings and their aerodynamic performances. Moreover, the Bayesian optimization algorithm is employed to select new candidate points and update the surrogate model. With this framework, the parameter space can be explored and exploited adaptively. Two optimization tasks of tandem wings are carried out using this surrogate-based framework by optimizing thrust and propulsion efficiency. The response surfaces predicted from the updated surrogate model present the influence of the flapping frequency, phase, and separation distance on thrust and efficiency. It is found that the time-average thrust of the hind flapping wing increases with the frequency. However, the increase in frequency may lead to a decrease in propulsive efficiency in some circumstances.

Multifidelity regression of sparse plasma transport data available in disparate physical regimes.

Physical data are typically generated by experiments and computations in limited parameter regimes. When datasets generated using such disparate methods are combined into one dataset, the resulting dataset is typically sparse, with dense "islands" in a potentially high-dimensional parameter space, and predictions must be interpolated among such islands. Using plasma transport data as our example, we propose a multifidelity Gaussian-process regression framework that incorporates physical data from multiple sources at multiple fidelities. The impact of the proposed framework varies from little improvement over simpler approaches to qualitatively changing the prediction with consistently increased confidence in regions lacking high-fidelity data. By varying low- and high-fidelity data sources, we demonstrate an approach for determining when multifidelity Gaussian-process regression adds value over single-fidelity regression and therefore when its additional computational costs are merited. We also examine the case in which the outputs of the low- and high-fidelity models correspond to different physical quantities, one of which may be intrinsically computationally cheaper to compute. We conclude by analyzing strategies for sampling high-fidelity data for use in this framework, and we develop a simple sampling approach for reducing regression error across large gaps in data.

A hybrid solution for offshore wind resource assessment from limited onshore measurements

In wind resource assessments, which are critical to the pre-construction of wind farms, measurements by LiDARs or masts are a source of high-fidelity data, but are expensive and scarce in space and time, particularly for offshore sites. On the other hand, numerical simulations, using for example the Weather Research and Forecasting (WRF) model, generate temporally and spatially continuous data with relatively low-fidelity. A hybrid approach is proposed here to combine the merit of measurements and simulations for the assessment of offshore wind. Firstly a temporal data fusion using deep Multi Fidelity Gaussian Process Regression (MF-GPR) is performed to combine the intermittent measurement and the continuous simulation data at an onshore location. Then a spatial data fusion using a neural network with Non-linear Autoregression (NAR) and Non-linear Autoregression with external input (NARX) are conducted to project the wind from onshore to offshore. The numerical and measured wind speeds along the west coast of Denmark were used to evaluate the method. We show that the proposed data fusion technique using a gappy onshore measurement results in accurate offshore wind resource assessment within a 2% margin error.

Multifidelity Data Fusion via Gradient-Enhanced Gaussian Process Regression

We propose a data fusion method based on multi-fidelity Gaussian process regression (GPR) framework. This method combines available data of the quantity of interest (QoI) and its gradients with different fidelity levels, namely, it is a Gradient-enhanced Cokriging method (GE-Cokriging). It provides the approximations of both the QoI and its gradients simultaneously with uncertainty estimates. We compare this method with the conventional multi-fidelity Cokriging method that does not use gradients information, and the result suggests that GE-Cokriging has a better performance in predicting both QoI and its gradients. Moreover, GE-Cokriging even shows better generalization result in some cases where Cokriging performs poorly due to the singularity of the covariance matrix. We demonstrate the application of GE-Cokriging in several practical cases including reconstructing the trajectories and velocity of an underdamped oscillator with respect to time simultaneously, and investigating the sensitivity of power factor of a load bus with respect to varying power inputs of a generator bus in a large scale power system. We also show that though GE-Cokriging method requires a little bit higher computational cost than Cokriging method, the result of accuracy comparison shows that this cost is usually worth it.

Multi-fidelity design optimisation strategy under uncertainty with limited computational budget

In this work, a design optimisation strategy is presented for expensive and uncertain single- and multi-objective problems. Computationally expensive design fitness evaluations prohibit the application of standard optimisation techniques and the direct calculation of risk measures. Therefore, a surrogate-assisted optimisation framework is presented. The computational budget limits the number of high-fidelity simulations which makes impossible to accurately approximate the landscape. This motivates the use of cheap low-fidelity simulations to obtain more information about the unexplored locations of the design space. The information stemming from numerical experiments of various fidelities can be fused together with multi-fidelity Gaussian process regression to build an accurate surrogate model despite the low number of high-fidelity simulations. We propose a new strategy for automatically selecting the fidelity level of the surrogate model update. The proposed method is extended to multi-objective applications. Although, Gaussian processes can inherently model uncertain processes, here the deterministic input and uncertain parameters are treated separately and only the design space is modelled with a Gaussian process. The probabilistic space is modelled with a polynomial chaos expansion to allow also uncertainties of non-Gaussian type. The combination of the above techniques allows us to efficiently carry out a (multi-objective) design optimisation under uncertainty which otherwise would be impractical.

A non-intrusive multifidelity method for the reduced order modeling of nonlinear problems

We propose a non-intrusive reduced basis (RB) method for parametrized nonlinear partial differential equations (PDEs) that leverages models of different accuracy. From a collection of low-fidelity (LF) snapshots, parameter locations are extracted for the evaluations of high-fidelity (HF) snapshots to recover a reduced basis. Multi-fidelity Gaussian process regression (GPR) is employed to approximate the combination coefficients of the reduced basis. LF data is assimilated either via projection onto an LF basis or via an interpolation approach inspired by bifidelity reconstruction. The correlation between HF and LF data is modeled with hyperparameters whose values are automatically determined in the regression step. The proposed methods not only leverage the assimilated LF data to reduce the cost of the offline phase, but also allow for a fast evaluation during the online stage, independent of the computational cost of neither the low-nor the high-fidelity solution. Numerical studies demonstrate the effectiveness of the proposed approach on manufactured examples and problems in nonlinear structural mechanics. Clear benefits of using lower resolution models rather than reduced physics models are observed in both the basis selection and the regression step. An active learning scheme is used for additional snapshot selection at locations with high error. The speed-up in the online evaluation and the high accuracy of extracted quantities of interest makes the multifidelity RB method a powerful tool for outer-loop applications in engineering, as exemplified in uncertainty quantification.

Multi-fidelity Gaussian process based empirical potential development for Si:H nanowires

ABSTRACT In material modeling, the calculation speed using the empirical potentials is fast compared to the first principle calculations, but the results are not as accurate as of the first principle calculations. First principle calculations are accurate but slow and very expensive to calculate. In this work, first, the H-H binding energy and H2-H2 interaction energy are calculated using the first principle calculations which can be applied to the Tersoff empirical potential. Second, the H-H parameters are estimated. After fitting H-H parameters, the mechanical properties are obtained. Finally, to integrate both the low-fidelity empirical potential data and the data from the high-fidelity first-principle calculations, the multi-fidelity Gaussian process regression is employed to predict the H-H binding energy and the H2-H2 interaction energy. Numerical results demonstrate the accuracy of the developed empirical potentials.

Multifidelity kinematic parameter optimization of a flapping airfoil.

We construct a multifidelity framework for the kinematic parameter optimization of flapping airfoil. We employ multifidelity Gaussian process regression and Bayesian optimization to effectively synthesize the aerodynamic performance of the flapping airfoil with the kinematic parameters under multiresolution numerical simulations. The objective of this work is to demonstrate that the multifidelity framework can efficiently discover the optimal kinematic parameters of the flapping airfoil with specific aerodynamic performance using a limited number of expensive high-fidelity simulations combined with a larger number of inexpensive low-fidelity simulations. We efficiently identify the optimal kinematic parameters of an asymmetrically flapping airfoil with various target aerodynamic forces in the design space of heaving amplitude, flapping frequency, angle of attack amplitude, and stroke angle. Notably, it is found that the angle of attack can significantly affect the magnitude of aerodynamic forces by facilitating the generation of the leading-edge vortex. In the meanwhile, its combination effect with the stroke angle can determine the attitude and trajectory of the flapping airfoil, thus further affect the direction of the aerodynamic forces. With the influence of the streamwise in-line motion, the asymmetrical vortex structures emerge in the wake fields because the streamwise velocities of shedding vortices are different in the upstroke and downstroke. Furthermore, we conduct the kinematic parameter optimization for a three-dimensional asymmetrically flapping wing. Compared to the two-dimensional simulations, we further investigate the flow induced by the vortex ring and its unsteady effects on the vortex structure and aerodynamic performance.

Propagation of uncertainty in the mechanical and biological response of growing tissues using multi-fidelity Gaussian process regression

A key feature of living tissues is their capacity to remodel and grow in response to environmental cues. Within continuum mechanics, this process can be captured with the multiplicative split of the deformation gradient into growth and elastic contributions. The mechanical and biological response during tissue adaptation is characterized by inherent variability. Accounting for this uncertainty is critical to better understand tissue mechanobiology, and, moreover, it is of practical importance if we aim to develop predictive models for clinical use. However, the current gold standard in computational models of growth and remodeling remains the use of deterministic finite element (FE) simulations. Here we focus on tissue expansion, a popular technique in which skin is stretched by a balloon-like device inducing its growth. We construct FE models of tissue expansion with various levels of detail, and show that a sufficiently broad set of FE simulations from these models can be used to train an accurate and efficient multi-fidelity Gaussian process (GP) surrogate. The approach is not limited to simulation data, rather, it can fuse different kinds of data, including from experiments. The main appeal of the framework relies on the common experience that highly detailed models (or experiments) are more accurate but also more costly, while simpler models (or experiments) can be easily evaluated but are bound to have some error. In these situations, doing uncertainty analysis tasks with the high fidelity models alone is not feasible and, conversely, relying solely on low fidelity approximations is also undesirable. We show that a multi-fidelity GP outperforms the high fidelity GP and low fidelity GP when tested against the most detailed FE model. In turn, having trained the multi-fidelity GP model, we showcase the propagation of uncertainty from the mechanical and biological response parameters to the spatio-temporal growth outcomes. We expect that the methods and applications in this paper will enable future research in parameter calibration under uncertainty and uncertainty propagation in real clinical scenarios involving tissue growth and remodeling.

Machine learning in drug development: Characterizing the effect of 30 drugs on the QT interval using Gaussian process regression, sensitivity analysis, and uncertainty quantification

Prolonged QT intervals are a major risk factor for ventricular arrhythmias and a leading cause of sudden cardiac death. Various drugs are known to trigger QT interval prolongation and increase the proarrhythmic potential. Yet, how precisely the action of drugs on the cellular level translates into QT interval prolongation on the whole organ level remains insufficiently understood. Here we use machine learning techniques to systematically characterize the effect of 30 common drugs on the QT interval. We combine information from high fidelity three-dimensional human heart simulations with low fidelity one-dimensional cable simulations to build a surrogate model for the QT interval using multi-fidelity Gaussian process regression. Once trained and cross-validated, we apply our surrogate model to perform sensitivity analysis and uncertainty quantification. Our sensitivity analysis suggests that compounds that block the rapid delayed rectifier potassium current IKr have the greatest prolonging effect of the QT interval, and that blocking the L-type calcium current ICaL and late sodium current INaL shortens the QT interval. Our uncertainty quantification allows us to propagate the experimental variability from individual block-concentration measurements into the QT interval and reveals that QT interval uncertainty is mainly driven by the variability in IKr block. In a final validation study, we demonstrate an excellent agreement between our predicted QT interval changes and the changes observed in a randomized clinical trial for the drugs dofetilide, quinidine, ranolazine, and verapamil. We anticipate that both the machine learning methods and the results of this study will have great potential in the efficient development of safer drugs.

Improving SWATH Seakeeping Performance using Multi-Fidelity Gaussian Process and Bayesian Optimization

We construct a multi-fidelity framework for statistical learning and global optimization that is capable of effectively synthesizing seakeeping predictions having two different levels of modeling fidelity, namely a strip theory and a boundary element method based on potential flow assumption. The objective of this work is to demonstrate that the multi-fidelity framework can be used efficiently to discover optimal small waterplane area twin hull shapes having superior seakeeping performance using a limited number of expensive high-fidelity simulations combined with a larger number of inexpensive low-fidelity simulations. Specifically, we employ multi-fidelity Gaussian process regression and Bayesian optimization to build probabilistic surrogate models and efficiently explore a 35-dimensional design space to optimize hull shapes that minimize wave-induced motions and accelerations, and satisfy specific requirements in terms of displacement and metacentric height. Our results demonstrate the superior characteristics of this optimization framework in constructing accurate surrogate models and identifying optimal designs with a significant reduction in the computational effort. 1. Introduction During the past decades, estimating the seaworthiness of a ship in the early design stages has become a primary concern for naval architects. Increased requirements in terms of comfort and ergonomics have steered the research in developing innovative hull forms, with the specific target of decreasing motions in waves. From a safety point of view, extreme accelerations can exert harmful dynamic loads (on the vessel, cargo, or equipment), slamming, or green water effects that can severely damage the structural integrity of the vessel or lead to stability losses. When ship behavior in waves becomes a quantity of interest in the hull-form optimization process, seakeeping performance needs to be predicted with numerical models able to combine high fidelity and high computational efficiency. Nowadays, ship motion predictions mainly rely on three families of numerical models, here sorted by increasing level of fidelity: 2-D strip theories, 3-D boundary element methods (BEM), both developed under the assumption of potential flow, and unsteady fully viscous nonlinear 3-D methods in which ship motions are simulated in six Degrees of Freedom (DOF) for incident regular or irregular waves (see Fig. 5). In this article, we introduce a probabilistic method for constructing surrogate models using multi-fidelity training datasets that allows to increase the accuracy of the model with significant savings in computational resources. For the purpose of demonstration of the multi-fidelity framework, the training datasets used in this article are composed of classical 2-D and 3-D potential flow predictions; for this reason, we will refer to 2-D strip theories as low-fidelity models and a more accurate 3-D BEM as high-fidelity models. In the present study, our goal is to demonstrate the ability of the multi-fidelity framework in efficiently discovering hull forms with superior seakeeping characteristics by using simplified prediction models. Although in the present study, we do not include any viscous effects, the methodology is general and can combine any type of high-fidelity simulations or experimental data with low-fidelity simulations, experimental data, or even empirical correlations.

Multi-fidelity optimization of super-cavitating hydrofoils

We present an effective multi-fidelity framework for shape optimization of super-cavitating hydrofoils using viscous solvers. We employ state-of-the-art machine learning tools such as multi-fidelity Gaussian process regression and Bayesian optimization to synthesize data obtained from multi-resolution simulations, and efficiently identify optimal configurations in the design space. We validate our simulation results against experimental data, and showcase the efficiency of the proposed work-flow in a realistic design problem involving the shape optimization of a three-dimensional super-cavitating hydrofoil parametrized by 17 design variables.

Multi-fidelity Gaussian process regression for prediction of random fields

We propose a new multi-fidelity Gaussian process regression (GPR) approach for prediction of random fields based on observations of surrogate models or hierarchies of surrogate models. Our method builds upon recent work on recursive Bayesian techniques, in particular recursive co-kriging, and extends it to vector-valued fields and various types of covariances, including separable and non-separable ones. The framework we propose is general and can be used to perform uncertainty propagation and quantification in model-based simulations, multi-fidelity data fusion, and surrogate-based optimization. We demonstrate the effectiveness of the proposed recursive GPR techniques through various examples. Specifically, we study the stochastic Burgers equation and the stochastic Oberbeck–Boussinesq equations describing natural convection within a square enclosure. In both cases we find that the standard deviation of the Gaussian predictors as well as the absolute errors relative to benchmark stochastic solutions are very small, suggesting that the proposed multi-fidelity GPR approaches can yield highly accurate results.