Multi-fidelity Surrogate Model Research Articles

Multi-fidelity surrogate modeling using long short-term memory networks

When evaluating quantities of interest that depend on the solutions to differential equations, we inevitably face the trade-off between accuracy and efficiency. Especially for parametrized, time-dependent problems in engineering computations, it is often the case that acceptable computational budgets limit the availability of high-fidelity, accurate simulation data. Multi-fidelity surrogate modeling has emerged as an effective strategy to overcome this difficulty. Its key idea is to leverage many low-fidelity simulation data, less accurate but much faster to compute, to improve the approximations with limited high-fidelity data. In this work, we introduce a novel data-driven framework of multi-fidelity surrogate modeling for parametrized, time-dependent problems using long short-term memory (LSTM) networks, to enhance output predictions both for unseen parameter values and forward in time simultaneously — a task known to be particularly challenging for data-driven models. We demonstrate the wide applicability of the proposed approaches in a variety of engineering problems with high- and low-fidelity data generated through fine versus coarse meshes, small versus large time steps, or finite element full order versus deep learning reduced-order models. Numerical results show that the proposed multi-fidelity LSTM networks not only improve single-fidelity regression significantly, but also outperform the multi-fidelity models based on feed-forward neural networks.

Improvement of Mixed-Mode I/II Fracture Toughness Modeling Prediction Performance by Using a Multi-Fidelity Surrogate Model Based on Fracture Criteria.

Today, artificial intelligence plays a huge role in the mechanical engineering field for solving many complex problems and the problem with fracture mechanics is one of them. In fracture mechanics, artificial intelligence is used to predict crack behavior under various conditions such as mixed-mode loading. Many parameters are used for explaining the crack behavior under various conditions, but those parameters are obtained from destructive testing, in which usually, only one data point is obtained from each test. An artificial problem method requires a large amount of data to train the model to be able to learn crack behavior, which is a disadvantage of applying this method to fracture mechanics. To eliminate the disadvantage of the large amount of experiment data required for modeling, in this study, the small data obtained from the experiment along with data obtained from fracture criteria that were used for elementary prediction of mixed mode fracture toughness were used to create an artificial intelligence model. Data from the experiment was combined with fracture criteria data using the multi-fidelity surrogate model that is described in this study. The mixed mode I/II fracture toughness of the PMMA material was tested in order to primarily propose the data combination technique. After the modeling process, the prediction results indicated that the performance of a model in which the actual test data was combined with the data from the fracture criteria (multi-fidelity surrogate model) was more predictively effective compared to only actual data-based modeling.

A reanalysis-based multi-fidelity (RBMF) surrogate framework for efficient structural optimization

In recent years, research on multi-fidelity (MF) surrogate modeling, which integrates high-fidelity (HF) and low-fidelity (LF) models, has been conducted to improve efficiency in structural optimization. However, even the latest well-developed MF surrogate models inevitably require a certain number of samples to maintain the fidelity of the surrogate models. To overcome this issue, in this paper, a reanalysis-based multi-fidelity (RBMF) surrogate framework, which combines the MF surrogate modeling and a structural reanalysis method, is developed to reduce the computational cost for each sample, not the number of samples. The core idea of the developed framework is to approximately obtain a large number of samples based on a small number of exactly calculated data as prior knowledge. Each sub-strategy for RBMF surrogate framework is developed to maximize the performances of the reanalysis method. Specifically, a reanalysis sample classification method, reanalysis sample infilling method, and local convergence criteria are proposed to effectively reflect the characteristics of the reanalysis method. Therefore, the coupling between MF surrogate modeling and reanalysis method is strengthened. Finally, two numerical examples show that the developed framework outperforms the conventional one.

A multi-output multi-fidelity Gaussian process model for non-hierarchical low-fidelity data fusion

Multi-fidelity (MF) surrogate model has been widely used in dealing with computationally expensive problems as it can make a trade-off between computational cost and modeling accuracy. Most existing MF models are under the assumption of a hierarchical relationship between different fidelity models. In other words, the fidelity level of each low-fidelity (LF) model can be clearly identified. However, the fidelity rankings of the LF models are indistinguishable in many practical problems. Hence, the existing MF models may be inapplicable when dealing with multiple non-hierarchical LF data. To address this issue, a multi-fidelity Gaussian process model for multiple non-hierarchical LF data fusion is developed in multi-output scenarios. It focuses on approximating multiple dependent high-fidelity (HF) responses assisted by multiple non-hierarchical LF data. The multi-output Gaussian process modeling technique makes the proposed model able to catch the latent correlations between outputs. In such a structure, both the information of the multiple LF data and the correlations between outputs can be fully exploited to further enhance the prediction accuracy. The effectiveness of the proposed model is tested using two analytical examples and a metamaterial vibration isolator design problem. It is shown that the proposed model gives the best predictions among the tested models for most cases. Results show that the proposed method can be a potential multi-output multi-fidelity modeling approach for computationally expensive problems in dealing with multiple non-hierarchical LF data.

Multi-fidelity surrogate model ensemble based on feasible intervals

Multi-fidelity surrogate model ensemble based on feasible intervals

A probability of improvement-based multi-fidelity robust optimization approach for aerospace products design

Uncertainty is inevitable in aerospace products design optimization, which may lead to the undesired performance of aerospace products and even infeasible designs. Robust design optimization (RDO) aims to obtain a solution with a desirable performance mean and is insensitive to uncertainty. However, it is computationally intensive during the RDO process, which is unaffordable for aerospace product design optimization. Multi-fidelity (MF) surrogate modeling is a widely used strategy to alleviate the heavy computational burden in RDO problems. The existing MF surrogate modeling-based RDO approaches, however, generate the samples in a one-shot way, which requires sufficient samples distributed in the entire design space. In this work, a probability of improvement-based adaptive sampling approach is proposed for multi-fidelity robust design optimization. The multi-level hierarchical Kriging (MHK) model is used for multi-fidelity modeling, which allows the proposed approach to deal with data of multiple fidelities. Both design variable uncertainty and interpolation uncertainty are considered within the approach. An extended PI (EPI) function is developed to simultaneously select the design location and fidelity level of the updated sample. To deal with RDO problems with constraints, the proposed EPI function is extended by combining with the probability of feasibility functions. The proposed approach is demonstrated using four numerical examples and an engineering example of robust design optimization for a micro-aerial vehicle fuselage.

Multi-Fidelity Low-Rank Approximations for Uncertainty Quantification of a Supersonic Aircraft Design

Uncertainty quantification has proven to be an indispensable study for enhancing reliability and robustness of engineering systems in the early design phase. Single and multi-fidelity surrogate modelling methods have been used to replace the expensive high fidelity analyses which must be repeated many times for uncertainty quantification. However, since the number of analyses required to build an accurate surrogate model increases exponentially with the number of random input variables, most surrogate modelling methods suffer from the curse of dimensionality. As an alternative approach, the Low-Rank Approximation method can be applied to high-dimensional uncertainty quantification studies with a low computational cost, where the number of coefficients for building the surrogate model increases only linearly with the number of random input variables. In this study, the Low-Rank Approximation method is implemented for multi-fidelity applications with additive and multiplicative correction approaches to make the high-dimensional uncertainty quantification analysis more efficient and accurate. The developed uncertainty quantification methodology is tested on supersonic aircraft design problems and its predictions are compared with the results of single- and multi-fidelity Polynomial Chaos Expansion and Monte Carlo methods. For the same computational cost, the Low-Rank Approximation method outperformed both in surrogate modeling and uncertainty quantification cases for all the benchmarks and real-world engineering problems addressed in the present study.

Integrated optimization design for horizontal well spacing and fracture stage placement in shale gas reservoir

Horizontal well drilling and hydraulic fracturing technologies play an essential role in improving gas recovery from shale reservoirs. However, they are expensive technologies and resource intensive production strategies. Optimization design of horizontal well spacing and fracture stage placement helps in striking a balance between gas production and economic benefits. However, previous researches relied mainly on numerical simulation technique which is computationally expensive and time-consuming. To reduce the computational burden, a novel multi-fidelity support vector regression (MFSVR) surrogate model assisted horizontal well spacing and fracture stage placement integrated optimization method, namely WSF-MFSVR, is proposed in this study. In the WSF-MFSVR method, both low-fidelity (LF) and high-fidelity (HF) numerical simulation models were applied to establish the multi-fidelity (MF) surrogate model so as to lessen the computational burden and guarantee its quality. In order to enhance the evaluation accuracy, the particle swarm optimization (PSO) algorithm was adopted to find the optimal hyper-parameters of the MFSVR model. Two cases with different wells and fracture types based on the shale gas reservoir with Barnett shale properties were employed to verify the WSF-MFSVR method. The results indicated that a combination of 300 H F and 3500 LF samples was the most suitable for establishing the MFSVR model to approximate the numerical simulation model. In terms of computational efficiency, the WSF-MFSVR method was about 50 times faster than the HF numerical simulation model-based method. Furthermore, the relative hyper-area difference (RHD) and overall spread (OS) of the WSF-MFSVR method were similar to that of the HF numerical simulation model-based method. However, the RHD and OS of the WSF-MFSVR method were superior to that of the LF numerical simulation model-based and single-fidelity support vector regression model-assisted methods. The data of these indexes quantitatively showed the superior convergence and diversity of the final optimal solutions obtained by WSF-MFSVR method.

A surrogate-assisted evolutionary algorithm with hypervolume triggered fidelity adjustment for noisy multiobjective integer programming

Although surrogate-assisted evolutionary algorithms (SAEAs) have been widely developed to address computationally expensive multi-objective optimization problems (MOPs), they still encounter difficulties in solving the expensive and noisy combinatorial MOPs. To this end, we propose a novel SAEA to handle this kind of problem. In the proposed algorithm, the averaging method is used to denoise. To balance the conflict between the time cost and the effect of noises, multi-fidelity surrogate models are constructed according to the averaged evaluation results. The number of independent repeated evaluations represents the fidelity level of surrogate models. In the optimization process, the hypervolume indicator is employed as a trigger to determine whether the fidelity level should be increased. In addition, a lightweight local search method, the semi-variable neighborhood search, is proposed to improve the global search efficiency of the proposed algorithm in discrete decision spaces. Experimental results show that our proposed algorithm achieves competitive performance on most benchmark problems.

An optimized numerical tornado simulator and its application to transient wind-induced response of a long-span bridge

The long-span bridges in the tornado-prone area would be potentially struck by extreme transient winds. In this study, the tornado-like wind field is simulated by the numerical Ward-type tornado simulator based on computational fluid dynamics (CFD) techniques. To minimize the discrepancy between the simulated and field-measured tornado winds, the optimization strategy is developed to achieve optimal parameters of the numerical Ward-type tornado simulator, namely the inflow angle (θ) and translation speed (VT). To facilitate the optimization process, a multi-fidelity surrogate model is utilized to effectively integrate both low-fidelity and high-fidelity data for accurate and efficient simulations. Specifically, the cokriging model is constructed by the CFD data associated with both low-cost Reynolds-averaged Navier-Stokes (RANS) equations and high-cost large-eddy simulation (LES) techniques. The “best” parameters (i.e., θ and VT) based on the multi-fidelity surrogate model are input to the numerical Ward-type tornado simulator (using LES technique), and the obtained wind field matches excellently with the field measurements. Finally, the transient wind field generated using the validated numerical Ward-type tornado simulator is employed as the dynamic inputs to the finite element (FE) model of a long-span bridge, and the results highlight the important contribution of the transient bridge aerodynamics.

Modified Multifidelity Surrogate Model Based on Radial Basis Function with Adaptive Scale Factor

Multifidelity surrogates (MFSs) replace computationally intensive models by synergistically combining information from different fidelity data with a significant improvement in modeling efficiency. In this paper, a modified MFS (MMFS) model based on a radial basis function (RBF) is proposed, in which two fidelities of information can be analyzed by adaptively obtaining the scale factor. In the MMFS, an RBF was employed to establish the low-fidelity model. The correlation matrix of the high-fidelity samples and corresponding low-fidelity responses were integrated into an expansion matrix to determine the scaling function parameters. The shape parameters of the basis function were optimized by minimizing the leave-one-out cross-validation error of the high-fidelity sample points. The performance of the MMFS was compared with those of other MFS models (MFS-RBF and cooperative RBF) and single-fidelity RBF using four benchmark test functions, by which the impacts of different high-fidelity sample sizes on the prediction accuracy were also analyzed. The sensitivity of the MMFS model to the randomness of the design of experiments (DoE) was investigated by repeating sampling plans with 20 different DoEs. Stress analysis of the steel plate is presented to highlight the prediction ability of the proposed MMFS model. This research proposes a new multifidelity modeling method that can fully use two fidelity sample sets, rapidly calculate model parameters, and exhibit good prediction accuracy and robustness.

Multi-fidelity surrogate modeling through hybrid machine learning for biomechanical and finite element analysis of soft tissues

Biomechanical simulation enables medical researchers to study complex mechano-biological conditions, although for soft tissue modeling, it may apply highly nonlinear multi-physics theories commonly implemented by expensive finite element (FE) solvers. This is a significantly time-consuming process on a regular computer and completely inefficient in urgent situations. One remedy is to first generate a dataset of the possible inputs and outputs of the solver in order to then train an efficient machine learning (ML) model, i.e., the supervised ML-based surrogate, replacing the expensive solver to speed up the simulation. But it still requires a large number of expensive numerical samples. In this regard, we propose a hybrid ML (HML) method that uses a reduced-order model defined by the simplification of the complex multi-physics equations to produce a dataset of the low-fidelity (LF) results. The surrogate then has this efficient numerical model and an ML model that should increase the fidelity of its outputs to the level of high-fidelity (HF) results. Based on our empirical tests via a group of diverse training and numerical modeling conditions, the proposed method can improve training convergence for very limited training samples. In particular, while considerable time gains comparing to the HF numerical models are observed, training of the HML models is also significantly more efficient than the purely ML-based surrogates. From this, we conclude that this non-destructive HML implementation may increase the accuracy and efficiency of surrogate modeling of soft tissues with complex multi-physics properties in small data regimes.

SCGAN: stacking-based generative adversarial networks for multi-fidelity surrogate modeling

SCGAN: stacking-based generative adversarial networks for multi-fidelity surrogate modeling

Seismic reliability analysis using a multi-fidelity surrogate model: Example of base-isolated buildings

An approach is presented for conducting simulation-based seismic reliability analysis using a multi-fidelity surrogate model. To illustrate the approach, a base-isolated building is considered comprising a reinforced concrete frame for primary building and lead-rubber bearings for isolation devices. The high-fidelity model is a nonlinear finite-element model used in time history analysis. Ground motions are generated from a parametric power spectral density function that is compatible with a target response spectrum at the site of interest. The low-fidelity model uses a linear-elastic stick model for the superstructure, while stochastic linearization is used to capture the nonlinearity of the base isolation. Cokriging is used as a multi-fidelity surrogate model for fusing a large number of low-fidelity model evaluations with few high-fidelity model evaluations, in order to attain high-fidelity model accuracy while mitigating the computational cost. After training the Cokriging model, Monte Carlo simulation is carried out by sampling the cheap multi-fidelity surrogate, and the probability of failure is subsequently evaluated for different response thresholds. The relative error in failure probability prediction of Cokriging is shown to be in the range of 6% with a maximum of 10%, for a critical range of sample response variable threshold values, in contrast to errors as high as 60% provided by the low-fidelity model.

Nonlinear Multi-Fidelity Bayesian Optimization: An Application in the Design of Blast Mitigating Structures

<div class="section abstract"><div class="htmlview paragraph">A common scenario in engineering design is the availability of several black-box functions that describe an event with different levels of accuracy and evaluation cost. Solely employing the highest fidelity, often the most expensive, black-box function leads to lengthy and costly design cycles. Multi-fidelity modeling improves the efficiency of the design cycle by combining information from a small set of observations of the high-fidelity function and large sets of observations of the low-fidelity, fast-to-evaluate functions. In the context of Bayesian optimization, the most popular multi-fidelity model is the auto-regressive (AR) model, also known as the co-kriging surrogate. The main building block of the AR model is a weighted sum of two Gaussian processes (GPs). Therefore, the AR model is well suited to exploit information generated by sources that present strong linear correlations. Recently, the non-linear auto-regressive Gaussian process (NARGP) model has appeared as an alternative to integrate information generated by non-linearly correlated black-box functions. The performance of the NARGP model in structural optimization has remained largely unexplored. This investigation presents a Bayesian optimization approach that implements the NARGP model as the multi-fidelity surrogate model. The optimization strategy is utilized in the design sandwich composite armors for blast mitigation. The armors are made of four layers: steel, carbon fiber reinforced polymer (CFRP), aluminum honeycomb (HC), and CFRP. The optimization problem has three design variables, which are the thickness of the CFRP and aluminum HC layers. Two objectives are minimized: the armor’s penetrations and the reaction force at the armor’s supports. The black-box functions are two finite element models with different levels of fidelity. The low-fidelity model assumes elastic behavior of the sandwich composite. The high-fidelity model considers the nonlinear behavior of each layer of the armor. The results show that the proposed non-linear multi-fidelity Bayesian optimization approach produces a more stable expansion of the Pareto front than an optimization strategy that employs the AR model. This outcome suggests that the NARGP model is an appealing alternative in design problems with a limited number of function evaluations of the high-fidelity source.</div></div>

A new multi-fidelity surrogate modelling method for engineering design based on neural network and transfer learning

PurposeCompared with the low-fidelity model, the high-fidelity model has both the advantage of high accuracy, and the disadvantage of low efficiency and high cost. A series of multi-fidelity surrogate modelling method were developed to give full play to the respective advantages of both low-fidelity and high-fidelity models. However, most multi-fidelity surrogate modelling methods are sensitive to the amount of high-fidelity data. The purpose of this paper is to propose a multi fidelity surrogate modelling method whose accuracy is less dependent on the amount of high-fidelity data.Design/methodology/approachA multi-fidelity surrogate modelling method based on neural networks was proposed in this paper, which utilizes transfer learning ideas to explore the correlation between different fidelity datasets. A low-fidelity neural network was built by using a sufficient amount of low-fidelity data, which was then finetuned by a very small amount of HF data to obtain a multi-fidelity neural network based on this correlation.FindingsNumerical examples were used in this paper, which proved the validity of the proposed method, and the influence of neural network hyper-parameters on the prediction accuracy of the multi-fidelity model was discussed.Originality/valueThrough the comparison with existing methods, case study shows that when the number of high-fidelity sample points is very small, the R-square of the proposed model exceeds the existing model by more than 0.3, which shows that the proposed method can be applied to reducing the cost of complex engineering design problems.

Multi-Fidelity Sparse Polynomial Chaos and Kriging Surrogate Models Applied to Analytical Benchmark Problems

In this article, multi-fidelity kriging and sparse polynomial chaos expansion (SPCE) surrogate models are constructed. In addition, a novel combination of the two surrogate approaches into a multi-fidelity SPCE-Kriging model will be presented. Accurate surrogate models, once obtained, can be employed for evaluating a large number of designs for uncertainty quantification, optimization, or design space exploration. Analytical benchmark problems are used to show that accurate multi-fidelity surrogate models can be obtained at lower computational cost than high-fidelity models. The benchmarks include non-polynomial and polynomial functions of various input dimensions, lower dimensional heterogeneous non-polynomial functions, as well as a coupled spring-mass-system. Overall, multi-fidelity models are more accurate than high-fidelity ones for the same cost, especially when only a few high-fidelity training points are employed. Full-order PCEs tend to be a factor of two or so worse than SPCES in terms of overall accuracy. The combination of the two approaches into the SPCE-Kriging model leads to a more accurate and flexible method overall.

Stochastic multi-fidelity surrogate modeling of dendritic crystal growth

In this work, we propose a novel framework coupling state-of-the-art multi-fidelity Gaussian Process modeling techniques with input-space warping for a cost-efficient construction of a stochastic surrogate model. During model generation, we achieve high computational efficiency by combining a large number of cheap estimates (low-fidelity model) with only a few, computationally expensive, high-fidelity measurements. We base the fidelity hierarchy on coarse-grid approximations of high-fidelity numerical simulations and show its successful application within the proposed framework. Utilizing coarse-grid approximations for multi-fidelity modeling is attractive for many practical applications, since it often allows for multi-fidelity data generation with a single simulator. As benchmark, we apply this framework to generate a surrogate model for crystal growth velocities in directional dendritic solidification. The derivation of a relation between this tip velocity and process parameters, such as undercooling, thermal diffusivity, capillarity, and capillary anisotropy, has been in the focus of research for decades due to its important role on microstructure evolution during solidification. It defines the thermo-mechanical properties of the solidified part and influences its behavior in subsequent manufacturing steps.As data generator, we use the open-source simulation framework ALPACA, applying a conservative sharp-interface level-set model. We assess the accuracy of the multi-fidelity tip velocity model by using cross-validation techniques. Compared to single-fidelity models purely based on high-fidelity data, our approach improves prediction accuracy significantly but only requires a little cost overhead for data generation. The stochastic nature of the resulting surrogate model allows for quantifying the uncertainty associated with predictions. This motivates the application of the model in Bayesian-optimization algorithms for inverse problems. Also, it may serve as input for microstructure simulations which rely on accurate relations between local solidification velocities and process parameters such as undercooling to predict grain-scale crystalline structures and which need material-dependent model calibration.

Building a Lightweight Digital Twin of a Crane Boom for Structural Safety Monitoring Based on a Multifidelity Surrogate Model

Abstract Undetected fatigue and overload damages at the key locations of the crane boom are among the biggest threats in construction, leading to structural failure. Thus, the structural health of the crane boom should be monitored in real time to ensure that it works under the designed load capacity. In this study, we developed a lightweight digital twin by the multifidelity surrogate (MFS) model to improve the real-time monitoring and prediction accuracy of the structural safety of a crane boom. Digital twin technology, which can establish real-time mapping between the physical space and the digital space, has a promising potential for online monitoring and analysis of structures, equipment, and even human bodies. By combining the MFS model and sensor data, the lightweight digital twin can dynamically mirror the crane boom postures and predict its structural performance in real time. In this study, the structural analysis of the crane boom is limited to the linear elastic stage of materials. Numerical experiments showed that the accuracy of the lightweight digital twin was enhanced compared with that established by the single-fidelity surrogate model, and the computational cost of the lightweight digital twin was decreased with respect to the digital twin built by the numerical method. Meanwhile, the uncertainty from the physical space was analyzed to enhance the reliability of the lightweight digital twin. Thus, the lightweight digital twin developed in our work can ensure accurate safety prediction and design optimization for crane booms.

Design Rule Extraction Using Multi-Fidelity Surrogate Model for Unmanned Combat Aerial Vehicles

Due to the instability of the pitching moment induced by the strong vortical flow at moderate to high angles of attack in the unmanned combat aerial vehicle (UCAV) configuration, previous design studies focused primarily on low-speed flight conditions where the longitudinal stability is a serious issue. Furthermore, because of the limitation of computational cost, univariate analyses based on low-fidelity flow solvers are performed to investigate the relationships between design variables and quantities of interest. In this study, since the high-speed condition may occupy a high proportion in the UCAV mission, it was additionally considered through multi-objective optimization. A high-fidelity flow solver is used for both low-speed and high-speed conditions to accurately capture the vortical flow structure, and corresponding computational costs are relieved by multi-fidelity surrogate modeling. Based on this surrogate model, design rules are extracted, and the effects of design variables are investigated through univariate and multivariate data mining techniques. At last, universal design rules that can be referenced regardless of the parameterization method are visualized in an airfoil shape using proper orthogonal decomposition. The time-efficient design rule extraction framework and the approach to analyzing extracted design rules presented in this study can be extended to any engineering field.