Construction Of Surrogate Models Research Articles

Multiple Tensor Train Approximation of Parametric Constitutive Equations in Elasto-Viscoplasticity

This work presents a novel approach to construct surrogate models of parametric differential algebraic equations based on a tensor representation of the solutions. The procedure consists of building simultaneously an approximation given in tensor-train format, for every output of the reference model. A parsimonious exploration of the parameter space coupled with a compact data representation allows alleviating the curse of dimensionality. The approach is thus appropriate when many parameters with large domains of variation are involved. The numerical results obtained for a nonlinear elasto-viscoplastic constitutive law show that the constructed surrogate model is sufficiently accurate to enable parametric studies such as the calibration of material coefficients.

Efficient multi-response adaptive sampling algorithm for construction of variable-fidelity aerodynamic tables

Adaptive sampling is an iterative process for the construction of a global approximation model. Most of engineering analysis tools computes multiple parameters in a single run. This research proposes a novel multi-response adaptive sampling algorithm for simultaneous construction of multiple surrogate models in a time-efficient and accurate manner. The new algorithm uses the Jackknife cross-validation variance and a minimum distance metric to construct a sampling criterion function. A weighted sum of the function is used to consider the characteristics of multiple surrogate models. The proposed algorithm demonstrates good performance on total 22 numerical problems in comparison with three existing adaptive sampling algorithms. The numerical problems include several two-dimensional and six-dimensional functions which are combined into single-response and multi-response systems. Application of the proposed algorithm for construction of aerodynamic tables for 2D airfoil is demonstrated. Scaling-based variable-fidelity modeling is implemented to enhance the accuracy of surrogate modeling. The algorithm succeeds in constructing a system of three highly nonlinear aerodynamic response surfaces within a reasonable amount of time while preserving high accuracy of approximation.

On Nonintrusive Uncertainty Quantification and Surrogate Model Construction in Particle Accelerator Modeling

Using a cyclotron-based model problem, we demonstrate for the first time the applicability and usefulness of an uncertainty quantification (UQ) approach in order to construct surrogate models. The ...

Construction of surrogate model for prediction of crack propagation using deep learning

ILSVRC 2012における深層学習(ディープラーニング)の有効性が画像認識において示されその可能性に多くの人々が期待する状況になって久しい。しかし，計算力学やCAEといった分野こそ，ディープラーニングとの親和性が高いと期待するもののその応用方法・利用方法については先行研究事例すら少ない状況である。著者らはいくつかの基本的な問題を通じディープラーニングのCAEやシミュレーション自体の置き換え(サロゲートモデルの構築)を念頭にその可能性を模索してきた。本稿では，サロゲートモデル構築へ向けた大まかなニューラルネットワーク構築の指針(対象とする問題とニューラルネットワークの関係に関する仮定)を紹介し，き裂進展サロゲートモデルの可能性について言及する。

Methodology of Robust Optimal Design of the Multistage Axial Compressor on the Basis of Discrete Analogue-Related Data

Consideration was given to the formulation and methodology of the direct nonlinear design problem of structural dimensional circuits in conditions of a priori parametric uncertainty. It was shown that the solution of the direct nonlinear design problem of structural dimensional circuits can be reduced to the multicriteria problems of stochastic optimization with mixed conditions. Mathematical models and methods of the solution of multicriteria problems of stochastic optimization with mixed conditions were analyzed. A.N. Tikhonov method of the regulation of solution of incorrectly formulated problems was used as a computation method for the synthesis of quasi-solutions of incorrectly formulated problems. An effective memetic algorithm for the synthesis of the solutions of MV-problems was suggested. It is based on the genetic algorithm combined with the randomized path tracing method. The suggested methodology allows us to search for rational solutions of the multicriteria problems of system modification through the creation of hierarchic two-level scheme for the synthesis of solutions that includes the construction of robust surrogate models of the systems and processes with subsequent robust estimation of sought values for the parametric data uncertainty. As an example, we obtained the solution data for the problem of robust optimal design of two-stage axial compressor in conditions of a stochastic nature of the input data obtained using the interactive decision support computer system «Concept_Pro_St ® ».

Sensitivity-Driven Adaptive Construction of Reduced-space Surrogates

Surrogate modeling has become a critical component of scientific computing in situations involving expensive model evaluations. However, training a surrogate model can be remarkably challenging and even computationally prohibitive in the case of intensive simulations and large-dimensional systems. We develop a systematic approach for surrogate model construction in reduced input parameter spaces. A sparse set of model evaluations in the original input space is used to approximate derivative based global sensitivity measures (DGSMs) for individual uncertain inputs of the model. An iterative screening procedure is developed that exploits DGSM estimates in order to identify the unimportant inputs. The screening procedure forms an integral part of an overall framework for adaptive construction of a surrogate in the reduced space. The framework is tested for computational efficiency through an initial implementation in simple test cases such as the classic Borehole function, and a semilinear elliptic PDE with a random source function. The framework is then deployed for a realistic application from chemical kinetics, where we study the ignition delay in an $$\hbox {H}_2{/}\hbox {O}_2$$ reaction mechanism with 19 and 33 uncertain rate-controlling parameters. It is observed that significant computational gains can be attained by constructing accurate low-dimensional surrogates using the proposed framework.

Multi-fidelity model based optimization of shaped film cooling hole and experimental validation

An optimization framework is developed for the film cooling hole shape design to achieve highest film cooling effectiveness with affordable cost. A key to this design method is the development and applications of a multi-fidelity model (MFM) to improve the efficiency of surrogate model construction. To construct multi-fidelity model, empirical correlations from existing literature are utilized as low-fidelity data and Reynolds-Averaged Navier-Stokes (RANS) simulations provide high-fidelity data. Compared with single-fidelity surrogate models, multi-fidelity model developed in this work combines vast existing knowledge about shaped film cooling hole and three dimensional numerical simulation. As a result, the computation cost has been substantially decreased by 64.5% while accuracy is maintained with similar level, which provides a firm basis for the optimization. Considering three geometric parameters as design variables, i.e. laidback angel, lateral angle, and hole length-diameter ratio, the shaped hole is optimized with genetic algorithm (GA) combined with sequential quadratic programming (SQP) algorithm. After the optimization, an experiment campaign is further carried out to validate the optimization result using pressure sensitive paint (PSP) technology. With current multi-fidelity model, the spatially averaged film cooling effectiveness has been improved by 39% compared to the baseline, which is an already well-designed shaped hole and the improvement is also verified by experiment. This work verifies the effectiveness of current multi-fidelity model for the design and optimization of shaped film cooling hole for better performance.

Surrogate definition and homogeneous chemical kinetic model for two alkane-rich FACE gasoline fuels

A surrogate formulation methodology is proposed by directly using functional groups , , , and phenyl to build the surrogate models for the FACE (fuels for advanced combustion engines) A and C gasoline fuels. The surrogates were formulated with the mixture of n-butane/n-heptane/iso-octane/2,5-dimethylhexane for FACE A, and with the mixture of n-butane/n-heptane/iso-octane/2,5-dimethylhexane/toluene for FACE C gasoline fuel. Based on the kinetic mechanisms of the candidate components, the surrogate fuel models were assembled through a hierarchical construction way. In order to get a simpler surrogate model, the detailed mechanism of 2,5-dimethylhexane has been reduced by the path flux analysis (PFA) method and optimised by updating the reaction rate constants of selected reactions before the surrogated fuel model construction. Thereafter, the accuracy of the surrogate models was demonstrated by comparing the model predictions against ignition delay measurements for FACE A and C gasoline fuels in a wide range of conditions (10–40 bar; 0.5–1 equivalence ratios), and the results show that the presented surrogate fuel models can reproduce the ignition behaviours better especially in the low-temperature conditions. The surrogate chemical kinetic model construction method can be used in other fuels to get simpler and lower computational cost surrogate fuel models.

Pumping Optimization of Coastal Aquifers Using Seawater Intrusion Models of Variable-Fidelity and Evolutionary Algorithms

Variable-fidelity modelling has been utilized in several engineering optimization studies to construct surrogate models. However, similar approaches have received much less attention in coastal aquifer management problems. A variable-fidelity optimization framework was developed utilizing a lower-fidelity and computationally cheap model of seawater intrusion, based on the sharp interface assumption, and a simple correction process. The variable-fidelity method was compared to the direct optimization with the high-fidelity variable density and salt transport model and to conventional surrogate-based optimization. The surrogate-based approaches were embedded into the operations of an evolutionary algorithm to implement an efficient online update of the surrogate models and control the feasibility of the optimal solutions. Multiple independent optimization runs were performed to provide more insightful comparison outcomes. Although the variable-fidelity method found a better optimum than the conventional approach, the overall sample statistics showed that the surrogate-based optimization frameworks performed equally well and provided good approximations to the high-fidelity solution. Despite the potential for an improved exploration of the optimal search space by using the variable-fidelity method, the conventional approach had a 30% faster average convergence time.

Fast predictive multi-fidelity prediction with models of quantized fidelity levels

In this paper, we introduce a novel approach for the construction of multi-fidelity surrogate models with “discrete” fidelity levels. The notion of a discrete level of fidelity is in contrast to a mathematical model, for which the notion of refinement towards a high-fidelity model is relevant to sending a discretization parameter toward zero in a continuous way. Our notion of discrete fidelity levels encompasses cases for which there is no notion of convergence in terms of a fidelity parameter that can be sent to zero or infinity. The particular choice of how levels of fidelity are defined in this framework paves the way for using models that may have no apparent physical or mathematical relationship to the target high-fidelity model. However, our approach requires that models can produce results with a common set of parameters in the target model. Hence, fidelity level in this work is not directly representative of the degree of similarity of a low-fidelity model to a target high-fidelity model. In particular, we show that our approach is applicable to competitive ecological systems with different numbers of species, discrete-state Markov chains with a different number of states, polymer networks with a different number of connections, and nano-particle plasmonic arrays with a different number of scatterers. The results of this study demonstrate that our procedure boasts computational efficiency and accuracy for a wide variety of models and engineering systems.

Bayesian approach to determine critical dimensions from scatterometric measurements

Supplement one of the ‘Guide to the Expression of Uncertainties in Measurements (GUM)’ recommends to estimate measurement uncertainties by Monte Carlo calculations. For computationally expensive models, Monte Carlo sampling can be often only realized for a small number of samples. This may cause inaccurate or even useless results for problems with a slow convergence rate. In this paper we apply a recently developed method that enables the Bayesian approach to determine the geometry of photo masks from measurements in extreme ultraviolet (EUV) scatterometry. The key idea of this approach is the construction of an efficient surrogate model which reduces the computational costs of sampling steps by several orders of magnitude. This allows us to use Markov Chain Monte Carlo sampling to determine the posterior distribution of desired critical dimensions of the photo mask geometry. The Bayesian approach leads to substantially larger uncertainties of the critical dimension than the previously employed maximum likelihood approach. The results for critical dimensions and line edge roughness are consistent with independent atomic force measurements of a test photo mask arrays from which scatterometric data were used.

Estimating the optimum size of a tidal array at a multi-inlet system considering environmental and performance constraints

This paper investigates the optimum tidal energy converter array density at a tidal inlet by applying surrogate-based optimisation. The SBO procedure comprises problem formulation, design of experiments, numerical simulations, surrogate model construction and constrained optimisation. This study presents an example for the Faro-Olhão Inlet in the Ria Formosa (Portugal), a potential site for tidal in-stream energy extraction. A 35 kW Evopod™ floating tidal energy converter from Oceanflow Energy Ltd. has been used for array size calculations considering two design variables: (1) number of array rows, and (2) number of tidal energy converter per row. Arrays up to 13 rows with 6–11 tidal energy converters each are studied to assess their impacts on array performance, inlets discharges and bathymetry changes. The analysis identified the positive/negative feedbacks between the two design variables in real case complex flow fields under variable bathymetry and channel morphology. The non-uniformity of tidal currents along the array region causes the variability of the resource in each row, as well as makes it difficult to predict the resultant array configuration interactions. Four different multi-objective optimisation models are formulated subject to a set of performance and environmental constraints. Results from the optimisation models imply that the largest array size that meets the environmental constraints is made of 5 rows with 6 tidal energy converter each and an overall capacity factor of 11.6% resulting in an energy production of 1.01 GWh year−1. On the other hand, a higher energy production (1.20 GWh year−1) is achieved by an optimum array configuration, made of 3 rows with 10 tidal energy converters per row, which maximises power output satisfying environmental and performance restrictions. This optimal configuration permits a good level of energy extraction while having a reduced effect on the hydrodynamic functioning of the multi-inlet system. These results prove the suitability and the potential wide use of the surrogate-based optimisation method to define array characteristics that enhance power production and at the same time respect the environmental surrounding conditions.

Gradient-based optimization for regression in the functional tensor-train format

Predictive analysis of complex computational models, such as uncertainty quantification (UQ), must often rely on using an existing database of simulation runs. In this paper we consider the task of performing low-multilinear-rank regression on such a database. Specifically we develop and analyze an efficient gradient computation that enables gradient-based optimization procedures, including stochastic gradient descent and quasi-Newton methods, for learning the parameters of a functional tensor-train (FT). We compare our algorithms with 22 other nonparametric and parametric regression methods on 10 real-world data sets and show that for many physical systems, exploiting low-rank structure facilitates efficient construction of surrogate models. We use a number of synthetic functions to build insight into behavior of our algorithms, including the rank adaptation and group-sparsity regularization procedures that we developed to reduce overfitting. Finally we conclude the paper by building a surrogate of a physical model of a propulsion plant on a naval vessel.

Surrogate-based parameter inference in debris flow model

This work tackles the problem of calibrating the unknown parameters of a debris flow model with the drawback that the information regarding the experimental data treatment and processing is not available. In particular, we focus on the evolution over time of the flow thickness of the debris with dam-break initial conditions. The proposed methodology consists of establishing an approximation of the numerical model using a polynomial chaos expansion that is used in place of the original model, saving computational burden. The values of the parameters are then inferred through a Bayesian approach with a particular focus on inference discrepancies that some of the important features predicted by the model exhibit. We build the model approximation using a preconditioned non-intrusive method and show that a suitable prior parameter distribution is critical to the construction of an accurate surrogate model. The results of the Bayesian inference suggest that utilizing directly the available experimental data could lead to incorrect conclusions, including the over-determination of parameters. To avoid such drawbacks, we propose to base the inference on few significant features extracted from the original data. Our experiments confirm the validity of this approach, and show that it does not lead to significant loss of information. It is further computationally more efficient than the direct approach, and can avoid the construction of an elaborate error model.

Domain segmentation for low‐cost surrogate‐assisted multi‐objective design optimisation of antennas

Information regarding the best possible design trade-offs of an antenna structure can be obtained through multi-objective optimisation (MO). Unfortunately, MO is extremely challenging if full-wave electromagnetic (EM) simulation models are used for performance evaluation. Yet, for the majority of contemporary antennas, EM analysis is the only tool that ensures reliability. This study introduces a procedure for accelerated MO design of antennas that exploits cheap data-driven surrogates, as well as MO evolutionary algorithm to yield an initial approximation of the Pareto set. The final Pareto set is found using output space mapping. The major contribution of this work is a further reduction of the computational cost of surrogate model construction using a design space segmentation mechanism, where a part of the space containing the Pareto front is first identified by means of single-objective optimisation runs and subsequently represented by a set of adjacent compartments with separate surrogate models established within them. Segmentation allows for significant reduction of the number of training points necessary to build a reliable surrogate and thus the overall optimisation cost. The presented technique is demonstrated using two antenna structures and compared with a state-of-the-art surrogate-assisted techniques. Experimental validation is also provided.

CMA evolution strategy assisted by kriging model and approximate ranking

The covariance matrix adaptation evolution strategy (CMA-ES) is a competitive evolutionary algorithm (EA) for difficult continuous optimization problems. However, expensive function evaluation of many real-world optimization problems poses a serious challenge to the application of CMA-ES (and other EAs) to these problems. To address this challenge, surrogate-assisted EAs has attracted increasing attention and become popular. In this paper, a new surrogate-assisted CMA-ES algorithm in which Kriging model is used to enhance CMA-ES via approximate ranking procedure is proposed. In the proposed algorithm, the approximate ranking procedure which estimates the rank of current population by using Kriging model and the exact fitness function together is adopted. In addition, the confidence interval method of training set selection is introduced for surrogate model construction. An initial sampling is performed before entering the evolution loop. In each iteration (generation), after the population sampling, the approximate ranking procedure is called instead of the original fitness evaluation, then, parameters of the sampling distribution are updated. This iterative search process continues until the target fitness is reached or the computational budget is exhausted. The proposed algorithm and confidence interval method of training set selection are analyzed through experimental study. The results demonstrate that the confidence interval method works well in Kriging-assisted CMA-ES, and that the proposed algorithm significantly reduces the number of function evaluations of CMA-ES and outperforms the Kriging-assisted CMA-ES using pre-selection and generation-based control on the tested problems.

Fast predictive models based on multi-fidelity sampling of properties in molecular dynamics simulations

In this paper we introduce a novel approach for enhancing the sampling convergence for properties predicted by molecular dynamics. The proposed approach is based upon the construction of a multi-fidelity surrogate model using computational models with different levels of accuracy. While low fidelity models produce result with a lower level of accuracy and computational cost, in this framework they can provide the basis for identification of the optimal sparse sampling pattern for high fidelity models to construct an accurate surrogate model. Such an approach can provide a significant computational saving for the estimation of the quantities of interest for the underlying physical/engineering systems. In the present work, this methodology is demonstrated for molecular dynamics simulations of a Lennard-Jones fluid. Levels of multi-fidelity are defined based upon the integration time step employed in the simulation. The proposed approach is applied to two different canonical problems including (i) single component fluid and (ii) binary glass-forming mixture. The results show about 70% computational saving for the estimation of averaged properties of the systems such as total energy, self diffusion coefficient, radial distribution function and mean squared displacements with a reasonable accuracy.

Accelerated scale-bridging through adaptive surrogate model evaluation

Multiscale modeling is a systematic approach for the development of high-fidelity models of complex systems. However, multiscale models are often extremely computationally demanding, which precludes their use for practical applications. In this article, we introduce a computational framework for scale-bridging combined with an algorithm to automatically and adaptively replace at-scale models within a multiscale model hierarchy with surrogate models in order to reduce the computational cost of multiscale simulations. A standalone module is introduced and it is responsible for the on-the-fly construction and evaluation of surrogate models within the framework. Such an approach allows multiscale models to easily incorporate surrogate models with minimal code modifications. We employ the framework to construct a multiscale model of 1,3,5-trinitrohexahydro-s-triazine, in which a continuum finite element macroscale solver acquires equation of state through evaluation of a microscale dissipative particle dynamics model. We utilize the model for the simulation of a Taylor impact experiment and demonstrate that the error in the solution incurred by the dynamic use of surrogate models is controllable. Furthermore, we show that the use of surrogate models leads to a reduction in computational cost of between 1/20 and 1/5000 compared to a simulation evaluated without the surrogate modeling approach. In addition, we present a high-resolution simulation of a Taylor impact experiment, which is intractable without surrogate models. We illustrate how the dynamic nature of surrogate model evaluation in these simulations, while reducing computational cost, also increases load imbalance. Finally, we end with a discussion on how the inherent variability in these simulations may constitute a challenge for the current high performance computer systems given their static nature.

A classification approach to efficient global optimization in presence of non-computable domains

Gaussian-Process based optimization methods have become very popular in recent years for the global optimization of complex systems with high computational costs. These methods rely on the sequential construction of a statistical surrogate model, using a training set of computed objective function values, which is refined according to a prescribed infilling strategy. However, this sequential optimization procedure can stop prematurely if the objective function cannot be computed at a proposed point. Such a situation can occur when the search space encompasses design points corresponding to an unphysical configuration, an ill-posed problem, or a non-computable problem due to the limitation of numerical solvers. To avoid such a premature stop in the optimization procedure, we propose to use a classification model to learn non-computable areas and to adapt the infilling strategy accordingly. Specifically, the proposed method splits the training set into two subsets composed of computable and non-computable points. A surrogate model for the objective function is built using the training set of computable points, only, whereas a probabilistic classification model is built using the union of the computable and non-computable training sets. The classifier is then incorporated in the surrogate-based optimization procedure to avoid proposing new points in the non-computable domain while improving the classification uncertainty if needed. The method has the advantage to automatically adapt both the surrogate of the objective function and the classifier during the iterative optimization process. Therefore, non-computable areas do not need to be a priori known. The proposed method is applied to several analytical problems presenting different types of difficulty, and to the optimization of a fully nonlinear fluid-structure interaction system. The latter problem concerns the drag minimization of a flexible hydrofoil with cavitation constraints. The efficiency of the proposed method compared favorably to a reference evolutionary algorithm, except for situations where the feasible domain is a small portion of the design space.

Gradient-enhanced kriging for high-dimensional problems

Surrogate models provide an affordable alternative to the evaluation of expensive deterministic functions. However, the construction of accurate surrogate models with many independent variables is currently prohibitive because they require a large number of function evaluations for the desired accuracy. Gradient-enhanced kriging has the potential to reduce the number of evaluations when efficient gradient computation, such as an adjoint method, is available. However, current gradient-enhanced kriging methods do not scale well with the number of sampling points because of the rapid growth in the size of the correlation matrix, where new information is added for each sampling point in each direction of the design space. Furthermore, they do not scale well with the number of independent variables because of the increase in the number of hyperparameters that must be estimated. To address this issue, we develop a new gradient-enhanced surrogate model approach that drastically reduces the number of hyperparameters through the use of the partial least squares method to maintain accuracy. In addition, this method is able to control the size of the correlation matrix by adding only relevant points defined by the information provided by the partial least squares method. To validate our method, we compare the global accuracy of the proposed method with conventional kriging surrogate models on two analytic functions with up to 100 dimensions, as well as engineering problems of varied complexity with up to 15 dimensions. We show that the proposed method requires fewer sampling points than conventional methods to obtain the desired accuracy, or it provides more accuracy for a fixed budget of sampling points. In some cases, we get models that are over three times more accurate than previously developed surrogate models for the same computational time, and over 3200 times faster than standard gradient-enhanced kriging models for the same accuracy.