Construction Of Surrogate Models Research Articles

Hybrid uncertain buckling analysis for engineering structures through machine learning method

Buckling analysis is fundamental yet essential for engineering structures in stability assessment, design optimization, structural design, and the like. Uncertainty as an inherent feature in engineering significantly affects structural buckling behaviour. A hybrid uncertain buckling analysis accounting aleatoric and epistemic uncertainties simultaneously is introduced. To determine the statistical features of the relevant bounds on the concerned structural response, an optimization strategy is developed based on a surrogate model. For surrogate model construction, an improved machine-learning technique is developed by possessing the global trend regression and local information fitting. A convex optimization program can be formulated to capture the global trend with optimal solutions, and subsequently, local information fitting improves the accuracy in handling unbiased datasets. The sampling method on random variables led the hybrid uncertain buckling analysis to be solved as a series of interval analyses on the established surrogate model. Essential statistical data regarding the limits of critical structural reactions can be assessed both effectively and efficiently. Additional features, such as information updates, further underscores the viability of the proposed scheme. Finally, the crucial buckling loads of two engineering structures facing hybrid uncertainty are thoroughly examined to highlight the potential benefits of the introduced method.

Fast EM-driven nature-inspired optimization of antenna input characteristics using response features and variable-resolution simulation models

Utilization of optimization technique is a must in the design of contemporary antenna systems. Often, global search methods are necessary, which are associated with high computational costs when conducted at the level of full-wave electromagnetic (EM) models. In this study, we introduce an innovative method for globally optimizing reflection responses of multi-band antennas. Our approach uses surrogates constructed based on response features, smoothing the objective function landscape processed by the algorithm. We begin with initial parameter space screening and surrogate model construction using coarse-discretization EM analysis. Subsequently, the surrogate evolves iteratively into a co-kriging model, refining itself using accumulated high-fidelity EM simulation results, with the infill criterion focusing on minimizing the predicted objective function. Employing a particle swarm optimizer (PSO) as the underlying search routine, extensive verification case studies showcase the efficiency and superiority of our procedure over benchmarks. The average optimization cost translates to just around ninety high-fidelity EM antenna analyses, showcasing excellent solution repeatability. Leveraging variable-resolution simulations achieves up to a seventy percent speedup compared to the single-fidelity algorithm.

An active learning method using deep adversarial autoencoder-based sufficient dimension reduction neural network for high-dimensional reliability analysis

Reliability analysis often requires time-consuming evaluations, especially when dealing with high-dimensional and nonlinear problems. To address this challenge, surrogate model methods are frequently employed. One way to improve the efficiency of surrogate model methods involves selecting informative samples that significantly enhance the accuracy of the surrogate model. This paper introduces a novel approach to facilitate the construction of surrogate models and selection of informative samples in high-dimensional reliability analysis, through an active learning method based on a deep adversarial autoencoder-based sufficient dimension reduction (AAE-SDR) neural network. The AAE-SDR neural network serves as a surrogate model, transforming complex high-dimensional variables into tractable, low-dimensional embeddings relevant to the target. These embeddings are Gaussian-distributed with a distinct latent limit state boundary. A new sampling strategy is proposed to select informative misclassified samples by iteratively identifying candidate samples near the latent limit state boundary and uniformly sampling from the candidate sample dataset based on the latent Gaussian distribution. The effectiveness of the proposed approach is demonstrated through two high-dimensional numerical examples and a cable-stayed bridge case study. Results show that the proposed method simplifies complex high-dimensional reliability problems and provides a relatively accurate estimated failure probability with a limited number of samples.

An improved optimization method combining particle swarm optimization and dimension reduction kriging surrogate model for high-dimensional optimization problems

An improved optimization method is proposed which combines the particle swarm optimization (PSO) algorithm with the dimension reduction kriging surrogate model (DK), namely PSO + DK. In this method, a surrogate model is dynamically constructed and updated between the lower dimensional principal components and the response, instead of between the high-dimensional design variables and the response as in traditional methods. The key point of DK is that the principal components analysis (PCA) is integrated with the active learning kriging surrogate model (AK). Since PCA can reduce the dimension effectively, DK makes surrogate model construction possible for complex high-dimensional optimization problems. Numerical examples and four classical engineering problems are presented to validate the effectiveness of the proposed method. The results show that the proposed method can decrease the computational cost significantly while guaranteeing the precision compared with PSO and PSO + AK for high-dimensional optimization problems.

Cross-Scale Reliability Analysis Framework for LNG Storage Tanks Considering Concrete Material Uncertainty

The reliability of liquefied natural gas (LNG) storage tanks is an important factor that must be considered in their structural design. Concrete is a core component of LNG storage tanks, and the geometric uncertainty of concrete aggregate material has a significant impact on their reliability. However, owing to the significant size difference between the concrete aggregate compared to the LNG storage tank, structural analysis using an accurate finite element model that includes all the geometric characteristics of the aggregate incurs significant analytical costs. In particular, for reliability analysis requiring a large number of samples, the computational costs incurred by finite element models are infeasible. Therefore, a dual acceleration strategy based on the asymptotic homogenization method and surrogate model technology is proposed to improve the efficiency of LNG storage tank reliability analysis. In the cross-scale analysis of a LNG storage tank based on asymptotic homogenization, order reduction of the LNG storage tank analysis model was realized. Based on this, a surrogate model construction method with the aggregate fraction and mass moment as inputs was proposed to further accelerate the reliability analysis of LNG storage tanks. Subsequently, a Monte Carlo method was used to perform a reliability analysis of the LNG storage tank considering the uncertainty of the concrete aggregate geometry and distribution under the action of liquid weight and wind load. The analysis showed that the wind load has a significant influence on the safety of the design of the roof of a LNG storage tank. The directionality of the wind load has a significant impact on the distribution of the sample point response for reliability analysis and the failure mode of the LNG storage tank. Owing to the directionality of the wind load, the response distributions of the maximum displacement and maximum stress of LNG were more concentrated, and the reliability of the LNG storage tank decreased after considering the wind load. In particular, the stress reliability of the tank decreased by 5.86%. When only the liquid load was considered, the maximum displacement and stress exhibited asynchronous failure, and the two almost never occurred simultaneously. When the wind load was considered, the failure mode of the LNG storage tank was dominated by the maximum stress. Moreover, the numerical example also demonstrated that the degree of freedom involved in structural analysis, as well as the time of structural analysis can be significantly reduced. So, the proposed cross-scale analysis framework can significantly improve the efficiency of reliability analysis. The conclusions established in this study provide theoretical and methodological guidance for the reliable design of LNG storage tanks.

CSP-free adaptive Kriging surrogate model method for reliability analysis with small failure probability

In the field of reliability engineering, the active learning reliability method that amalgamates Kriging model and Monte Carlo simulation has been devised and proven to be efficacious in reliability analysis. Nevertheless, the performance of this method is sensitive to the magnitude of candidate sample pool, particularly for systems with small failure probabilities. To surmount these limitations, this paper proposes an active learning method that obviates the need for candidate sample pools. The proposed method comprises two stages: construction of surrogate model and Monte Carlo simulation for failure probability estimation. During the surrogate model construction stage, the surrogate model is iteratively refined based on the representative samples selected by solving the optimization problem facilitated by the particle swarm optimization algorithm. To achieve an optimal balance between solution accuracy and efficiency, the penalty intensity control and the density control for the experimental design points are incorporated to modify the objective function in optimization. The performance of the proposed method is evaluated using numerical examples, and results indicate that by leveraging an optimization algorithm to select representative samples, the proposed method overcomes the limitations of traditional active learning methods based on candidate sample pool and exhibits exceptional performance in addressing small failure probabilities.

Perspectives of physics-based machine learning strategies for geoscientific applications governed by partial differential equations

Abstract. An accurate assessment of the physical states of the Earth system is an essential component of many scientific, societal, and economical considerations. These assessments are becoming an increasingly challenging computational task since we aim to resolve models with high resolutions in space and time, to consider complex coupled partial differential equations, and to estimate uncertainties, which often requires many realizations. Machine learning methods are becoming a very popular method for the construction of surrogate models to address these computational issues. However, they also face major challenges in producing explainable, scalable, interpretable, and robust models. In this paper, we evaluate the perspectives of geoscience applications of physics-based machine learning, which combines physics-based and data-driven methods to overcome the limitations of each approach taken alone. Through three designated examples (from the fields of geothermal energy, geodynamics, and hydrology), we show that the non-intrusive reduced-basis method as a physics-based machine learning approach is able to produce highly precise surrogate models that are explainable, scalable, interpretable, and robust.

A construction strategy of Kriging surrogate model based on Rosenblatt transformation of associated random variables and its application in groundwater remediation

When using simulation-optimization models for optimizing the design of groundwater pumping-treatment plans for pollution, building a surrogate model for the numerical simulation model has become an effective means of overcoming the computational load of such models. However, previous studies often treated pumping time as a single optimization variable, leading to unnecessary excessive pumping. This paper considers the location, pumping rate, start time, and end time of each candidate pumping well as optimization variables, and proposes a Rosenblatt-transform-based optimal Latin hypercube sampling method for the associated random variables to ensure that the start time is less than or equal to the end time. This method is coupled with an adaptive sampling method based on batch local optimal solutions to construct a dynamic adaptive Kriging surrogate model for the numerical model, ensuring that the true value of the optimal remediation scheme is not lost. The results show that, at the final stage of remediation, the pollutant concentration in the 4 scenarios achieves comprehensive compliance. However, when considering the minimization of remediation costs as the evaluation criterion, the remediation scheme in scenario 1 (the pumping start and end times are independent optimization variables for all candidate pumping wells) is optimal. In the optimization design of groundwater pumping-treatment plans, the pumping wells should be arranged in the midstream and downstream regions of the contaminant plume and parallel to the regional flow direction. This paper provides a method reference for the construction and adaptive updating of surrogate models involving associated random variables, as well as guidance for the dynamic optimization of groundwater pumping and treatment at contaminated sites.

Stochastic domain decomposition based on variable-separation method

This work proposes a stochastic domain decomposition method for solving steady-state partial differential equations (PDEs) with random inputs. Specifically, based on the efficiency of the variable-separation (VS) method in simulating stochastic partial differential equations (SPDEs), we extend it to stochastic algebraic systems and employ it in the context of stochastic domain decomposition. The resulting method, termed stochastic domain decomposition based on variable-separation (SDD-VS) alleviates the challenge commonly known as the “curse of dimensionality” notably by leveraging explicit representations of stochastic functions derived from physical systems. The primary objective of the proposed SDD-VS method is to obtain a separate representation of the solution for the stochastic interface problem. To enhance computational efficiency, we introduce a two-phase approach consisting of offline and online computation. In the offline phase, we establish an affine representation of stochastic algebraic systems by systematically applying the VS method. During the online phase, we estimate the interface unknowns of SPDEs using a quasi-optimal separated representation, facilitating the construction of efficient surrogate models for subproblems. We substantiate the effectiveness of our proposed approach through numerical experiments involving three specific instances, demonstrating its capability to provide accurate solutions.

Conditional Karhunen–Loève regression model with Basis Adaptation for high-dimensional problems: Uncertainty quantification and inverse modeling

We propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems as a function of the systems’ spatially heterogeneous parameter fields, with applications to uncertainty quantification and parameter estimation in high-dimensional problems. Practitioners often formulate finite-dimensional representations of spatially heterogeneous parameter fields using truncated unconditional Karhunen–Loève expansions (KLEs) for a certain choice of unconditional covariance kernel and construct surrogate models of the observable response with respect to the KLE coefficients. When direct measurements of the parameter fields are available, we propose improving the accuracy of these surrogate models by representing the parameter fields via conditional Karhunen-Loève expansions (CKLEs). CKLEs are constructed by conditioning the covariance kernel of the unconditional expansion on the direct measurements of the parameter field via Gaussian process regression, and then truncating the corresponding KLE.We apply the proposed methodology to constructing surrogate models via the Basis Adaptation (BA) method of the stationary hydraulic head response, measured at spatially discrete observation locations, of a groundwater flow model of the Hanford Site, as a function of the 1000-dimensional representation of the model’s log-transmissivity field. We find that BA surrogate models of the hydraulic head based on CKLEs are more accurate than BA surrogate models based on unconditional expansions for forward uncertainty quantification tasks. Furthermore, we find that inverse estimates of the hydraulic transmissivity field computed using CKLE-based BA surrogate models are more accurate than those computed using unconditional BA surrogate models.

Multi-fidelity neural network for uncertainty quantification of chemical reaction models

Surrogate models are often used to accelerate the uncertainty quantification (UQ) of chemical kinetic models. However, the construction of surrogate models usually requires the repetitive generation of high-fidelity input-output samples, which is time-consuming. In this work, we utilize a multi-fidelity neural network to speed up the surrogate model construction, yielding a multi-fidelity neural network-based surrogate model (MFNNSM). MFNNSM consists of two separate neural networks. The first neural network learns from high and low-fidelity samples, and then generates samples to train the second neural network for the subsequent UQ analysis. Based on the fact that the uncertainty similarity (sensitivity-based similarity) does exist between the predictions from the reduced and detailed combustion kinetics models, or between the predictions under different conditions, MFNNSM can use reduced model predictions as low-fidelity samples to generate detailed model predictions as the high-fidelity samples, or can transfer samples across different simulations conditions. To demonstrate the MFNNSM method, the ignition delay time (IDT) and laminar flame velocity (LFV) of methanol and n-decane are chosen as model prediction targets for UQ analysis. The results show that MFNNSM can achieve acceleration factors up to 6, by utilizing reduced model samples to generate detailed model samples under the same conditions, and when reusing the reduced model samples under different conditions, the acceleration factor can increase to 10. In addition, the influences of the relative error of the reduced models and the uncertainty similarity coefficients between combustion conditions on the performance of MFNNSM are further discussed, providing the guidance for future applications of MFNNSM.

A novel second‐order mixture importance sampling strategy for system reliability analysis

AbstractReliability analysis and evaluation of systems with multiple failure modes is extremely challenging. In this paper, a novel, efficient, and accurate system reliability method is proposed based on the second‐order approximation and the mixture important sampling (mixture‐IS) method. The proposed method does not require the construction of surrogate models and integrates the advantages of second‐order approximation and mixture‐IS strategies, which can greatly reduce the computational effort of estimating the system failure probability as well as ensuring the accuracy. The accuracy and efficiency of the proposed method are verified by several numerical and engineering problems, and comparisons are made against other existing reliability methods. The results indicate that the method can effectively reduce the number of limit state function calls and achieve high computational accuracy.

Multi-fidelity and multi-objective aerodynamic short nacelle shape optimisation under different flight conditions

AbstractThroughout the course of a flight mission, a range of aerodynamic conditions, including design-point conditions and off-design conditions, are encountered. As the bypass ratio increases and the fan-pressure ratio decreases to reduce the engine’s specific fuel consumption, the engine diameters increase, which results in an increase in the nacelle weight and overall drag. To reduce its weight and drag, a shorter nacelle with a length-to-diameter ratio $L/D = 0.35$ is investigated. In this study, an adaptive cokriging-based multi-objective optimisation method is applied to the design of a short aero-engine nacelle. Two nacelle performance metrics were employed as the objective functions for the optimisation routine. The cruise drag coefficient is evaluated under cruise conditions, whereas the intake pressure recovery is evaluated under takeoff conditions. The cokriging metamodel are refined using an effective infilling strategy, where high-fidelity samples are infilled via the modified Pareto fitness, and low-fidelity samples are infilled via the Pareto front. By combining parameterised geometry generation, automated mesh generation, numerical simulations, surrogate model construction, Pareto front exploration based on the non-dominated sorting genetic algorithm-II and sample infilling, an integrated multi-objective optimisation framework for short aero-engine nacelles is developed. Two-objective and three-objective test functions are used to validate the effectiveness of the proposed framework. After the optimisation process, a set of non-dominated nacelle designs is obtained with better aerodynamic performance than the original design, demonstrating the effectiveness of the optimisation framework. Compared with the kriging-based optimisation framework, the cokriging-based optimisation framework outperforms the single-fidelity method with a higher hypervolume value at the same number of iteration loops.

A novel deep learning-based automatic search workflow for CO2 sequestration surrogate flow models

Numerical simulation can significantly enhance subsurface resource utilisation's efficiency and economic benefits by multiphase flow in heterogeneous porous media. However, numerical simulation brings enormous computational demands and time consumption due to high-dimensional nonlinearity, heterogeneity, and coupling of multiple physical processes. Surrogate models can accelerate the establishment of complex models without sacrificing accuracy. However, creating well-performing surrogate models requires extensive human intervention and trial-and-error processes, even for cross-domain experts. This study proposes an automated surrogate flow model workflow based on deep learning called Surrogate Flow Model Search (SFMS). By incorporating neural architectures and loss functions into joint hyperparameter optimisation, SFMS automates many complex and time-consuming tasks involved in model development, such as designing neural architectures and loss functions. The automated surrogate model construction workflow enables researchers to develop high-quality surrogate models without extensive deep-learning expertise. We demonstrate the effectiveness of SFMS using saline aquifer CO2 injection as an example. The results show that SFMS can automatically generate a highly accurate surrogate flow model in a short time (<1300 s), capable of accurately predicting 120-time steps under different well controls and placements with a low average relative error (<0.4%). Therefore, SFMS can significantly reduce the time and effort required to develop accurate and reliable surrogate models, providing a new approach to building surrogate models.

QMugs 1.1: Quantum mechanical properties of organic compounds commonly encountered in reactivity datasets

Here, the Quantum Mechanical Properties of Drug-like Molecules (QMugs) dataset is expanded to facilitate its use as training data for surrogate machine learning models to predict quantum mechanical properties for tasks related to chemical reactivity. Small molecules from reaction databases as well as charged and boron-containing compounds from ChEMBL were added. Each of these compounds was passed through a pipeline of MMFF94s/UFF conformer generation, followed by GFN2-xTB optimization and finally a density functional theory single-point calculation at the ωB97X-D/def2-SVP level of theory. In total, 71,632 new molecules were evaluated in this manner. Steric (SASA) and dispersion (Pint) descriptors were computed at the semiempirical GFN2-xTB level of theory for the lowest energy conformer of all species in the enlarged QMugs dataset. The expanded dataset aims to facilitate the construction of surrogate models of much broader scope than the original QMugs dataset which was limited to biologically active compounds.

A Hybrid Regressor and Classifier-Assisted Evolutionary Algorithm for Expensive Optimization With Incomplete Constraint Information

Surrogate-assisted evolutionary algorithms (SAEAs) have become a popular tool to solve expensive optimization problems and have been gradually used to deal with expensive constraints. To handle inequality expensive constraints, existing SAEAs need both the information of constraint violation and satisfaction to construct surrogate models for constraints. However, many problems only feedback whether the candidate solution is feasible or how much degree it violates constraints. There is no detailed information of how much degree the candidate satisfies constraints. The performance of most existing SAEAs decreases a lot in solving expensive constrained optimization problems (ECOPs) with such incomplete constraint information. To solve the problem, this article proposes a hybrid regressor and classifier-assisted evolutionary algorithm (HRCEA). HRCEA adopts a radial basis function regression model to approximate the degree of constraint violation. In order to make a more credible prediction, a logistic regression classifier (LRC) is constructed for the offspring rectification. The classifier works in cooperation with the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha$</tex-math> </inline-formula> -cut strategy, in which the high confidence level can significantly improve the prediction reliability. Besides, the LRC is built based on the boundary training data selection strategy, which is devised to select samples around feasible boundaries. This strategy is helpful for the LRC to fit the local feasibility structure. Extensive experiments on commonly used benchmark functions in CEC2006 and CEC 2010 demonstrate that HRCEA has satisfactory performance in found results and execution efficiency when solving ECOPs with incomplete constraint information. Furthermore, HRCEA is utilized to solve ceramic formula design optimization problem, which shows its promising application in real-world optimization problems.

An operator inference oriented approach for linear mechanical systems

Model order reduction techniques allow the construction of low-dimensional surrogate models that can accelerate engineering design processes. Often, these techniques are intrusive, meaning that they require direct access to underlying high-fidelity models. Accessing these models is laborious or may not even be possible in some cases. Therefore, there is an interest in developing non-intrusive model order reduction techniques to construct low-dimensional models directly from simulated or experimental data. In this work, we focus on a recent data-driven methodology, namely operator inference, that aims at inferring the reduced operators using only trajectories of high-fidelity models. We present an extension of operator inference for linear mechanical systems, preserving the underlying second-order structure. Furthermore, we study a particular case in which complete information about the external forces is available. In this formulation, the reduced operators, having certain properties inspired by the original system matrices, are enforced by adding constraints to the inference problem. We illustrate the proposed methodologies using three numerical examples.

Recent Advances in Bayesian Optimization

Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this article attempts to provide a comprehensive and updated survey of recent advances in Bayesian optimization that are mainly based on Gaussian processes and identify challenging open problems. We categorize the existing work on Bayesian optimization into nine main groups according to the motivations and focus of the proposed algorithms. For each category, we present the main advances with respect to the construction of surrogate models and adaptation of the acquisition functions. Finally, we discuss the open questions and suggest promising future research directions, in particular with regard to heterogeneity, privacy preservation, and fairness in distributed and federated optimization systems.

Enhanced universal kriging for transformed input parameter spaces

With computational models becoming more expensive and complex, surrogate models have gained increasing attention in many scientific disciplines and are often necessary to conduct sensitivity studies, parameter optimization etc. In the scientific discipline of uncertainty quantification (UQ), model input quantities are often described by probability distributions. For the construction of surrogate models, space-filling designs are generated in the input space to define training points, and evaluations of the computational model at these points are then conducted. The physical parameter space is often transformed into an i.i.d. uniform input space in order to apply space-filling training procedures in a sensible way. Due to this transformation surrogate modeling techniques tend to suffer with regard to their prediction accuracy. Therefore, a new method is proposed in this paper where input parameter transformations are applied to basis functions for universal kriging. To speed up hyperparameter optimization for universal kriging, suitable expressions for efficient gradient-based optimization are developed. Several benchmark functions are investigated and the proposed method is compared with conventional methods.

Machine learning of evolving physics-based material models for multiscale solid mechanics

In this work we present a hybrid physics-based and data-driven learning approach to construct surrogate models for concurrent multiscale simulations of complex material behavior. We start from robust but inflexible physics-based constitutive models and increase their expressivity by allowing a subset of their material parameters to change in time according to an evolution operator learned from data. This leads to a flexible hybrid model combining a data-driven encoder and a physics-based decoder. Apart from introducing physics-motivated bias to the resulting surrogate, the internal variables of the decoder act as a memory mechanism that allows path dependency to arise naturally. We demonstrate the capabilities of the approach by combining an FNN encoder with several plasticity decoders and training the model to reproduce the macroscopic behavior of fiber-reinforced composites. The hybrid models are able to provide reasonable predictions of unloading/reloading behavior while being trained exclusively on monotonic data. Furthermore, in contrast to traditional surrogates mapping strains to stresses, the specific architecture of the hybrid model allows for lossless dimensionality reduction and straightforward enforcement of frame invariance by using strain invariants as the feature space of the encoder.