Popular Surrogate Models Research Articles

An adversarial diverse deep ensemble approach for surrogate‐based traffic signal optimization

AbstractSurrogate‐based traffic signal optimization (TSO) is a computationally efficient alternative to simulation‐based TSO. By replacing the simulation‐based objective function, a surrogate model can quickly identify solutions by searching for extreme points on its response surface. As a popular surrogate model, the ensemble of multiple diverse deep learning models can approximate complicated systems with a strong generalizability. However, existing ensemble methods barely focus on strengthening the prediction of extreme points, which we found can be realized by further diversifying base learners in the ensemble. The study proposes an adversarial diverse ensemble (ADE) method for online TSO with limited computational resources, comprising two stages: In the offline stage, base extractors are diversified with unlabeled data by a designed adversarial diversity training algorithm; in the online stage, base predictors are trained in parallel with limited labeled data, and the ensemble then serves as the surrogate model to search for solutions iteratively for TSO. First, it is demonstrated that the prediction accuracy on extreme points, and associated solution quality, can be constantly improved with base learners’ diversity enhanced by ADE. Case studies of TSO conducted on a four‐intersection arterial further demonstrate the superior solution quality and computational efficiency of the ADE surrogate model in a wide range of traffic scenarios. Moreover, a large‐scale online TSO experiment under dynamic traffic demand proves ADE's effectiveness in practical applications.

An integrated surrogate model-driven and improved termite life cycle optimizer for damage identification in dams

Structural Damage Identification (SDI) is a crucial branch in the field of structural health monitoring, providing an essential support for the safe operation of structures. In this paper, a novel structural damage identification method based on a surrogate-assisted evolutionary optimization algorithm is proposed. This method incorporates cost-effective surrogate modelling techniques and swarm intelligence optimization algorithms with powerful global optimization capabilities. Three popular surrogate models are fused based on a weighted average strategy to construct an integrated surrogate model with stronger generalization ability and higher accuracy. In addition, the self-designed movement strategy is more suitable for the characteristics of Termite Life Cycle Optimizer (TLCO), allowing the improved TLCO to have stronger robustness and the capability to escape from local optimum. The effectiveness of the proposed method for solving the SDI problem is confirmed in a damaged dam model with different complexities. Some important findings are as follows: (i) Compared with the conventional SDI methods, which directly combines optimization algorithms and finite element models, the proposed method maintains reliable accuracy, while improving computational efficiency by a factor of more than 100. (ii) There is no direct relationship between the accuracy of the surrogate models and the effectiveness of the SDI method based on their combination with ITLCO. The proposed method can serve as a promising tool for damage identification in large-scale structures.

Prediction and Global Sensitivity Analysis of Long-Term Deflections in Reinforced Concrete Flexural Structures Using Surrogate Models.

Reinforced concrete (RC) is the result of a combination of steel reinforcing rods (which have high tensile) and concrete (which has high compressive strength). Additionally, the prediction of long-term deformations of RC flexural structures and the magnitude of the influence of the relevant material and geometric parameters are important for evaluating their serviceability and safety throughout their life cycles. Empirical methods for predicting the long-term deformation of RC structures are limited due to the difficulty of considering all the influencing factors. In this study, four popular surrogate models, i.e., polynomial chaos expansion (PCE), support vector regression (SVR), Kriging, and radial basis function (RBF), are used to predict the long-term deformation of RC structures. The surrogate models were developed and evaluated using RC simply supported beam examples, and experimental datasets were collected for comparison with common machine learning models (back propagation neural network (BP), multilayer perceptron (MLP), decision tree (DT) and linear regression (LR)). The models were tested using the statistical metrics R2, RAAE, RMAE, RMSE, VAF, PI, A10-index and U95. The results show that all four proposed models can effectively predict the deformation of RC structures, with PCE and SVR having the best accuracy, followed by the Kriging model and RBF. Moreover, the prediction accuracy of the surrogate model is much lower than that of the empirical method and the machine learning model in terms of the RMSE. Furthermore, a global sensitivity analysis of the material and geometric parameters affecting structural deflection using PCE is proposed. It was found that the geometric parameters are more influential than the material parameters. Additionally, there is a coupling effect between material and geometric parameters that works together to influence the long-term deflection of RC structures.

Efficient aerodynamic analysis and optimization under uncertainty using multi-fidelity polynomial chaos-Kriging surrogate model

Surrogate model has been extensively employed in uncertainty-based design optimization (UBDO) for computationally expensive engineering problems. However, it often causes great difficulties to designers due to the unsatisfactory accuracy and the high sensitivity of surrogate prediction in presence of uncertainties. Worse still, some popular metamodeling methods also require a substantially higher computational cost than that in deterministic design to get an acceptable accuracy. To address the challenging problem, an UBDO framework based on the proposed multi-fidelity polynomial chaos-Kriging (MF PC-Kriging) surrogate model is proposed, with particular superiority for complex aerodynamic applications. The construction principle of the MF PC-Kriging model and the rationality of the superiority of it with respect to popular surrogate models are explained in detail. Meantime, it is examined by investigating an analytical function and a transonic aerodynamic application with both geometrical and operational uncertainties. Thus, the MF PC-Kriging with easier understanding and better modeling capabilities is involved in UBDO to resolve the proposed difficulty. Finally, an uncertainty-based aerodynamic design optimization problem is performed using this proposed framework. It is observed that for the considered examples, the developed methodology is more efficient and provides the better performance for aerodynamic uncertainty analysis, and complex aerodynamic analysis and optimization under uncertainty compared with universal Kriging and PC-Kriging methods.

Ensemble of surrogates combining Kriging and Artificial Neural Networks for reliability analysis with local goodness measurement

In engineering problems, reliability assessment is often computationally expensive. Surrogate models with active learning approaches are commonly used to solve this problem. Kriging is one of the most popular surrogate models used in this domain. Recently, artificial neural networks (ANN) have also attracted a lot of interest for structural reliability assessment due to their powerful capability. Many active learning approaches have been developed based on Kriging models and ANN. But selecting an appropriate model or technique for a reliability assessment problem with limited knowledge of the limit state function remains a challenging task. Ensemble of surrogates seems to be a good approach to tackle this challenge. In this work, two active learning approaches are proposed to combine Kriging and ANN models for reliability analysis. One is the local best surrogate (LBS) approach and the other is the local weighted average surrogate (LWAS) approach. Cross-validation and Jackknife techniques are used to estimate prediction errors of the surrogate models. In addition, two methods are proposed to locally measure the goodness of the surrogate models and calculate the prediction errors of the ensemble of surrogates. The surrogate models are updated by selecting the new sample points that have large prediction errors and are close to the limit state. The efficiency and accuracy of the proposed approaches are demonstrated by 4 representative examples and two finite element problems. The results show that the proposed methods can be effective in evaluating the reliability of high dimension and rare event problems with less computational costs than the single typical surrogate model with active learning approaches (e.g. AK-MCS). Moreover, compared with the ensemble of surrogate models based on global goodness measurement, the proposed approaches also outperform in most cases. Finally, it should be mentioned that the proposed approaches are not only suitable for the combination of Kriging and ANN but also can be extended to other multiple surrogate models including support vector machine, polynomial chaos expansion, and so on.

Adult zebrafish ventricular electrical gradients as tissue mechanisms of ECG patterns under baseline vs. oxidative stress.

In mammalian ventricles, electrical gradients establish electrical heterogeneities as essential tissue mechanisms to optimize mechanical efficiency and safeguard electrical stability. Electrical gradients shape mammalian electrocardiographic patterns; disturbance of electrical gradients is proarrhythmic. The zebrafish heart is a popular surrogate model for human cardiac electrophysiology thanks to its remarkable recapitulation of human electrocardiogram and ventricular action potential features. Yet, zebrafish ventricular electrical gradients are largely unexplored. The goal of this study is to define the zebrafish ventricular electrical gradients that shape the QRS complex and T wave patterns at baseline and under oxidative stress. We performed in vivo electrocardiography and ex vivo voltage-sensitive fluorescent epicardial and transmural optical mapping of adult zebrafish hearts at baseline and during acute H2O2 exposure. At baseline, apicobasal activation and basoapical repolarization gradients accounted for the polarity concordance between the QRS complex and T wave. During H2O2 exposure, differential regional impairment of activation and repolarization at the apex and base disrupted prior to baseline electrical gradients, resulting in either reversal or loss of polarity concordance between the QRS complex and T wave. KN-93, a specific calcium/calmodulin-dependent protein kinase II inhibitor (CaMKII), protected zebrafish hearts from H2O2 disruption of electrical gradients. The protection was complete if administered prior to oxidative stress exposure. Despite remarkable apparent similarities, zebrafish and human ventricular electrocardiographic patterns are mirror images supported by opposite electrical gradients. Like mammalian ventricles, zebrafish ventricles are also susceptible to H2O2 proarrhythmic perturbation via CaMKII activation. Our findings suggest that the adult zebrafish heart may constitute a clinically relevant model to investigate ventricular arrhythmias induced by oxidative stress. However, the fundamental ventricular activation and repolarization differences between the two species that we demonstrated in this study highlight the potential limitations when extrapolating results from zebrafish experiments to human cardiac electrophysiology, arrhythmias, and drug toxicities.

An efficient kriging modeling method for high-dimensional design problems based on maximal information coefficient

Kriging, one of the most popular surrogate models, is widely used in computationally expensive optimization problems to improve the design efficiency. However, due to the “curse-of-dimensionality,” the time for generating the kriging model increases exponentially as the dimension of the problem grows. When it comes to the cases that the kriging model needs to be frequently constructed, such as sequential sampling for kriging modeling or global optimization based on kriging model, the increased modeling time should be taken into consideration. To overcome this challenge, we propose a novel kriging modeling method which combines kriging with maximal information coefficient (MIC). Taking the features of the optimized hyper-parameters into consideration, MIC is utilized for estimating the relative magnitude of hyper-parameters. Then this knowledge of hyper-parameters is incorporated into the maximum likelihood estimation problem to reduce the dimensionality. In this way, the high dimensional optimization can be transformed into a one-dimensional optimization, which can significantly improve the modeling efficiency. Five representative numerical examples from 20-D to 80-D and an industrial example with 35 variables are used to show the effectiveness of the proposed method. Results show that compared with the conventional kriging, the modeling time of the proposed method can be ignored, while the loss of accuracy is acceptable. For the problems with more than 40 variables, the proposed method can even obtain a more accurate kriging model with given computational resources. Besides, the proposed method is also compared with KPLS (kriging combined with the partial least squares method), another state-of-the-art kriging modeling method for high-dimensional problems. Results show that the proposed method is more competitive than KPLS, which means the proposed method is an efficient kriging modeling method for high-dimensional problems.

Lattice Convolutional Neural Network Modeling of Adsorbate Coverage Effects

Coverage effects, known also as lateral interactions, are often important in surface processes, but their study via exhaustive density functional theory (DFT) is impractical because of the large configurational degrees of freedom. The cluster expansion (CE) is the most popular surrogate model accounting for coverage effects but suffers from slow convergence, its linear form, and its tendency to be biased toward the selection of smaller clusters. We develop a novel lattice convolutional neural network (LCNN) that improves upon some of CE’s limitations and exhibits better performance (test RMSE of 4.4 meV/site) compared to state-of-the-art methods, such as the CE assisted by a genetic algorithm and the convolution operation of the crystal graph convolutional neural network (test RMSE of 5.5 and 6.8 meV/site, respectively) by 20–30%. Furthermore, LCNN can outperform other methods with less training data, implying accuracy with less DFT calculations. We analyze the van der Waals interaction via visualization of the hidden representation of the adsorbate lattice system in terms of individual site formation energies.

A surrogate assisted adaptive framework for robust topology optimization

This paper presents a novel approach for robust topology optimization (RTO). The proposed approach integrates the hybrid polynomial correlated function expansion (H-PCFE) with the deterministic topology optimization (DTO) algorithm, where H-PCFE is used for computing the moments associated with the objective function in robust design optimization (RDO). It is argued that re-training the H-PCFE model at every iteration will make the overall algorithm computationally expensive and time-consuming. To alleviate this issue, an adaptive algorithm is proposed that automatically decides whether to retrain the H-PCFE model or to use the available H-PCFE model. Since RTO involves a large number of design variables, we also argue that, from a computational point of view, gradient-based optimization algorithms are better suited over gradient-free optimization techniques. However, the bottleneck of using gradient-based optimization algorithms is associated with the computation of the gradients. To address this issue, we propose a procedure for computing the gradients of the objective function in RTO. The proposed approach is highly flexible and can be integrated with the already available DTO codes from literature. To illustrate the performance of the proposed approach, three well-known benchmark problems have been solved; two of them being on the 2D MBB beam and one more on a 3D cantilever beam. As compared to the results of the DTO, RTO yields conservative results which are visible from the additional limbs appearing in the optimized topology of the structure. Furthermore, to justify the use of H-PCFE as a surrogate within the proposed framework, a comparative assessment of the proposed approach against the popular surrogate models has also been presented. It is observed that H-PCFE outperforms other popular surrogate models such as Kriging, radial basis function and polynomial chaos expansion, in both accuracy and efficiency.

An improved support vector regression using least squares method

Due to the good performance in terms of accuracy, sparsity and flexibility, support vector regression (SVR) has become one of the most popular surrogate models and has been widely researched and applied in various fields. However, SVR only depends on a subset of the training data, because the 𝜖-insensitive loss function ignores any training data that is within the threshold 𝜖. Therefore, some extra information may be extracted from these training data to improve the accuracy of SVR. By using the least squares method, a new improved SVR (ISVR) is developed in this paper, which combines the characteristics of SVR and traditional regression methods. ISVR is based on a two-stage procedure. The principle of ISVR is to treat the response of SVR obtained in the first stage as feedback, and then add some highly nonlinear ingredients and extra linear ingredients accordingly in the second stage by utilizing a correction function. Particularly, three types of ISVR are constructed by selecting different correction functions. Additionally, the performance of ISVR is investigated through eight mathematical problems of varying dimensions and one structural mechanics problem. The results show that ISVR has some advantages in accuracy when compared with SVR, even though the number of training points varies.

Quantifying uncertainties in first-principles alloy thermodynamics using cluster expansions

The cluster expansion is a popular surrogate model for alloy modeling to avoid costly quantum mechanical simulations. As its practical implementations require approximations, its use trades efficiency for accuracy. Furthermore, the coefficients of the model need to be determined from some known data set (training set). These two sources of error, if not quantified, decrease the confidence we can put in the results obtained from the surrogate model. This paper presents a framework for the determination of the cluster expansion coefficients using a Bayesian approach, which allows for the quantification of uncertainties in the predictions. In particular, a relevance vector machine is used to automatically select the most relevant terms of the model while retaining an analytical expression for the predictive distribution. This methodology is applied to two binary alloys, SiGe and MgLi, including the temperature dependence in their effective cluster interactions. The resulting cluster expansions are used to calculate the uncertainty in several thermodynamic quantities: ground state line, including the uncertainty in which structures are thermodynamically stable at 0 K, phase diagrams and phase transitions. The uncertainty in the ground state line is found to be of the order of meV/atom, showing that the cluster expansion is reliable to ab initio level accuracy even with limited data. We found that the uncertainty in the predicted phase transition temperature increases when including the temperature dependence of the effective cluster interactions. Also, the use of the bond stiffness versus bond length approximation to calculate temperature dependent properties from a reduced set of alloy configurations showed similar uncertainty to the approach where all training configurations are considered but at a much reduced computational cost.

An efficient reanalysis assisted optimization for variable-stiffness composite design by using path functions

An efficient reanalysis assisted optimization method is proposed for variable-stiffness composite design. The contour lines of functions can be used to describe fiber paths by using newly developed path functions. The parameters of path functions are used as design variables, and the initial objective function is evaluated by using Finite Element Method (FEM). The variable-stiffness composite laminate is formulated using FEM based on Mindlin shell theory. Manufacturing constraints are considered by examining the curvature and parallelism of fiber paths, therefore, the optimal solutions should be manufacturable. In order to improve the efficiency of optimization, reanalysis methods are employed instead of popular surrogate models. Compared with the Surrogate Assisted Optimization (SAO), the advantage of reanalysis is that the error can be well controlled, thus the accuracy of optimization should be improved significantly. Two numerical examples are used to verify the performance of the proposed method. The comparison with the linear variation fiber orientation angles shows that the proposed method obtains better solutions by considering the manufacturing constraints simultaneously.

Adaptive response surface by kriging using pilot points for structural reliability analysis

Structural reliability analysis aims to compute the probability of failure by considering system uncertainties. However, this approach may require very time-consuming computation and becomes impracticable for complex structures especially when complex computer analysis and simulation codes are involved such as finite element method. Approximation methods are widely used to build simplified approximations, or metamodels providing a surrogate model of the original codes. The most popular surrogate model is the response surface methodology, which typically employs second order polynomial approximation using least-squares regression techniques. Several authors have been used response surface methods in reliability analysis. However, another approximation method based on kriging approach has successfully applied in the field of deterministic optimization. Few studies have treated the use of kriging approximation in reliability analysis and reliability-based design optimization. In this paper, the kriging approximation is used an alternative to the traditional response surface method, to approximate the performance function of the reliability analysis. The main objective of this work is to develop an efficient global approximation while controlling the computational cost and accurate prediction. A point method is proposed to the kriging approximation in order to increase the prior predictivity of the approximation, which the are good candidates for numerical simulation. In other words, the predictive quality of the initial kriging approximation is improved by adding adaptive information called pilot points in areas where the kriging variance is maximum. This methodology allows for an efficient modeling of highly non-linear responses, while the number of simulations is reduced compared to Latin Hypercubes approach. Numerical examples show the efficiency and the interest of the proposed method.