Surrogate-based Optimization Methods Research Articles

Kriging-Based Space Exploration Global Optimization Method in Aerodynamic Design

For complicated aerodynamic design problems, the efficient global optimization method suffered from the defect of the incorrect portrayal of the design space, resulting in bad global convergence and efficiency performance. To this end, a Kriging-based global optimization method, named the Kriging-based space exploration method (KSE), was proposed in this paper. It selected multiple promising local minima and classified them into partially and fully explored minima in terms of the fitting quality of the surrogate model. Then, the partially explored minima would be furtherly exploited. During the local search, an enhanced trust-region method was adopted to make deep exploitation. By combining local and global searches, the proposed method could improve the fitting quality of the surrogate model and the optimization efficiency. The KSE was compared to other global surrogate-based optimization methods on 12 bound-constrained testing functions with 2 to 16 design variables and 2 aerodynamic optimization problems with 24 to 77 design variables. The results indicated that the KSE generally took fewer function evaluations to find the global optima or reach the target value in most test problems, holding better efficiency and robustness.

Robust optimal operation of continuous catalytic reforming process under feedstock uncertainty

The continuous catalytic regenerative (CCR) reforming process is one of the most significant sources of hydrogen production in the petroleum refining process. However, the fluctuations in feedstock composition and flow rate could significantly affect both product distribution and energy consumption. In this study, a robust deviation criterion based multi-objective optimization approach is proposed to perform the optimal operation of CCR reformer under feedstock uncertainty, with simultaneous maximization of product yields and minimization of energy consumption. Minimax approach is adopted to handle these uncertain objectives, and the Latin hypercube sampling method is then used to calculate these robust deviation criteria. Multi-objective surrogate-based optimization methods are next introduced to effectively solve the robust operational problem with high computational cost. The level diagram method is finally utilized to assist in multi-criteria decision-making. Two robust operational optimization problems with different objectives are solved to demonstrate the effectiveness of the proposed method for robust optimal operation of the CCR reforming process under feedstock uncertainty.

An efficient global optimization algorithm for expensive constrained black-box problems by reducing candidate infilling region

Surrogate-based optimization methods are usually employed to solve expensive black-box optimization problems. Constraints bring more challenges for optimization than unconstrained problems. This study proposes a novel three-stage infilling framework to solve expensive single-objective problems with equality and/or inequality constraints. In the first stage, a criterion for locating a feasible region is designed. Until a feasible solution is found, the first stage is removed in the subsequent process. The second stage exploits the current best feasible solution and explores potentially better solutions. If the infilled site is infeasible, the third stage will proceed to improve the surrogate accuracy around the feasible region boundary and an extra constraint derived from the objective is proposed to reduce the candidate infill region in this stage. Otherwise, the process stays in stage two. Stage two continues once one third-stage point has been sampled. For brevity, the three-stage framework is called CEGO-DI. The efficacy of CEGO-DI is demonstrated by comparing it with two variants of it. In addition, the method is also compared with several state-of-the-art algorithms on extensive problems. The experimental results show that CEGO-DI has a better or competitive performance on most problems.

A radial basis function surrogate model assisted evolutionary algorithm for high-dimensional expensive optimization problems

Evolutionary algorithms require large number of function evaluations to locate the global optimum, making it computationally prohibitive on dealing with expensive problems. Surrogate-based optimization methods have shown promising ability on accelerating the convergence speed. However, it is still a challenging work for surrogate-assisted methods to deal with high-dimensional expensive problems because it is hard to approximate the objective function in high-dimensional space. In this paper, a novel radial basis function surrogate model assisted evolutionary algorithm for high-dimensional expensive optimization problems (RSAEH) is proposed. Specifically, the proposed algorithm consists of local search part and surrogate-guided prescreening part. In the local search part, the local surrogate is built by radial basis function with the most promising training sample points, and the optima (or near-optima) is located by optimizer to conduct exact function evaluation. In the surrogate-guided prescreening part, the current best sample point is refined by using sequential quadratic programming, thus guide the mutation direction by using differential evolution operator, and promising offspring prescreened by surrogate model are evaluated using exact function evaluation. To validate the effectiveness of the proposed algorithm, it is tested on benchmark problems with dimension ranging from 30 to 100, as well as a real-world oil reservoir production optimization problem. The proposed algorithm achieved best optimization results on 16 benchmark functions among 21 benchmark function sets in comparison with other algorithms. The performance of RSAEH is competitive especially on 100-D benchmark functions. In addition, RSAEH also showed promising performance on a real-world oil reservoir production optimization problem with 160 variables, in comparison with several state-of-the-art algorithms.

Stochastic Crashworthiness Optimization Accounting for Simulation Noise

Abstract Finite element-based crashworthiness optimization is extensively used to improve the safety of motor vehicles. However, the responses of crash simulations are characterized by a high level of numerical noise, which can hamper the blind use of surrogate-based design optimization methods. It is therefore essential to account for the noise-induced uncertainty when performing optimization. For this purpose, a surrogate, referred to as Non-Deterministic Kriging (NDK), can be used. It models the noise as a non-stationary stochastic process, which is added to a traditional deterministic kriging surrogate. Based on the NDK surrogate, this study proposes an optimization algorithm tailored to account for both epistemic uncertainty, due to the lack of data, and irreducible aleatory uncertainty, due to the simulation noise. The variances are included within an extension of the well-known expected improvement infill criterion referred to as Modified Augmented Expected Improvement (MAEI). Because the proposed optimization scheme requires an estimate of the aleatory variance, it is approximated through a regression kriging, which is iteratively refined. The proposed algorithm is tested on a set of analytical functions and applied to the optimization of an Occupant Restraint System (ORS) during a crash.

An Efficient Surrogate-Based Optimization Method for BWBUG Based on Multifidelity Model and Geometric Constraint Gradients

Performing shape optimization of blended-wing-body underwater glider (BWBUG) can significantly improve its gliding performance. However, high-fidelity CFD analysis and geometric constraint calculation in traditional surrogate-based optimization methods are expensive. An efficient surrogate-based optimization method based on the multifidelity model and geometric constraint gradient information is proposed. By establishing a shape parameterized model, deriving analytical expression of geometric constraint gradient, constructing multifidelity surrogate model, the calculation times of high-fidelity CFD model and geometric constraints are reduced during the shape optimization process of BWBUG, which greatly improve the optimization efficiency. Finally, the effectiveness and efficiency of the proposed method are verified by performing the shape optimization of a BWBUG and comparing with traditional surrogate-based optimization methods.

A Double-Stage Surrogate-Based Shape Optimization Strategy for Blended-Wing-Body Underwater Gliders

In this paper, a Double-stage Surrogate-based Shape Optimization (DSSO) strategy for Blended-Wing-Body Underwater Gliders (BWBUGs) is proposed to reduce the computational cost. In this strategy, a double-stage surrogate model is developed to replace the high-dimensional objective in shape optimization. Specifically, several First-stage Surrogate Models (FSMs) are built for the sectional airfoils, and the second-stage surrogate model is constructed with respect to the outputs of FSMs. Besides, a Multi-start Space Reduction surrogate-based global optimization method is applied to search for the optimum. In order to validate the efficiency of the proposed method, DSSO is first compared with an ordinary One-stage Surrogate-based Optimization strategy by using the same optimization method. Then, the other three popular surrogate-based optimization methods and three heuristic algorithms are utilized to make comparisons. Results indicate that the lift-to-drag ratio of the BWBUG is improved by 9.35% with DSSO, which outperforms the comparison methods. Besides, DSSO reduces more than 50% of the time that other methods used when obtaining the same level of results. Furthermore, some considerations of the proposed strategy are further discussed and some characteristics of DSSO are identified.

A generalized methodology for multidisciplinary design optimization using surrogate modelling and multifidelity analysis

The advantages of multidisciplinary design are well understood, but not yet fully adopted by the industry where methods should be both fast and reliable. For such problems, minimum computational cost while providing global optimality and extensive design information at an early conceptual stage is desired. However, such a complex problem consisting of various objectives and interacting disciplines is associated with a challenging design space. This provides a large pool of possible designs, requiring an efficient exploration scheme with the ability to provide sufficient feedback early in the design process. This paper demonstrates a generalized optimization framework with rapid design space exploration capabilities in which a Multifidelity approach is directly adjusted to the emerging needs of the design. The methodology is developed to be easily applicable and efficient in computationally expensive multidisciplinary problems. To accelerate such a demanding process, Surrogate Based Optimization methods in the form of both Radial Basis Function and Kriging models are employed. In particular, a modification of the standard Kriging approach to account for Multifidelity data inputs is proposed, aiming to increasing its accuracy without increasing its training cost. The surrogate optimization problem is solved by a Particle Swarm Optimization algorithm and two constraint handling methods are implemented. The surrogate model modifications are visually demonstrated in a 1D and 2D test case, while the Rosenbrock and Sellar functions are used to examine the scalability and adaptability behaviour of the method. Our particular Multiobjective formulation is demonstrated in the common RAE2822 airfoil design problem. In this paper, the framework assessment focuses on our infill sampling approach in terms of design and objective space exploration for a given computational cost.

A multi-fidelity RBF surrogate-based optimization framework for computationally expensive multi-modal problems with application to capacity planning of manufacturing systems

This paper presents a multi-fidelity RBF (radial basis function) surrogate-based optimization framework (MRSO) for computationally expensive multi-modal optimization problems when multi-fidelity (high-fidelity (HF) and low-fidelity (LF)) models are available. The HF model is expensive and accurate while the LF model is cheaper to compute but less accurate. To exploit the correlation between the LF and HF models and improve algorithm efficiency, in MRSO, we first apply the DYCORS (dynamic coordinate search algorithm using response surface) algorithm to search on the LF model and then employ a potential area detection procedure to identify the promising points from the LF model. The promising points serve as the initial start points when we further search for the optimal solution based on the HF model. The performance of MRSO is compared with 6 other surrogate-based optimization methods (4 are using a single-fidelity surrogate and the rest 2 are using multi-fidelity surrogates). The comparisons are conducted on a multi-fidelity optimization test suite containing 10 problems with 10 and 30 dimensions. Besides the benchmark functions, we also apply the proposed algorithm to a practical and computationally expensive capacity planning problem in manufacturing systems which involves discrete event simulations. The experimental results demonstrate that MRSO outperforms all the compared methods.

A comparison study of two multifidelity methods for aerodynamic optimization

Industrial aerodynamic design applications require multiobjective optimization tools able to provide design feedback to the engineers. This is true especially when optimization studies are carried out during the conceptual design stage. The need for fast optimization methods has led to the development of multifidelity methods in a surrogate based optimization environment. Multifidelity tools have the potential to accelerate the design process, primarily due to the lower cost associated with the low fidelity tool. In addition to this, the design stage is shortened as mature and reliable high fidelity design information is provided earlier in the design cycle. Despite this high potential of these methods, there is no explicit comparison available in the literature between multifidelity surrogate based optimization tools for industrial aerodynamic problems. This paper aims at providing a direct comparison between two multiobjective multifidelity surrogate based optimization methods developed by our group. The first approach uses a trust region formulation for efficient multiobjective that does not require gradients. The second is using the concept of expected improvement to perform fast design space exploration based on a novel Kriging modification for multifidelity data. The tools are applied in two aerodynamic design problems: optimization of a high lift configuration in respect to maximum lift maximization and an airfoil design for transonic cruising conditions. These problems feature characteristics of industrial interest. They involve difficult physical analyses in the case of the high lift configuration and a more complex optimization formulation due to the increased dimensionality in the case of the transonic airfoil. Our presented methods are compared against a CFD-based optimization, a surrogate based optimization using only high fidelity data and a multifidelity surrogate based optimization based on Co-Kriging. Early results suggest that the trust region method can quickly provide improved designs leading to an efficient Pareto front. The expected improvement based method shows fast exploration attributes and a wide Pareto front.

Optimization of variable blank holder force in deep drawing based on support vector regression model and trust region

Blank holder force (BHF) is one of the important process parameters for successful sheet metal forming. Variable blank holder force (VBHF) that the BHF varies through the forming process is recognized as one of the advanced manufacturing technologies. Therefore, optimization of VBHF trajectory is a crucial issue in industries. One of the effective approaches to determine the VBHF trajectory is to use the surrogate modeling techniques. However, it is very inaccurate and time-consuming to determine the VBHF trajectory for successful sheet forming through surrogate-based optimization methods. Therefore, this paper proposes an improved surrogate-based optimization method by integration of support vector regression (SVR) and trust region strategy to optimize VBHF in deep drawing. First, a random sampling test of VBHF in deep drawing is designed and a SVR approximate model of VBHF under random sampling is developed. Then, a trust region algorithm is adopted to predict and control the accuracy of the SVR approximate model of VBHF. Response surface is repeatedly constructed and optimized that is adopted to identify the Pareto-frontier of VBHF. The validity of the proposed approach is examined through the comparison of numerical and experimental results. The results of this research provide a reliable reference for future efforts to optimize VBHF in deep drawing.

A Bayesian Stochastic Kriging Optimization Model Dealing with Heteroscedastic Simulation Noise for Freeway Traffic Management

Since advanced traveler information and traffic management systems have become popular, it is vital to capture the joint impact of various strategies on transportation systems. Developing analytical models to incorporate traffic dynamics and travel behavior is challenging. Based on observation of the heteroscedasticity of the simulation noise in the stochastic simulator, this paper develops the Bayesian stochastic Kriging (BSK) model to adapt for the heteroscedastic noise and metamodel parameter uncertainty in a Bayesian framework, which enhances the existing surrogate-based optimization methods. This paper presents a metamodel for large scale simulation-based freeway traffic management optimization problems. Simulation-based optimization combines the advantages of simulation models and mathematical optimization methods. The parameter estimation of the BSK model is accomplished by the Bayesian inference. The proposed methodology enables the efficient use of large scale high-resolution traffic simulation models for simulation-based optimization, while accounting for travelers’ behavioral responses to information provision. We demonstrate the advantages of BSK compared to other existing metamodels using a numerical example of a synthetic network and a mathematical example. In a work zone scenario on a real-world freeway/arterial corridor of I-270 and MD-355 in the State of Maryland, the BSK model is applied to the freeway traffic management via optimizing the high-occupancy/toll rate and deploying dynamic message signs. Field traffic measurements by loop/microwave detections are used to calibrate travel demand and to supply the simulation parameters. The optimization results are promising in reducing the corridor-wide travel delay and enhancing the vehicle throughput. The online appendix is available at https://doi.org/10.1287/trsc.2018.0819 .

Adaptive optimization methodology based on Kriging modeling and a trust region method

Surrogate-Based Optimization (SBO) is becoming increasingly popular since it can remarkably reduce the computational cost for design optimizations based on high-fidelity and expensive numerical analyses. However, for complicated optimization problems with a large design space, many design variables, and strong nonlinearity, SBO converges slowly and shows imperfection in local exploitation. This paper proposes a trust region method within the framework of an SBO process based on the Kriging model. In each refinement cycle, new samples are selected by a certain design of experiment method within a variable design space, which is sequentially updated by the trust region method. A multi-dimensional trust-region radius is proposed to improve the adaptability of the developed methodology. Further, the scale factor and the limit factor of the trust region are studied to evaluate their effects on the optimization process. Thereafter, different SBO methods using error-based exploration, prediction-based exploitation, refinement based on the expected improvement function, a hybrid refinement strategy, and the developed trust-region-based refinement are utilized in four analytical tests. Further, the developed optimization methodology is employed in the drag minimization of an RAE2822 airfoil. Results indicate that it has better robustness and local exploitation capability in comparison with those of other SBO methods.

Meta-Model Framework for Surrogate-Based Parameter Estimation in Dynamical Systems

The central task in modeling complex dynamical systems is parameter estimation. This task involves numerous evaluations of a computationally expensive objective function. Surrogate-based optimization introduces a computationally efficient predictive model that approximates the value of the objective function. The standard approach involves learning a surrogate from training examples that correspond to past evaluations of the objective function. Current surrogate-based optimization methods use static, predefined substitution strategies that decide when to use the surrogate and when the true objective. We introduce a meta-model framework where the substitution strategy is dynamically adapted to the solution space of the given optimization problem. The meta model encapsulates the objective function, the surrogate model and the model of the substitution strategy, as well as components for learning them. The framework can be seamlessly coupled with an arbitrary optimization algorithm without any modification: it replaces the objective function and autonomously decides how to evaluate a given candidate solution. We test the utility of the framework on three tasks of estimating parameters of real-world models of dynamical systems. The results show that the meta model significantly improves the efficiency of optimization, reducing the total number of evaluations of the objective function up to an average of 77%.

Variable-fidelity shape optimization of dual-rotor wind turbines

PurposeDual-rotor wind turbines (DRWTs) are a novel type of wind turbines that can capture more power than their single-rotor counterparts. Because their surrounding flow fields are complex, evaluating a DRWT design requires accurate predictive simulations, which incur high computational costs. Currently, there does not exist a design optimization framework for DRWTs. Since the design optimization of DRWTs requires numerous model evaluations, the purpose of this paper is to identify computationally efficient design approaches.Design/methodology/approachSeveral algorithms are compared for the design optimization of DRWTs. The algorithms vary widely in approaches and include a direct derivative-free method, as well as three surrogate-based optimization methods, two approximation-based approaches and one variable-fidelity approach with coarse discretization low-fidelity models.FindingsThe proposed variable-fidelity method required significantly lower computational cost than the derivative-free and approximation-based methods. Large computational savings come from using the time-consuming high-fidelity simulations sparingly and performing the majority of the design space search using the fast variable-fidelity models.Originality/valueDue the complex simulations and the large number of designable parameters, the design of DRWTs require the use of numerical optimization algorithms. This work presents a novel and efficient design optimization framework for DRWTs using computationally intensive simulations and variable-fidelity optimization techniques.

Efficient Global Optimization of Actuator Based on a Surrogate Model Assisted Hybrid Algorithm

Computationally expensive numerical techniques are often involved in the actuator design optimization process, where efficiency is a major issue. Although surrogate-based optimization is a promising solution, the challenge to the optimization efficiency is still considerable. Aiming to address this challenge, a new method, called the parallel adjoint sensitivity and Gaussian process assisted hybrid optimization technique (PAGHO), is presented. The central concept is a new optimization framework employing computationally cheap partial derivatives obtained by the adjoint sensitivity method to tackle computationally expensive infill sampling for surrogate-based optimization. A silicon microactuator and a mathematical benchmark problem with different kinds of challenges are selected as the test cases. Comparison results show that PAGHO can obtain comparable results with popular global optimization methods, while at the same time having significant advantages in efficiency compared to standard global optimization methods and state-of-the-art surrogate-based optimization methods.

Implicit Space Mapping for Variable-Fidelity EM-Driven Design of Compact Circuits

Space mapping (SM) belongs to the most successful surrogate-based optimization (SBO) methods in microwave engineering. Among available SM variations, implicit SM (ISM) is particularly attractive due to its simplicity and separation of extractable surrogate model parameters and design variables of the circuit/system at hand. Unlike other SM approaches, ISM exploits a set of preassigned parameters to align the surrogate with the high-fidelity EM model. However, application of ISM is challenging if equivalent network model of the structure is unavailable or of poor quality. In this letter, a modified ISM that exploits variable-fidelity EM simulation models is proposed. Here, preassigned parameters are introduced in the coarsely discretized EM surrogate by dividing the substrate below selected components of the structure into segments with different permittivity and common thickness. The proposed method has been verified (also experimentally) using a rat-race coupler and favorably compared with state-of-the-art SBO methods.

Surrogate-based pumping optimization of coastal aquifers under limited computational budgets

AbstractThe computationally expensive variable density and salt transport numerical models hinder the implementation of simulation-optimization routines for coastal aquifer management. To reduce the computational cost, surrogate models have been utilized in pumping optimization of coastal aquifers. However, it has not been previously addressed whether surrogate modelling is effective given a limited number of numerical simulations with the seawater intrusion model. To that end, two surrogate-based optimization (SBO) frameworks are employed and compared against the direct optimization approach, under restricted computational budgets. The first, a surrogate-assisted algorithm, employs a strategy which aims at a fast local improvement of the surrogate model around optimal values. The other, balances global and local improvement of the surrogate model and is applied for the first time in coastal aquifer management. The performance of the algorithms is investigated for optimization problems of moderate and large dimensionalities. The statistical analysis indicates that for the specified computational budgets, the sample means of the SBO methods are statistically significantly better than those of the direct optimization. Additionally, the selection of cubic radial basis functions as surrogate models, enables the construction of very fast approximations for problems with up to 40 decision variables and 40 constraint functions.

A liquid metal-based structurally embedded vascular antenna: II. Multiobjective and parameterized design exploration

Research on the structurally embedded vascular antenna concept leverages past efforts on liquid metal (LM) reconfigurable electronics, microvascular composites, and structurally integrated and reconfigurable antennas. Such a concept has potential for reducing system weight or volume while simultaneously allowing in situ adjustment of resonant frequencies and/or changes in antenna directivity. This work considers a microvascular pattern embedded in a laminated composite and filled with LM. The conductive liquid provides radio frequency (RF) functionality while also allowing self-cooling. Models describing RF propagation and heat transfer, in addition to the structural effects of both the inclusion of channels and changes in temperature, were described in part 1 of this two-part work. In this part 2, the engineering models developed and demonstrated in part 1 toward the initial exploration of design trends are implemented into multiple optimization frameworks for more detailed design studies, one of which being novel and particularly applicable to this class of problem. The computational expense associated with the coupled multiphysical analysis of the structurally embedded LM transmitting antenna motivates the consideration of surrogate-based optimization methods. Both static and adaptive approaches are explored; it is shown that iteratively correcting the surrogate leads to more accurate optimized design predictions. The expected strong dependence of antenna performance on thermal environment motivates the consideration of a novel ‘parameterized’ optimization approach that simultaneously calculates whole families of optimal designs based on changes in design or operational variables generally beyond the control of the designer. The change in Pareto-optimal response with evolution in operating conditions is clearly demonstrated.

Precise control of reflection response in bandwidth-enhanced planar antennas

In this work, the issues of bandwidth enhancement of planar antennas and the relevance of precise and automated response control through numerical optimization have been investigated. Using an example of a planar antenna with parasitic radiator we illustrate possible effects of even minor modifications of the antenna geometry (here, applied to the ground plane) on its reflection performance. In particular, a proper handling of geometry parameters may lead to considerable broadening of the antenna bandwidth. For the sake of computational efficiency, the adjustment of geometry parameters is carried out using surrogate-based optimization methods exploiting coarse-discretization EM simulations as the underlying low-fidelity antenna model. Additionally, suitably defined penalty function allows us to precisely control the maximum in-band reflection so that sufficient margin to accommodate possible manufacturing tolerances can be achieved. The optimized designs of the two antenna structures considered in this work exhibit over 1.75 GHz (>31%) and 2.15 GHz (>38%) bandwidth, respectively, for the center frequency of 5.6 GHz. Simulation results are validated using measurements of the fabricated prototypes. Comparison with state-of-the-art designs is also provided. © 2016 Wiley Periodicals, Inc. Int J RF and Microwave CAE 26:653–659, 2016.