Expensive Black Box Research Articles

KASRA: A Kriging-based Adaptive Space Reduction Algorithm for global optimization of computationally expensive black-box constrained problems

Efficient Global Optimization (EGO) methodology over the entire design space can be considerably time-consuming as much as the expensive simulation computer codes on High-multimodal and computationally Expensive Black-box (HEB) constrained problems. This paper introduces a strategy specifically, the Kriging-based Adaptive Space Reduction Algorithm, named KASRA, to enhance the performance of EGO for HEB constrained optimization problems. A new measure is proposed according to the activity of the decision variables to adaptively reduce the size of design intervals centered at the current best solution. The shrunken intervals gradually are expanded to decrease the risk of missing the desirable region. The design sub-spaces are explored based on the weighed constrained expected improvement criterion. The weighting coefficients of exploration and exploitation dynamically are regulated according to the volume ratio of the current hyper-box-shaped region and the original one. The sequential quadratic programming and exponential tunneling algorithms as two local and global optimizers are employed on Kriging-based functions to achieve a more accurate solution at the end of the procedure if necessary. The genetic algorithm with different tuning strategies is used to defeat the extreme time challenge of constructing Kriging-based surrogates. The proposed algorithm is applicable even if there is no feasible point in the initial samples. The efficiency of KASRA is demonstrated on twenty-two mathematical and ten classical engineering benchmark problems. Experimental results and comparative studies confirm that the proposed approach has a promising performance to deal with HEB constrained optimization problems and generally performs better than the competitor methods on most of the benchmark problems.

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints (SHEBO) v1.0

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints (SHEBO) v1.0

An Improved Surrogate Based Optimization Method for Expensive Black-box Problems

For expensive black-box problems, surrogate modelling techniques are generally used to decrease the computational source. In this study, an improved surrogate based optimization (SBO) method is presented to solve the real-world engineering applications with expensive black-box objective responses. An optimized ensemble of surrogates combing three typical surrogate modelling techniques is adapted to efficiently predict the objective response. Meanwhile, the hierarchical design space reduction (HSR) strategy is employed for obtaining the smaller design subspace for improving the optimization efficiency. During the search, all test problems are considered as the real-world engineering applications whereas the actual global optima as well as the function characteristics are unknown in advance. The results show that the proposed method is superior in identifying the global optimum.

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints

We introduce the algorithm SHEBO (surrogate optimization of problems with hidden constraints and expensive black-box objectives), an efficient optimization algorithm that employs surrogate models to solve computationally expensive black-box simulation optimization problems that have hidden constraints. Hidden constraints are encountered when the objective function evaluation does not return a value for a parameter vector. These constraints are often encountered in optimization problems in which the objective function is computed by a black-box simulation code. SHEBO uses a combination of local and global search strategies together with an evaluability prediction function and a dynamically adjusted evaluability threshold to iteratively select new sample points. We compare the performance of our algorithm with that of the mesh-based algorithms mesh adaptive direct search (MADS, NOMAD [nonlinear optimization by mesh adaptive direct search] implementation) and implicit filtering and SNOBFIT (stable noisy optimization by branch and fit), which assigns artificial function values to points that violate the hidden constraints. Our numerical experiments for a large set of test problems with 2–30 dimensions and a 31-dimensional real-world application problem arising in combustion simulation show that SHEBO is an efficient solver that outperforms the other methods for many test problems.

Robust propagation of probability boxes by interval predictor models

This paper proposes numerical strategies to robustly and efficiently propagate probability boxes through expensive black box models. An interval is obtained for the system failure probability, with a confidence level. The three proposed algorithms are sampling based, and so can be easily parallelised, and make no assumptions about the functional form of the model. In the first two algorithms, the performance function is modelled as a function with unknown noise structure in the aleatory space and supplemented by a modified performance function. In the third algorithm, an Interval Predictor Model is constructed and a re-weighting strategy used to find bounds on the probability of failure. Numerical examples are presented to show the applicability of the approach. The proposed method is flexible and can account for epistemic uncertainty contained inside the limit state function. This is a feature which, to the best of the authors’ knowledge, no existing methods of this type can deal with.

The Statistical Filter Approach to Constrained Optimization

Expensive black box systems arise in many engineering applications but can be difficult to optimize because their output functions may be complex, multi-modal, and difficult to understand. The task becomes even more challenging when the optimization is subject to multiple constraints and no derivative information is available. In this article, we combine response surface modeling and filter methods in order to solve problems of this nature. In employing a filter algorithm for solving constrained optimization problems, we establish a novel probabilistic metric for guiding the filter. Overall, this hybridization of statistical modeling and nonlinear programming efficiently utilizes both global and local search in order to quickly converge to a global solution to the constrained optimization problem. To demonstrate the effectiveness of the proposed methods, we perform numerical tests on a synthetic test problem, a problem from the literature, and a real-world hydrology computer experiment optimization problem.

Bayesian Optimal Design of Experiments for Inferring the Statistical Expectation of Expensive Black-Box Functions

Abstract Bayesian optimal design of experiments (BODEs) have been successful in acquiring information about a quantity of interest (QoI) which depends on a black-box function. BODE is characterized by sequentially querying the function at specific designs selected by an infill-sampling criterion. However, most current BODE methods operate in specific contexts like optimization, or learning a universal representation of the black-box function. The objective of this paper is to design a BODE for estimating the statistical expectation of a physical response surface. This QoI is omnipresent in uncertainty propagation and design under uncertainty problems. Our hypothesis is that an optimal BODE should be maximizing the expected information gain in the QoI. We represent the information gain from a hypothetical experiment as the Kullback–Liebler (KL) divergence between the prior and the posterior probability distributions of the QoI. The prior distribution of the QoI is conditioned on the observed data, and the posterior distribution of the QoI is conditioned on the observed data and a hypothetical experiment. The main contribution of this paper is the derivation of a semi-analytic mathematical formula for the expected information gain about the statistical expectation of a physical response. The developed BODE is validated on synthetic functions with varying number of input-dimensions. We demonstrate the performance of the methodology on a steel wire manufacturing problem.

A novel global optimization algorithm and data-mining methods for turbomachinery design

A new multi-objective, multi-disciplinary global optimization strategy is proposed to address the high-dimensional, computationally expensive black box problem (HEB) in turbomachinery design. The strategy consists of an adaptive sampling hybrid optimization algorithm (ASHOA), two data-mining techniques, a 3D blade parameterization method, and the aerodynamic/mechanical codes. Firstly, the ASHOA is established by integrating a novel adaptive sampling Kriging metamodel and a new hybrid optimization method. Secondly, two data-mining methods (analysis of variance (ANOVA) and self-organizing map (SOM)) are applied to set the initial design space and optimization objectives of the transonic centrifugal compressor. A refined design space and objective parameters of the optimization problem are eventually obtained. Finally, the optimization process of a transonic centrifugal compressor is carried out based on the refined design space and objectives using ASHOA. The results show that the search efficiency of the optimization strategy is 2–10 times higher when compared to other excellent optimization algorithms. For the optimized compressor, both isentropic efficiency and total pressure ratio at design condition are improved by 1.61% and 4.13%, respectively, and the maximum stress decreases by 9.68%.

Expensive Black-Box Model Optimization Via a Gold Rush Policy

The optimization of black-box models is a challenging task owing to the lack of analytic gradient information and structural information about the underlying function, and also due often to significant run times. A common approach to tackling such problems is the implementation of Bayesian global optimization techniques. However, these techniques often rely on surrogate modeling strategies that endow the approximation of the underlying expensive function with nonexistent features. Further, these techniques tend to push new queries away from previously queried design points, making it difficult to locate an optimum point that rests near a previous model evaluation. To overcome these issues, we propose a gold rush (GR) policy that relies on purely local information to identify the next best design alternative to query. The method employs a surrogate constructed pointwise, that adds no additional features to the approximation. The result is a policy that performs well in comparison to state of the art Bayesian global optimization methods on several benchmark problems. The policy is also demonstrated on a constrained optimization problem using a penalty method.

Selection of a covariance function for a Gaussian random field aimed for modeling global optimization problems

Bayesian approach is actively used to develop global optimization algorithms aimed at expensive black box functions. One of the challenges in this approach is the selection of an appropriate model for the objective function. Normally, a Gaussian random field is chosen as a theoretical model. However, the problem of estimation of parameters, using objective function values, is not thoroughly researched. In this paper, we consider the behavior of the maximum likelihood estimators of parameters of the homogeneous isotropic Gaussian random field with squared exponential covariance function. We also compare properties of exponential covariance function models.

Aerodynamic drag reduction in a vehicle based on efficient global optimisation

Vehicle aerodynamic shape optimisation is a typical non-linear and computationally expensive black box problem, which is severely limited by time and cost of the objective function evaluations during the global optimisation process. To solve the shortcomings of low efficiency and high cost of the existing vehicle drag reduction method, an improved efficient global optimisation (EGO) algorithm is used to optimise a four-dimensional aerodynamic drag reduction design of a vehicle combined with computational fluid dynamics numerical simulation technology. Moreover, data mining technologies are used to reveal the influence mechanisms of design variables on aerodynamic drag and to analyse the relationship between the variables. It is demonstrated that the improved EGO algorithm, based on the kriging response surface and expected improvement function, can achieve the global optimum with minimum function evaluations. The aerodynamic drag of the optimal design is 1.56% lower than that of the original vehicle. The data mining results showed that the engine hood inclination and the tail upturn angle play a leading role in the vehicle's aerodynamic drag, and the hood inclination has the greatest impact.

A new Kriging–Bat Algorithm for solving computationally expensive black-box global optimization problems

ABSTRACTMany global optimization (GO) algorithms have been introduced in recent decades to deal with the Computationally Expensive Black-Box (CEBB) optimization problems. The high number of objective function evaluations, required by conventional GO methods, is prohibitive or at least inconvenient for practical design applications. In this work, a new Kriging–Bat algorithm (K–BA) is introduced for solving CEBB problems with further improved search efficiency and robustness. A Kriging surrogate model (SM) is integrated with the Bat Algorithm (BA) to find the global optimum using substantially reduced number of evaluations of the computationally expensive objective function. The new K–BA algorithm is tested and compared with other well-known GO algorithms, using a set of standard benchmark problems with 2 to 16 design variables, as well as a real-life engineering optimization application, to determine its search capability, efficiency and robustness. Results of the comprehensive tests demonstrated the suitability and superior capability of the new K–BA.

Optimal design of a parallel beam system with elastic supports to minimize flexural response to harmonic loading using a combined optimization algorithm

Mechanical systems subject to vibration are prevalent across many industries. Although potentially different in application, they sometimes share the need to minimize aspects of flexural deformation given harmonic loading and the need to consider a variety of both input and response-based constraints in the process. Practical design efforts also sometimes include the need for consideration of the optimal response of a platform-style product, including responses of multiple design variants supported by a common base structure. Harmonic problems can be especially challenging to optimize due to the likelihood that the response will be multi-modal; influenced by system natural frequencies throughout the design space. Further, analysis of these systems often involves large and complex computer models which require significant resources to execute. A harmonically loaded, platform-style parallel beam system with multiple family variants is used as an example in this work to demonstrate a proposed method for identifying an optimum in a constrained, multi-modal response environment with consideration for Expensive Black Box Functions (EBBF). The presented method proposes a combined approach where the high modality, EBBF’s domain is first surveyed for potential areas of optimal response using a method of Steepest Feasible Descent (SFD), followed by a local search in the optimal region using a more efficient direct search method. The method of SFD is a modification of the classical method of Steepest Descent, made useful for constrained models by a penalty system including both deterministic and programmatic methods. A sensitivity-based search vector method also helps to manage situations where significant difference in magnitude exists among the design variables. Evidentiary support for these key program elements is provided using standardized test functions. The effectiveness of the method is demonstrated by seeking a minimum flexural response for a parallel beam system subject to elastic support and response constraints.

Iterative Kriging-Based Methods for Expensive Black-Box Models

Reliability-based design optimization (RBDO) in electromagnetic field problems requires the calculation of probability of failure, leading to a huge computational cost in the case of expensive models. Three different RBDO approaches using a kriging surrogate model are proposed to overcome this difficulty by introducing an approximation of the objective function and constraints. These methods use different infill sampling criteria to add samples in the process of optimization or/and in the reliability analysis. Several enrichment criteria and strategies are compared in terms of the number of evaluations and accuracy of the solution.

Kriging-based optimization design for a new style shell with black box constraints

Complex engineering applications generally have the black box and computationally expensive characteristics. Surrogate-based optimization algorithms can effectively solve expensive black box optimization problems. This paper employs the kriging predictor to construct a surrogate model and uses an initial multistart optimization process to realize the global search on this kriging model. Based on a proposed trust region framework, a local search is carried out around the current promising solution. The whole optimization algorithm is implemented to solve a new style shell design of the autonomous underwater vehicle. Based on finite element analysis, buoyancy–weight ratio, maximum von Mises stress, and buckling critical load of the new style shell are calculated and stored as expensive sample values to construct the kriging model. Finally, the better design parameters of the new shell are obtained by this proposed optimization algorithm. In addition, compared with the traditional shell, the new shell shows the stronger stability and better buoyancy–weight ratio.

Machine–Learning in Optimization of Expensive Black–Box Functions

Abstract Modern engineering design optimization often uses computer simulations to evaluate candidate designs. For some of these designs the simulation can fail for an unknown reason, which in turn may hamper the optimization process. To handle such scenarios more effectively, this study proposes the integration of classifiers, borrowed from the domain of machine learning, into the optimization process. Several implementations of the proposed approach are described. An extensive set of numerical experiments shows that the proposed approach improves search effectiveness.

고비용 블랙박스 함수의 RBF기반 근사 최적화 기법

고비용 블랙박스 함수의 RBF기반 근사 최적화 기법

Extending Expected Improvement for High-Dimensional Stochastic Optimization of Expensive Black-Box Functions

Design optimization under uncertainty is notoriously difficult when the objective function is expensive to evaluate. State-of-the-art techniques, e.g., stochastic optimization or sampling average approximation, fail to learn exploitable patterns from collected data and require a lot of objective function evaluations. There is a need for techniques that alleviate the high cost of information acquisition and select sequential simulations optimally. In the field of deterministic single-objective unconstrained global optimization, the Bayesian global optimization (BGO) approach has been relatively successful in addressing the information acquisition problem. BGO builds a probabilistic surrogate of the expensive objective function and uses it to define an information acquisition function (IAF) that quantifies the merit of making new objective evaluations. In this work, we reformulate the expected improvement (EI) IAF to filter out parametric and measurement uncertainties. We bypass the curse of dimensionality, since the method does not require learning the response surface as a function of the stochastic parameters, and we employ a fully Bayesian interpretation of Gaussian processes (GPs) by constructing a particle approximation of the posterior of its hyperparameters using adaptive Markov chain Monte Carlo (MCMC) to increase the methods robustness. Also, our approach quantifies the epistemic uncertainty on the location of the optimum and the optimal value as induced by the limited number of objective evaluations used in obtaining it. We verify and validate our approach by solving two synthetic optimization problems under uncertainty and demonstrate it by solving the oil-well placement problem (OWPP) with uncertainties in the permeability field and the oil price time series.

Toward design optimization of a Pelton turbine runner

The objective of the present paper is to propose a strategy to optimize the performance of a Pelton runner based on a parametric model of the bucket geometry, massive particle based numerical simulations and advanced optimization strategies to reduce the dimension of the design problem. The parametric model of the Pelton bucket is based on four bicubic Bezier patches and the number of free parameters is reduced to 21. The numerical simulations are performed using the finite volume particle method, which benefits from a conservative, consistent, arbitrary Lagrangian Eulerian formulation. The resulting design problem is of High-dimension with Expensive Black-box (HEB) performance function. In order to tackle the HEB problem, a preliminary exploration is performed using 2'000 sampled runners geometry provided by a Halton sequence. A cubic multivariate adaptive regression spline surrogate model is built according to the simulated performance of these runners. Moreover, an original clustering approach is proposed to decompose the design problem into four sub-problems of smaller dimensions that can be addressed with more conventional optimization techniques.

Development of a metamodel assisted sampling approach to aerodynamic shape optimization problems

A new metamodel-assisted sampling search approach applied to the aerodynamic shape optimization of turbomachinery airfoils is presented in this paper. The proposed methodology integrates a non-uniform rational B-spline (NURBS) geometry representation, a twodimensional flow analysis, and an improved metamodel driven optimization algorithm named approximated promising region identifier (APRI), which represents a momentous advancement of the existing space exploration techniques specifically for the high-dimensional expensive black-box (HEB) problems. The novel optimization method prospects the whole design space by generating sample points, reporting evaluating information using a surrogate model, and then focusing the search in the most promising region by deploying more agents. Using the integration of these adaptive tools and methods, the optimization results are considerably promising in terms of computational efficiency and performance enhancement of the turbomachinery blade airfoil shape in both design and off-design conditions.