Expensive Cost Functions Research Articles

A Bayesian sampling optimisation strategy for finite element model updating

AbstractModel Updating (MU) aims to estimate the unknown properties of a physical system of interest from experimental observations. In Finite Element (FE) models, these unknowns are the elements’ parameters. Typically, besides model calibration purposes, MU and FEMU procedures are employed for the Non-Destructive Evaluation (NDE) and damage assessment of structures. In this framework, damage can be located and quantified by updating the parameters related to stiffness. However, these procedures require the minimisation of a cost function, defined according to the difference between the model and the experimental data. Sophisticated FE models can generate expensive and non-convex cost functions, which minimization is a non-trivial task. To deal with this challenging optimization problem, this work makes use of a Bayesian sampling optimisation technique. This approach consists of generating a statistical surrogate model of the underlying cost function (in this case, a Gaussian Process is used) and applying an acquisition function that drives the intelligent selection of the next sampling point, considering both exploitation and exploration needs. This results in a very efficient yet very powerful optimization technique, necessitating of minimal sampling volume. The performance of this proposed scheme is then compared to three well-established global optimisation algorithms. This investigation is performed on numerical and experimental case studies based on the famous Mirandola bell tower.

An Efficient Hybrid Multi-Objective Optimization Method Coupling Global Evolutionary and Local Gradient Searches for Solving Aerodynamic Optimization Problems

Aerodynamic shape optimization is frequently complicated and challenging due to the involvement of multiple objectives, large-scale decision variables, and expensive cost function evaluation. This paper presents a bilayer parallel hybrid algorithm framework coupling multi-objective local search and global evolution mechanism to improve the optimization efficiency and convergence accuracy in high-dimensional design space. Specifically, an efficient multi-objective hybrid algorithm (MOHA) and a gradient-based surrogate-assisted multi-objective hybrid algorithm (GS-MOHA) are developed under this framework. In MOHA, a novel multi-objective gradient operator is proposed to accelerate the exploration of the Pareto front, and it introduces new individuals to enhance the diversity of the population. Afterward, MOHA achieves a trade-off between exploitation and exploration by selecting elite individuals in the local search space during the evolutionary process. Furthermore, a surrogate-assisted hybrid algorithm based on the gradient-enhanced Kriging with the partial least squares(GEKPLS) approach is established to improve the engineering applicability of MOHA. The optimization results of benchmark functions demonstrate that MOHA is less constrained by dimensionality and can solve multi-objective optimization problems (MOPs) with up to 1000 decision variables. Compared to existing MOEAs, MOHA demonstrates notable enhancements in optimization efficiency and convergence accuracy, specifically achieving a remarkable 5–10 times increase in efficiency. In addition, the optimization efficiency of GS-MOHA is approximately five times that of MOEA/D-EGO and twice that of K-RVEA in the 30-dimensional test functions. Finally, the multi-objective optimization results of the airfoil shape design validate the effectiveness of the proposed algorithms and their potential for engineering applications.

Adversarially robust Bayesian optimization for efficient auto‐tuning of generic control structures under uncertainty

AbstractThe performance of optimization‐ and learning‐based controllers critically depends on the selection of several tuning parameters that can affect the closed‐loop control performance and constraint satisfaction in highly nonlinear and nonconvex ways. Due to the black‐box nature of the relationship between tuning parameters and general closed‐loop performance measures, there has been a significant interest in automatic calibration (i.e., auto‐tuning) of complex control structures using derivative‐free optimization methods, including Bayesian optimization (BO) that can handle expensive unknown cost functions. Nevertheless, an open challenge when applying BO to auto‐tuning is how to effectively deal with uncertainties in the closed‐loop system that cannot be attributed to a lumped, small‐scale noise term. This article addresses this challenge by developing an adversarially robust BO (ARBO) method that is particularly suited to auto‐tuning problems with significant time‐invariant uncertainties in an expensive system model used for closed‐loop simulations. ARBO relies on a Gaussian process model that jointly describes the effect of the tuning parameters and uncertainties on the closed‐loop performance. From this joint Gaussian process model, ARBO uses an alternating confidence‐bound procedure to simultaneously select the next candidate tuning and uncertainty realizations, implying only one expensive closed‐loop simulation is needed at each iteration. The advantages of ARBO are demonstrated on two case studies, including an illustrative problem and auto‐tuning of a nonlinear model predictive controller using a benchmark bioreactor problem.

Bi-objective Bayesian optimization of engineering problems with cheap and expensive cost functions

Bi-objective Bayesian optimization of engineering problems with cheap and expensive cost functions

INSTANTANEOUS PARAMETER IDENTIFICATION FOR MILLING FORCE MODELS USING BAYESIAN OPTIMIZATION

The comparison between measured and simulated machining forces enables the evaluation of workpiece quality, process stability, and tool wear condition. To compute the machining forces that occur, mechanistic cutting force models are typically used. The cutting force coefficients (CFCs) of mechanistic force models are directly linked to the mechanics of chip formation and, thus, depend on the tool-workpiece combination and on the prevailing cutting conditions. CFCs are usually identified via the average cutting force identification method, which requires the execution of cutting tests under defined test conditions. Hence, determining CFCs for different cutting conditions is time-consuming and expensive. In this paper, the performance of an instantaneous CFC identification approach based on Bayesian Optimization during the machining of arbitrary workpiece geometries is studied. Bayesian Optimization is well suited for global optimization problems with computationally expensive cost functions. The simulated cutting forces are calculated using a dexel-based cutter workpiece engagement simulation and the actual cutting forces are measured during the machining process using a dynamometer. Thus, an efficient identification of CFCs could be achieved.

Solving Multiphysics, Multiparameter, Multimodal Inverse Problems: An Application to NMR Relaxation in Porous Media

A general and robust Bayesian optimization framework for the extraction of intrinsic physical properties from an integration of pore-scale forward modeling and experimental measurements of macroscopic system responses is developed. The efficiency of the scheme, which utilizes Gaussian process regression, enables the simultaneous extraction of multiple intrinsic physical properties with a minimal number of function evaluations. Here it is applied to nuclear magnetic resonance (NMR) relaxation responses, paving the way for inverse problem approaches to digital rock physics given its general nature. NMR relaxation responses of fluids in porous media may be described by sums of multiexponential decays resulting in a relaxation time distribution. The shape of this distribution is dependent on intrinsic physical system properties, but also effects like diffusion coupling between different relaxation regimes in heterogeneous porous materials. Forward models based on high-resolution images are employed to naturally incorporate structural heterogeneity and diffusive motion without limiting assumptions. Extracting the required multiple intrinsic parameters of the system poses an ill-conditioned multiphysics multiparameter inverse problem where multiple scales are covered by the underlying microstructure. Exploration of the multidimensional search space given an expensive cost function makes an efficient solution strategy mandatory. We propose a workflow to match experimental measurements with simulations via Bayesian optimization, with special attention paid to the multimodal nature of the topography of the objective function using solution space partitioning. A multimodal search strategy using state-of-the-art evolutionary algorithms and gradient-based optimization algorithms guarantees that the multimodal nature is captured. The workflow is demonstrated on ${T}_{2}$ relaxation responses of Bentheimer sandstone, extracting three physical parameters simultaneously: the surface relaxivity of quartz grains, the effective transverse relaxation time, and the effective diffusion coefficient in clay regions. Multiple mathematically sound and physically plausible solutions corresponding to global minimum and multiple local minima of the objective function are identified within a limited number of function evaluations. Importantly, the shape of the experimental ${T}_{2}$ distribution is recovered almost perfectly, enabling the use of classical interpretation techniques and local analysis of responses based on numerical simulation.

Multiobjective Bayesian optimization and joint inversion for active sensor fusion

A critical decision process in data acquisition for mineral and energy resource exploration is how to efficiently combine a variety of sensor types and how to minimize the total cost. We have developed a probabilistic framework for multiobjective optimization and inverse problems given an expensive cost function for allocating new measurements. This new method is devised to jointly solve multilinear forward models of 2D sensor data and 3D geophysical properties using sparse Gaussian process kernels while taking into account the cross-variances of different parameters. Multiple optimization strategies are tested and evaluated on a set of synthetic and real geophysical data. We determine the advantages on a specific example of a joint inverse problem, recommending where to place new drill-core measurements given 2D gravity and magnetic sensor data; the same approach can be applied to a variety of remote sensing problems with linear forward models — ranging from constraints limiting surface access for data acquisition to adaptive multisensor positioning.

Discrete Adjoint Optimization of a Hypersonic Inlet

Shape optimization of modern three-dimensional hypersonic inlets requires many geometric design parameters. Because the flow physics is complex, a Reynolds-averaged Navier–Stokes (RANS) analysis is also desirable when performing optimization. A computationally efficient means of handling many-parameter optimization with expensive cost functions is to use gradient-based searches informed by an adjoint flow solution. In this work, a discrete adjoint method for high-speed flows is documented. It is implemented in the open-source compressible-flow solver Eilmer. One difficulty with extending a hypersonic flow solver to include an adjoint solver is the differentiation of complex, nonlinear (and sometimes nondifferentiable) algorithms. Here, it is shown that difficulty can be overcome using flow Jacobians constructed with finite differences based on complex variables. The coupled flow and adjoint solvers are packaged together with a gradient-based optimizer. The flow and adjoint solvers are verified and validated for high-speed RANS analysis. The discrete adjoint method for optimization is demonstrated using the NASA P2 hypersonic inlet as a test case. The results show that the optimization method can remove an undesirable shock present in the original geometry, while achieving the desired compression ratio and improving overall performance metrics.

A Multifractal-Guided Multilevel Surrogate Model-Based Evolutionary Algorithm for Expensive Multiobjective Problems

In applied engineering, there are tremendous optimization problems which are multiobjective problems. Meanwhile, a number of them require large amount of time to evaluate their expensive cost function during optimization procedures. This kind of problems can be either financially expensive due to significant computational resources being required or time expensive due to numerous computational complexity. Aiming to this kind of problems, this paper proposed a multilevel surrogate model-based evolutionary algorithm. The proposed method employs DACE modeling method at the beginning to obtain a global trend in the decision domain. When more and more samples are involved and the sample distribution presents a trend or a manifold, the SVR model is utilized as a second-level surrogate model to achieve a better local search. The model transition is determined by the multifractal analysis on the solution set. Experimental results on ZDT and DTLZ standard test cases demonstrate that the time for EGO modeling can be reduced, and the accuracy can be better balanced by comparing to existing SVR and EGO methods.

A Guided Hybrid Genetic Algorithm for Feature Selection with Expensive Cost Functions

We present a guided hybrid genetic algorithm for feature selection which is tailored to minimize the number of cost function evaluations. Guided variable elimination is used to make the stochastic backward search of the genetic algorithm much more efficient. Guiding means that a promising feature set is selected from a population and suggestions (for example by a trained Random Forest) are made which variable could be removed. It uses implicit diversity management and is able to return multiple optimal solutions if present, which might be important for interpreting the results. It uses a dynamic cost function that avoids prescribing an expected upper limit of performance or the number of features of the optimal solution. We illustrate the performance of the algorithm on artificial data, and show that the algorithm provides accurate results and is very efficient in minimizing the number of cost function evaluations.

Linear Aperiodic Array Synthesis Using an Improved Genetic Algorithm

A novel algorithm on beam pattern synthesis for linear aperiodic arrays with arbitrary geometrical configuration is presented in this paper. Linear aperiodic arrays are attractive for their advantages on higher spatial resolution and lower sidelobe. However, the advantages are attained at the cost of solving a complex non-linear optimization problem. In this paper, we explain the Improved Genetic Algorithm (IGA) that simultaneously adjusts the weight coefficients and inter-sensor spacings of a linear aperiodic array in more details and extend the investigations to include the effects of mutual coupling and the sensitivity of the Peak Sidelobe Level (PSL) to steering angles. Numerical results show that the PSL of the synthesized beam pattern has been successfully lowered with the IGA when compared with other techniques published in the literature. In addition, the computational cost of our algorithm can be as low as 10% of that of a recently reported genetic algorithm based synthesis method. The excellent performance of IGA makes it a promising optimization algorithm where expensive cost functions are involved.

HYBRID-SURROGATE-MODEL-BASED EFFICIENT GLOBAL OPTIMIZATION FOR HIGH-DIMENSIONAL ANTENNA DESIGN

E-cient global optimization has been extensively used in problems with expensive cost functions. However, this method is not suitable for high-dimensional problems. In this paper, the radial basis function network is introduced into the e-cient global optimization, to avoid local optima and achieve a fast convergence for high-dimensional optimization. Our algorithm is applied to a 12-dimensional optimization of a transmitting antenna. Compared to the genetic-algorithm-based e-cient global optimization and the difierential evolution strategy, our algorithm converges to the global optimal value more e-ciently.

GATE: a genetic algorithm designed for expensive cost functions

The present paper introduces the GATE algorithm, which was specifically designed to lessen the cost of GAs for engineering design problems. The main strength of the algorithm is to find a good design using a relatively low number of function evaluations. The heart of the algorithm is a new heuristic called territorial core evolution (TE). TE regulates the mean step and the permitted search area of the GAs’ random search operators, depending on the state of convergence of the algorithm. As a result, more global or more local searches are made when necessary to better fit the specificities of each problem. GATE, which was initially calibrated using a Gaussian landscape generator as test case, is shown to be very efficient to solve that kind of topology, especially for large scale problems. Application of the GATE algorithm to various classical test cases allows a better understanding of the strengths and limitations of this algorithm.

Shape optimization for drag reduction in linked bodies using evolution strategies

We present results from the shape optimization of linked bodies for drag reduction in simulations of incompressible flow at moderate Reynolds numbers. The optimization relies on the covariance matrix adaptation evolution strategy (CMA-ES) and the flow simulations use vortex methods with the Brinkman penalization to enforce boundary conditions in complex bodies. We exploit the inherent parallelism of CMA-ES, by implementing a multi-host framework which allows for the distribution of the expensive cost function evaluations across parallel architectures, without being limited to one computing facility. This study repeats in silico for the first time Ingo Rechenberg’s pioneering wind tunnel experiments for drag reduction that led to the inception of evolution strategies. The simulations confirm that the results of these experimental studies indicate a flat plate is not the optimal solution for drag reduction in linked bodies. We present the vorticity field of the flow and identify the governing mechanisms for this drag reduction by the slightly corrugated linked plate configuration.

A HYBRID OPTIMIZED ALGORITHM BASED ON EGO AND TAGUCHI'S METHOD FOR SOLVING EXPENSIVE EVALUATION PROBLEMS OF ANTENNA DESIGN

In this paper, we propose a hybrid optimization approach that combines the E-cient Global Optimization (EGO) algorithm with Taguchi's method. This hybrid optimized algorithm is suited for problems with expensive cost functions. As a Bayesian analysis optimization algorithm, EGO algorithm begins with fltting the Kriging model with n sample points and flnds the (n + 1)th point where the expected improvement is maximized to update the model. We employ Taguchi's method in EGO to obtain the (n + 1)th point in this paper. A numerical simulation demonstrates that our algorithm has advantage over the original EGO. Finally, we apply this hybrid optimized algorithm to optimize an ultra-wide band (UWB) transverse electromagnetic (TEM) horn antenna and a linear antenna array. Compared to Taguchi's method and the Integer Coded Difierential Evolution Strategy, our algorithm converges to the global optimal value more e-ciently.

A simultaneous parameter adaptation scheme for genetic algorithms with application to phased array synthesis

Genetic algorithms are commonly used to solve many optimization and synthesis problems. An important issue facing the user is the selection of genetic algorithm parameters, such as mutation rate, mutation range, and number of crossovers. This paper demonstrates a real-valued genetic algorithm that simultaneously adapts several such parameters during the optimization process. This adaptive algorithm is shown to outperform its static counterparts when used to synthesize the phased array weights to satisfy specified far-field sidelobe constraints, and can perform amplitude-only, phase-only, and complex weight synthesis. When compared to conventional static parameter implementations, computation time is saved in two ways: 1) The algorithm converges faster and 2) the need to tune parameters by hand (generally done by repeatedly running the code with different parameter choices) is greatly reduced. By requiring less iteration to solve a given problem, this approach may benefit electromagnetic optimization problems with expensive cost functions, since genetic algorithms generally require many function evaluations to converge. The adaptive process also provides insight into the qualitative importance of parameters, and dynamically adjusting the mutation range is found to be especially beneficial.