Expensive Functions Research Articles

Kriging-assisted indicator-based evolutionary algorithm for expensive multi-objective optimization

Selection mechanisms based on performance indicators are popular for solving multi-objective optimization problems (MOPs), which can provide a comprehensive evaluation of the convergence and diversity of candidate solutions. However, these mechanisms recently encounter difficulties when tackling computationally expensive MOPs attributed to the restricted quantity of real function evaluations. To tackle this problem, this paper proposes a Kriging-assisted indicator-based evolutionary algorithm. The primary concept is to utilize the Kriging model to approximate computationally expensive objective functions, thereby reducing the computational cost, and to employ the R2 indicator to assess the quality of candidate solutions. In addition, a dual selection mechanism based on the lower confidence bound is proposed as the model management strategy to balance exploration and exploitation. The experimental results demonstrate that the proposed algorithm is competitive when compared to five representative algorithms.

Benchmarking surrogate-based optimisation algorithms on expensive black-box functions

Surrogate algorithms such as Bayesian optimisation are especially designed for black-box optimisation problems with expensive objectives, such as hyperparameter tuning or simulation-based optimisation. In the literature, these algorithms are usually evaluated with synthetic benchmarks which are well established but have no expensive objective, and only on one or two real-life applications which vary wildly between papers. There is a clear lack of standardisation when it comes to benchmarking surrogate algorithms on real-life, expensive, black-box objective functions. This makes it very difficult to draw conclusions on the effect of algorithmic contributions and to give substantial advice on which method to use when. A new benchmark library, EXPObench, provides first steps towards such a standardisation. The library is used to provide an extensive comparison of six different surrogate algorithms on four expensive optimisation problems from different real-life applications. This has led to new insights regarding the relative importance of exploration, the evaluation time of the objective, and the used model. We also provide rules of thumb for which surrogate algorithm to use in which situation. A further contribution is that we make the algorithms and benchmark problem instances publicly available, contributing to more uniform analysis of surrogate algorithms. Most importantly, we include the results of the six algorithms on all evaluated problem instances. This unique new dataset lowers the bar for researching new methods as the number of expensive evaluations required for comparison and for the creation of new surrogate models is significantly reduced.

Gradient-Enhanced Kriging for High-Dimensional Bayesian Optimization with Linear Embedding

This paper explores the application of gradient-enhanced (GE) kriging for Bayesian optimization (BO) problems with a high-dimensional parameter space. We utilize the active subspace method to embed the original parameter space in a low-dimensional subspace. The active subspace is detected by analyzing the spectrum of the empirical second-moment matrix of the gradients of the response function. By mapping the training data onto their respective subspace, the objective function and the constraint functions are efficiently approximated with low-dimensional GE-kriging models. In each cycle of the BO procedure, a new point is found by maximizing the constrained expected improvement function within a low-dimensional polytope, and it is mapped back to the original space for model evaluation. In this way, the computational costs are significantly reduced when compared with standard GE-kriging. We illustrate and assess the proposed approach with numerical experiments ranging from academic benchmark problems to aerodynamic engineering applications. The experimental results show that the proposed method is promising for optimizing high-dimensional expensive objective functions, especially for problems that exhibit a clear low-dimensional active subspace.

Bayesian parameter inference in hydrological modelling using a Hamiltonian Monte Carlo approach with a stochastic rain model

Abstract. Stochastic models in hydrology are very useful and widespread tools for making reliable probabilistic predictions. However, such models are only accurate at making predictions if model parameters are first of all calibrated to measured data in a consistent framework such as the Bayesian one, in which knowledge about model parameters is described through probability distributions. Unfortunately, Bayesian parameter calibration, a. k. a. inference, with stochastic models, is often a computationally intractable problem with traditional inference algorithms, such as the Metropolis algorithm, due to the expensive likelihood functions. Therefore, the prohibitive computational cost is often overcome by employing over-simplified error models, which leads to biased parameter estimates and unreliable predictions. However, thanks to recent advancements in algorithms and computing power, fully fledged Bayesian inference with stochastic models is no longer off-limits for hydrological applications. Our goal in this work is to demonstrate that a computationally efficient Hamiltonian Monte Carlo algorithm with a timescale separation makes Bayesian parameter inference with stochastic models feasible. Hydrology can potentially take great advantage of this powerful data-driven inference method as a sound calibration of model parameters is essential for making robust probabilistic predictions, which can certainly be useful in planning and policy-making. We demonstrate the Hamiltonian Monte Carlo approach by detailing a case study from urban hydrology. Discussing specific hydrological models or systems is outside the scope of our present work and will be the focus of further studies.

A Performance Indicator-Based Infill Criterion for Expensive Multi-/Many-Objective Optimization

In surrogate-assisted multi-/many-objective evolutionary optimization, each solution normally has an approximated value on each objective, resulting in increased difficulties in selecting solutions for expensive objective evaluations due to complicated trade-off between different objectives and accumulated uncertainty in the approximation of the objective functions. Thus, it is highly challenging to design an efficient model management strategy for surrogate-assisted expensive multi-/many-objective optimization. In this paper, a surrogate model is built for each objective function, based on which a set of promising candidate solutions are found. Additionally, a Gaussian process model is constructed to approximate a newly designed performance indicator measuring both convergence and diversity properties of individual solutions. Finally, the solution of the found candidate solutions having the maximum expected improvement in terms of the performance indicator is selected for evaluation using the expensive objective functions. Comparative experiments are conducted on 3-, 5-, and 10-objective DTLZ, WFG, and MaF test functions, as well as two real-world applications. The experimental results show that the proposed method is competitive compared to five state-of-the-art surrogate-assisted evolutionary algorithms for expensive multi-/many-objective optimization.

Multi-Objective Bayesian Optimization Supported by an Expected Pareto Distance Change

Abstract The solution to global (a posteriori) multi-objective optimization problems traditionally relies on population-based algorithms, which are very effective in generating a Pareto front. Unfortunately, due to the high number of function evaluations, these methods are of limited use in problems that involve expensive black-box functions. In recent years, multi-objective Bayesian optimization has emerged as a powerful alternative; however, in many applications, these methods fail to generate a diverse and well-spread Pareto front. To address this limitation, our work introduces a novel acquisition function (AF) for multi-objective Bayesian optimization that produces more informative acquisition landscapes. The proposed AF comprises two terms, namely, a distance-based metric and a diversity index. The distance-based metric, referred to as the expected Pareto distance change, promotes the evaluation of high-performing designs and repels low-performing design zones. The diversity term prevents the evaluation of designs that are similar to the ones contained in the current sampling plan. The proposed AF is studied using seven analytical problems and in the design optimization of sandwich composite armors for blast mitigation, which involves expensive finite element simulations. The results show that the proposed AF generates high-quality Pareto sets outperforming well-established methods such as the Euclidean-based expected improvement function. The proposed AF is also compared with respect to a recently proposed multi-objective approach. The difference in their performance is problem dependent.

Multi-fidelity optimization of metal sheets concerning manufacturability in deep-drawing processes

Multi-fidelity optimization, which complements an expensive high-fidelity function with cheaper low-fidelity functions, has been successfully applied in many fields of structural optimization. In the present work, an exemplary cross-die deep-drawing optimization problem is investigated to compare different objective functions and to assess the performance of a multi-fidelity efficient global optimization technique. To that end, hierarchical kriging is combined with an infill criterion called variable-fidelity expected improvement. Findings depend significantly on the choice of objective function, highlighting the importance of careful consideration when defining an objective function. We show that one function based on the share of bad elements in a forming limit diagram is not well suited to optimize the example problem. In contrast, two other definitions of objective functions, the average sheet thickness reduction and an averaged limit violation in the forming limit diagram, confirm the potential of a multi-fidelity approach. They significantly reduce computational cost at comparable result quality or even improve result quality compared to a single-fidelity optimization.

Generative Multiform Bayesian Optimization.

Many real-world problems, such as airfoil design, involve optimizing a black-box expensive objective function over complex-structured input space (e.g., discrete space or non-Euclidean space). By mapping the complex-structured input space into a latent space of dozens of variables, a two-stage procedure labeled as generative model-based optimization (GMO), in this article, shows promise in solving such problems. However, the latent dimension of GMO is hard to determine, which may trigger the conflicting issue between desirable solution accuracy and convergence rate. To address the above issue, we propose a multiform GMO approach, namely, generative multiform optimization (GMFoO), which conducts optimization over multiple latent spaces simultaneously to complement each other. More specifically, we devise a generative model which promotes a positive correlation between latent spaces to facilitate effective knowledge transfer in GMFoO. And furthermore, by using Bayesian optimization (BO) as the optimizer, we propose two strategies to exchange information between these latent spaces continuously. Experimental results are presented on airfoil and corbel design problems and an area maximization problem as well to demonstrate that our proposed GMFoO converges to better designs on a limited computational budget.

Parallel efficient global optimization by using the minimum energy criterion

In optimization problems, the expensive black-box function implies severely restricted budgets in terms of evaluation. Some Bayesian optimization methods have been proposed to solve this problem, such as expected improvement (EI) and hierarchical expected improvement (HEI). Neither EI nor HEI is parallel, which depends on a one-point-at-a-time strategy. In this work, a new parallel Bayesian framework based on the minimum energy criterion is proposed to improve these popular one-point methods. The new proposed framework can save time and costs by reducing the number of iterations and avoid the local optimization trap by encouraging the exploration of the optimization space. Additionally, a shrink-augment strategy is also introduced to correct the local surrogate model for the black-box function adaptively, which could also benefit the optimization. Some numerical and illustrative experiments are presented to demonstrate the superiority of our proposed method over some other Bayesian methods. The results show that the novel framework can balance exploitation and exploration well and has great performance in global optimization.

A reliable region information driven kriging-assisted multiobjective rough fuzzy clustering algorithm for color image segmentation

Multiobjective clustering algorithms (MOCAs) are becoming increasingly popular with the merit of segmenting images from multiple perspectives. The performances of MOCAs highly depend on their fitness functions. However, most existing MOCAs adopt one pair of complementary fitness functions, which always measure intra-class compactness and inter-class separation, respectively. This may result in insufficient ability to recognize and mine feature structures from complex images. Moreover, information within color images, such as region information and uncertain information, can barely receive enough attention in MOCAs. To resolve these problems, we propose a reliable region information driven Kriging-assisted multiobjective rough fuzzy clustering algorithm (RRI-KMRFC). Firstly, a reliability-based region information extraction strategy (RRIES) is designed to obtain reliable image information with satisfactory regional homogeneity and abundant image details. Secondly, the derived region information is used to construct three complementary fitness functions, focusing on rough intra-class compactness, rough inter-class separation, and regional consistency, respectively. Such fitness functions can effectively identify the clustering structure, maintain contour details, and characterize uncertain information from color images. To efficiently optimize the proposed functions, an incremental Kriging-assisted evolutionary framework is presented to decrease the expensive function evaluations in which an improved infill sampling strategy is devised to assist in finding unexplored areas in the decision space. Finally, a rough fuzzy clustering validity index with reliable region information is proposed to select the optimal trade-off solution. Experiments performed on Berkeley and Weizmann images confirm the effectiveness and robustness of RRI-KMRFC.

Efficient Calibration of Groundwater Contaminant Transport Models Using Bayesian Optimization.

Numerical modeling is a significant tool to understand the dynamic characteristics of contaminants transport in groundwater. The automatic calibration of highly parametrized and computationally intensive numerical models for the simulation of contaminant transport in the groundwater flow system is a challenging task. While existing methods use general optimization techniques to achieve automatic calibration, the large numbers of numerical model evaluations required in the calibration process lead to high computing overhead and limit the efficiency of model calibration. This paper presents a Bayesian optimization (BO) method for efficient calibration of numerical models of groundwater contaminant transport. A Bayes model is built to fully represent calibration criteria and derive the objective function for model calibration. The efficiency of model calibration is made possible by the probabilistic surrogate model and the expected improvement acquisition function in BO. The probabilistic surrogate model approximates the computationally expensive objective function with a closed-form expression that can be computed efficiently, while the expected improvement acquisition function proposes the most promising model parameters to improve the fitness to the calibration criteria and reduce the uncertainty of the surrogate model. These schemes allow us to find the optimized model parameters effectively by using a small number of numerical model evaluations. Two case studies for the calibration of the Cr(VI) transport model demonstrate that the BO method is effective and efficient in the inversion of hypothetical model parameters, the minimization of the objective function, and the adaptation of different model calibration criteria. Specifically, this promising performance is achieved within 200 numerical model evaluations, which substantially reduces the computing budget for model calibration.

Efficient model‐correction based reliability analysis of uncertain dynamical systems

AbstractIn this contribution, a model‐correction‐based strategy is applied for efficient reliability analysis of uncertain dynamical systems based on a low‐fidelity (LF) model whose outcomes are corrected in a probabilistic sense to represent the more realistic outcomes of the high‐fidelity (HF) model. In the model‐correction approach utilized, the LF model is calibrated to the HF model close to the so‐called most probable point (MPP) in standard normal space, which allows a more realistic assessment of the considered complex dynamical system. Since only few expensive limit state function evaluations of the HF model are required, an efficient reliability analysis is enabled. In an application example, the LF model describes an existing single span railway bridge modelled as simply supported Euler‐Bernoulli beam subjected to moving single forces representing the axle loads of a moving train. The HF modelling approach accounts for the bridge‐train interaction by modelling the passing train as mass‐spring‐damper (MSD) system, however increasing the computational effort of the limit state function evaluations. The failure probabilities evaluated with the model‐correction approach are contrasted and discussed with the failure probabilities of the sophisticated bridge‐train interaction model. It is shown that the model‐correction‐based approach provides reliable failure probability prediction of the HF model while leading to a significant reduction in computational effort.

Bayesian optimization for demographic inference.

Inference of demographic histories of species and populations is one of the central problems in population genetics. It is usually stated as an optimization problem: find a model's parameters that maximize a certain log-likelihood. This log-likelihood is often expensive to evaluate in terms of time and hardware resources, critically more so for larger population counts. Although genetic algorithm-based solution has proven efficient for demographic inference in the past, it struggles to deal with log-likelihoods in the setting of more than three populations. Different tools are therefore needed to handle such scenarios. We introduce a new optimization pipeline for demographic inference with time consuming log-likelihood evaluations. It is based on Bayesian optimization, a prominent technique for optimizing expensive black box functions. Comparing to the existing widely used genetic algorithm solution, we demonstrate new pipeline's superiority in the limited time budget setting with four and five populations, when using the log-likelihoods provided by the moments tool.

A surrogate-assisted evolutionary algorithm based on multi-population clustering and prediction for solving computationally expensive dynamic optimization problems

The computationally expensive dynamic optimization problems (CEDOPs) arise when the objective function of a dynamic optimization problem is prohibitively expensive, or even almost impossible, to obtain. Such problems are extremely challenging from a computational perspective, due to the combination of dynamic environments and computationally expensive objective functions. In this paper, we present a surrogate-assisted evolutionary algorithm based on multi-population clustering and prediction (SAEA/MPCP) to tackle CEDOPs. In the solution process, SAEA/MPCP employs three basic strategies. First, to improve the solution speed and the ability to interact with the environment of the algorithm, the initial population were selected using the strategy of clustering and differential prediction. Second, in order to increase the distribution and diversity of individuals and improve the solution accuracy of the algorithm, the generation of sample points for surrogate model was performed using integrated multi-population strategies. Third, we replace the two kinds of high and low confidence samples in the Co-kriging model with the generated samples from the previous strategy, which further improves the construction and optimization efficiency of the surrogate model. Finally, we set up two types of CEDOPs with different degrees of computational cost and experimentally compare them with a variety of state-of-the-art dynamic optimization algorithms (DOAs), respectively. The experiments fully demonstrate the effectiveness of SAEA/MPCP algorithm to some of the best known DOAs.

Expensive Multiobjective Optimization Based on Information Transfer Surrogate

Objective value estimation based on computationally efficient surrogate models is widely used to reduce the computational cost in solving expensive multiobjective optimization problems (MOPs). However, due to the scarcity of training data and the lack of data sharing between training tasks in a surrogate-based system, the estimation effectiveness of the surrogate models might not be satisfactory. In this study, we present a novel surrogate methodology based on information transfer to deal with this problem. Particularly, in the proposed framework, the objectives of an MOP that may have little apparent similarity or correlation are linearly mapped to a number of related tasks. Afterward, the related tasks are used to train a multitask Gaussian process (MTGP). MTGP expands the training data leading to more confident learning of the parameters of the model. The predicted values of the objective functions can be obtained by a reverse mapping from the learned MTGP model. In this way, the computational burden of the expensive objective functions of an MOP can be substantially reduced while maintaining good estimation accuracy. MTGP facilitates mutual information transfer across tasks, avoids learning from scratch for new tasks, and captures the underlying structural information between tasks. The proposed surrogate approach is merged into MOEA/D to address MOPs. Experimental tests under various scenarios indicate that the resultant algorithm outperforms other state-of-the-art surrogate-based multiobjective optimization algorithms.

Efficient model-correction-based reliability analysis of uncertain dynamical systems

The scope of this paper is to apply a model-correction-based strategy for efficient reliability analysis of uncertain dynamical systems based on a low-fidelity (LF) model whose outcomes are corrected in a probabilistic sense to represent the more realistic outcomes of a high-fidelity (HF) model. In the model-correction approach utilized, the LF model is calibrated to the HF model close to the so-called most probable point in standard normal space, which allows a more realistic assessment of the considered complex dynamical system. Since only few expensive limit state function evaluations of the HF model are required, an efficient reliability analysis is enabled. In an application example, the LF model describes an existing single-span railway bridge modelled as simply supported Euler–Bernoulli beam subjected to moving single forces representing the axle loads of a moving train. The HF modelling approach accounts for the bridge–train interaction by modelling the passing train as mass-spring-damper system, however increasing the computational effort of the limit state function evaluations. Failure probabilities evaluated with the model-correction approach are contrasted and discussed with failure probabilities of the sophisticated bridge–train interaction model evaluated with the first-order reliability method (FORM). It is demonstrated that the efficiency of the method depends on the correlation between the LF and the HF model. A comparison of the results of FORM and the model-correction-based approach shows that the latter provides reliable failure probability prediction of the HF model while leading to a significant reduction in computational effort.

Batched Data-Driven Evolutionary Multiobjective Optimization Based on Manifold Interpolation

Multiobjective optimization problems are ubiquitous in real-world science, engineering, and design optimization problems. It is not uncommon that the objective functions are as a black box, the evaluation of which usually involve time-consuming and/or costly physical experiments. Data-driven evolutionary optimization can be used to search for a set of nondominated tradeoff solutions, where the expensive objective functions are approximated as a surrogate model. In this article, we propose a framework for implementing batched data-driven evolutionary multiobjective optimization (EMO). It is so general that any off-the-shelf EMO algorithms can be applied in a plug-in manner. There are two unique components: 1) based on the Karush–Kuhn–Tucker conditions, a manifold interpolation approach that explores more diversified solutions with a convergence guarantee along the manifold of the approximated Pareto-optimal set and 2) a batch recommendation approach that reduces the computational time of the data-driven evolutionary optimization process by evaluating multiple samples at a time in parallel. Comparing against seven state-of-the-art surrogate-assisted evolutionary algorithms, experiments on 168 benchmark test problem instances with various properties and a real-world application on hyper-parameter optimization fully demonstrate the effectiveness and superiority of our proposed framework, which is featured with a faster convergence and a stronger resilience to various Pareto-optimal front shapes.

Slow TCA flux and ATP production in primary solid tumours but not metastases.

Tissues derive ATP from two pathways-glycolysis and the tricarboxylic acid (TCA) cycle coupled to the electrontransport chain. Most energy in mammals is produced via TCA metabolism1. In tumours, however, the absolute rates of these pathways remain unclear. Here we optimize tracer infusion approaches to measure the rates of glycolysis and the TCA cycle in healthy mouse tissues, Kras-mutant solid tumours, metastases and leukaemia. Then,given the rates of these two pathways, we calculate total ATP synthesis rates. We find that TCA cycle flux is suppressed in all five primary solid tumour models examined and is increased in lung metastases of breast cancer relative to primary orthotopic tumours. As expected, glycolysis flux is increased in tumours compared with healthy tissues (the Warburg effect2,3), but this increase is insufficient to compensate for low TCA flux in terms of ATP production. Thus, instead of being hypermetabolic, as commonly assumed, solid tumours generally produce ATP at a slower than normal rate. In mouse pancreatic cancer, this is accommodated by the downregulation of protein synthesis, one of this tissue's major energy costs. We propose that, as solid tumours develop, cancer cells shed energetically expensive tissue-specific functions, enabling uncontrolled growth despite a limited ability to produce ATP.

An ANN-assisted efficient enriched finite element method via the selective enrichment of moment fitting

Enrichment techniques that employ nonconforming mesh are effective in modeling structures with discontinuities because numerical issues regarding mesh quality are avoided. However, the accurate integration of the bilinear and linear forms on the discretized domain, which is required in the standard Galerkin-based finite element method, is computationally expensive due to the complexity of the enriched basis function. In this paper, we present a fast and accurate alternative method of numerical integration using nonlinear regression enabled by a multi-perceptron feedforward neural network. The relationship between an implicitly represented geometry and the quadrature rule derived from the moment fitting method is predicted by the neural network; the neural network-based regression model circumvents complex computation and significantly reduces the overall online time by avoiding expensive function evaluations. Through the selected numerical examples, we demonstrate the efficiency and accuracy of the current method, as well as the flexibility of the trained network to be used in different contexts.

Chiral-Induced Spin Selectivity and Non-equilibrium Spin Accumulation in Molecules and Interfaces: A First-Principles Study.

Electrons moving through chiral molecules are selected according to their spin orientation and the helicity of the molecule, an effect known as chiral-induced spin selectivity (CISS). The underlying physical mechanism is not yet completely understood. To help elucidate this mechanism, a non-equilibrium Green's function method, combined with a Landauer approach and density functional theory, is applied to carbon helices contacted by gold electrodes, resulting in spin polarization of transmitted electrons. Spin polarization is also observed in the non-equilibrium electronic structure of the junctions. While this spin polarization is small, its sign changes with the direction of the current and with the handedness of the molecule. While these calculations were performed with a pure exchange-correlation functional, previous studies suggest that computationally more expensive hybrid functionals may lead to considerably larger spin polarization in the electronic structure. Thus, non-equilibrium spin polarization could be a key component in understanding the CISS mechanism.