Surrogate-based Method Research Articles

A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods

Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying constraints. In many cases, the lack of explicit mathematical formulas of the objectives and constraints may necessitate conducting computationally expensive and time-consuming experiments and/or simulations. As another challenge, these problems may have either convex or nonconvex or even disconnected Pareto frontier consisting of Pareto optimal solutions. Because of the existence of many such solutions, typically, a decision maker is required to select the most preferred one. In order to deal with the high computational cost, surrogate-based methods are commonly used in the literature. This paper surveys surrogate-based methods proposed in the literature, where the methods are independent of the underlying optimization algorithm and mitigate the computational burden to capture different types of Pareto frontiers. The methods considered are classified, discussed and then compared. These methods are divided into two frameworks: the sequential and the adaptive frameworks. Based on the comparison, we recommend the adaptive framework to tackle the aforementioned challenges.

Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection

GOMORS is a parallel response surface-assisted evolutionary algorithm approach to multi-objective optimization that is designed to obtain good non-dominated solutions to black box problems with relatively few objective function evaluations. GOMORS uses Radial Basic Functions to iteratively compute surrogate response surfaces as an approximation of the computationally expensive objective function. A multi objective search utilizing evolution, local search, multi method search and non-dominated sorting is done on the surrogate radial basis function surface because it is inexpensive to compute. A balance between exploration, exploitation and diversification is obtained through a novel procedure that simultaneously selects evaluation points within an algorithm iteration through different metrics including Approximate Hypervolume Improvement, Maximizing minimum domain distance, Maximizing minimum objective space distance, and surrogate-assisted local search, which can be computed in parallel. The results are compared to ParEGO (a kriging surrogate method solving many weighted single objective optimizations) and the widely used NSGA-II. The results indicate that GOMORS outperforms ParEGO and NSGA-II on problems tested. For example, on a groundwater PDE problem, GOMORS outperforms ParEGO with 100, 200 and 400 evaluations for a 6 dimensional problem, a 12 dimensional problem and a 24 dimensional problem. For a fixed number of evaluations, the differences in performance between GOMORS and ParEGO become larger as the number of dimensions increase. As the number of evaluations increase, the differences between GOMORS and ParEGO become smaller. Both surrogate-based methods are much better than NSGA-II for all cases considered.

Surrogate modeling based on an adaptive network and granular computing

Reducing the number of evaluations of expensive fitness functions is one of the main concerns in evolutionary algorithms, especially when working with instances of contemporary engineering problems. As an alternative to this efficiency constraint, surrogate-based methods are grounded in the construction of approximate models that estimate the solutions' fitness by modeling the relationships between solution variables and their performance. This paper proposes a methodology based on granular computing for the construction of surrogate models for evolutionary algorithms. Under the proposed method, granules are associated with representative solutions of the problem under analysis. New solutions are evaluated with the expensive (original) fitness function only if they are not already covered by an existing granule. The parameters defining granules are periodically adapted as the search goes on using a neuro-fuzzy network that does not only reduce the number of fitness function evaluations, but also provides better convergence capabilities. The proposed method is evaluated on classical benchmark functions and on a recent benchmark created to test large-scale optimization models. Our results show that the proposed method considerably reduces the actual number of fitness function evaluations without significantly degrading the quality of solutions.

Variable-resolution shape optimisation: low-fidelity model selection and scalability

Computationally-efficient aerodynamic shape optimisation can be realised using surrogate-based methods. By shifting the optimisation burden to a cheap and yet reasonably accurate surrogate model, the design cost can be substantially reduced, particularly if the surrogate exploits an underlying physics-based low-fidelity model (e.g., the one obtained by coarse-discretisation computational fluid dynamics (CFD) simulation). The knowledge about the physical system of interest contained in the low-fidelity model allows us to construct an accurate representation of the original, high-fidelity CFD model, using a small amount of high-fidelity data and dramatically reduce the overall design cost. Two fundamental issues in such a process are a proper selection of the quality of the low-fidelity model (e.g., the model 'mesh coarseness' that may affect both the optimisation cost and the reliability of the design process), as well as the scaling properties of the surrogate-based design process with respect to the dimensionality of the design space. Our investigations are carried out for specific variable-resolution optimisation methodologies exploiting two types of correction methods: shape-preserving response prediction and space mapping.

Sequential RBF surrogate-based efficient optimization method for engineering design problems with expensive black-box functions

As a promising technique, surrogate-based design and optimization(SBDO) has been widely used in modern engineering design optimizations. Currently, static surrogate-based optimization methods have been successfully applied to expensive optimization problems. However, due to the low efficiency and poor flexibility, static surrogate-based optimization methods are difficult to efficiently solve practical engineering cases. At the aim of enhancing efficiency, a novel surrogate-based efficient optimization method is developed by using sequential radial basis function(SEO-SRBF). Moreover, augmented Lagrangian multiplier method is adopted to solve the problems involving expensive constraints. In order to study the performance of SEO-SRBF, several numerical benchmark functions and engineering problems are solved by SEO-SRBF and other well-known surrogate-based optimization methods including EGO, MPS, and IARSM. The optimal solutions, number of function evaluations, and algorithm execution time are recorded for comparison. The comparison results demonstrate that SEO-SRBF shows satisfactory performance in both optimization efficiency and global convergence capability. The CPU time required for running SEO-SRBF is dramatically less than that of other algorithms. In the torque arm optimization case using FEA simulation, SEO-SRBF further reduces 21% of the material volume compared with the solution from static-RBF subject to the stress constraint. This study provides the efficient strategy to solve expensive constrained optimization problems.

Low-cost EM-simulation-driven Multi-objective Optimization of Antennas

A surrogate-based method for efficient multi-objective antenna optimization is presented. Our technique exploits response surface approximation (RSA) model constructed from sampled low-fidelity antenna model (here, obtained through coarse-discretization EM simulation). The RSA model enables fast determination of the best available trade-offs between conflicting design goals. A low-cost RSA model construction is possible through initial reduction of the design space. Optimization of the RSA model has been carried out using a multi-objective evolutionary algorithm (MOEA). Additional response correction techniques have been subsequently applied to improve selected designs at the high-fidelity EM antenna model level. The refined designs constitute the final Pareto set representation. The proposed approach has been validated using an ultra-wideband (UWB) monocone and a planar Yagi-Uda antenna.

An evaluation of data-driven motion estimation in comparison to the usage of external-surrogates in cardiac SPECT imaging

Motion estimation methods in single photon emission computed tomography (SPECT) can be classified into methods which depend on just the emission data (data-driven), or those that use some other source of information such as an external surrogate. The surrogate-based methods estimate the motion exhibited externally which may not correlate exactly with the movement of organs inside the body. The accuracy of data-driven strategies on the other hand is affected by the type and timing of motion occurrence during acquisition, the source distribution, and various degrading factors such as attenuation, scatter, and system spatial resolution. The goal of this paper is to investigate the performance of two data-driven motion estimation schemes based on the rigid-body registration of projections of motion-transformed source distributions to the acquired projection data for cardiac SPECT studies. Comparison is also made of six intensity based registration metrics to an external surrogate-based method. In the data-driven schemes, a partially reconstructed heart is used as the initial source distribution. The partially-reconstructed heart has inaccuracies due to limited angle artifacts resulting from using only a part of the SPECT projections acquired while the patient maintained the same pose. The performance of different cost functions in quantifying consistency with the SPECT projection data in the data-driven schemes was compared for clinically realistic patient motion occurring as discrete pose changes, one or two times during acquisition. The six intensity-based metrics studied were mean-squared difference, mutual information, normalized mutual information (NMI), pattern intensity (PI), normalized cross-correlation and entropy of the difference. Quantitative and qualitative analysis of the performance is reported using Monte-Carlo simulations of a realistic heart phantom including degradation factors such as attenuation, scatter and system spatial resolution. Further the visual appearance of motion-corrected images using data-driven motion estimates was compared to that obtained using the external motion-tracking system in patient studies. Pattern intensity and normalized mutual information cost functions were observed to have the best performance in terms of lowest average position error and stability with degradation of image quality of the partial reconstruction in simulations. In all patients, the visual quality of PI-based estimation was either significantly better or comparable to NMI-based estimation. Best visual quality was obtained with PI-based estimation in one of the five patient studies, and with external-surrogate based correction in three out of five patients. In the remaining patient study there was little motion and all methods yielded similar visual image quality.

A new optimisation methodology used to study the effect of cover properties on night-time greenhouse climate

This study presents an optimisation method for improving greenhouse design. The approach was based on the surrogate-based method and combines the use of energy balance simulation (ES) and computational fluid dynamics (CFD). The optimisation was applied to study the effects of cover properties on the night-time greenhouse climate. The ES model was verified by comparing its predictions with experimental data. The root mean square error (RMSE) values for internal temperature and humidity showed there was good agreement between the ES and experimental values. The optimisation process highlighted the importance of using a low emissivity and low transmissivity cover material. Under a clear sky the predicted temperature gain by using a high reflectance material compared to a regular one on an unheated greenhouse was approximately 4 °C. The relative humidity in the same scenario was reduced by almost 10%. Two ES parametric studies were carried out, the first showed the effects of different combinations of thermal radiation properties in term of temperature and humidity; the second showed the effects of the high reflectance material under different external conditions (equivalent sky temperature and soil heat flux (SHF)). This paper suggests that ES and CFD can be used together to provide a complete approach to greenhouse modelling.

An efficient surrogate-based method for computing rare failure probability

In this paper, we present an efficient numerical method for evaluating rare failure probability. The method is based on a recently developed surrogate-based method from Li and Xiu [J. Li, D. Xiu, Evaluation of failure probability via surrogate models, J. Comput. Phys. 229 (2010) 8966–8980] for failure probability computation. The method by Li and Xiu is of hybrid nature, in the sense that samples of both the surrogate model and the true physical model are used, and its efficiency gain relies on using only very few samples of the true model. Here we extend the capability of the method to rare probability computation by using the idea of importance sampling (IS). In particular, we employ cross-entropy (CE) method, which is an effective method to determine the biasing distribution in IS. We demonstrate that, by combining with the CE method, a surrogate-based IS algorithm can be constructed and is highly efficient for rare failure probability computation—it incurs much reduced simulation efforts compared to the traditional CE-IS method. In many cases, the new method is capable of capturing failure probability as small as 10 −12 ∼ 10 −6 with only several hundreds samples.

Phase locking between excess spike synchrony and LFP is independent of rate covariation

A possible strategy of information coding employed by cortical networks involves the propagation of activity through synchronously firing groups of neurons, termed assemblies. A hallmark signature of such an assembly coding scheme is the detection of synchronous spiking in parallel recordings of neuronal activity. Experimental work (e.g., [1]) indirectly substantiates the assembly idea with findings of behavioral correlates of significant excess synchronous spiking activity. Recently, we demonstrated [2] that such excess coincident spikes, or Unitary Events [3], also show an exceptionally strong phase-locking to oscillations of the local field potential (LFP), a signal measured directly at the population level. A theoretical model consistently embeds this experimentally observed relationship within the assembly framework. In this context, the weak entrainment of spikes of a single neuron to the LFP is explained as a reflection of the neuron's participation in different assemblies across the network. Co-varying firing rate modulations induced by LFP oscillations were instead suggested as the cause for preferred locking of excess spike coincidences to the LFP phase. In this study we investigate if this alternative hypothesis can be validated. To this end, we first identify the extremal rate changes within those UE analysis windows that are assumed to be expressions of a rate step. We then relate the magnitude of individual rate steps to the degree of rate co-variation across neurons. In a final step, we relate these two measures to phase and amplitude of the LFP. We find that the locking of significant spike synchrony does not preferentially occur with rate changes. As an additional test we extended our previous work by replacing the parametric distribution of coincidences used for evaluating the significance of observed coincidence counts by numerically derived distributions based on surrogate data. The employed surrogate method (spike train dithering, see [4]) considers the statistical features (non-stationarity of rates, interspike interval distribution,...) of the experimental spike trains, while destroying precise spike coincidences. Simulations show that this method is conservative compared to the UE analysis used in earlier work. Despite the decreased sensitivity of the surrogate based method, this additional analysis confirms our previous findings on the relationship between spike synchrony and the LFP, and demonstrates that they are not explained as a consequence of rate covariation. Taken together, these two analyses corroborate our original interpretation of LFP oscillations as a reflection of assembly dynamics.

Use of surrogate modelling for multiobjective optimisation of urban wastewater systems

Simulation models are now available to represent the sewer network, wastewater treatment plant and receiving water as an integrated system. These models can be combined with optimisation methods to improve overall system performance through optimal control. Evolutionary algorithms (EAs) have been proven to be a powerful method in developing optimal control strategies; however, the intensive computational requirement of these methods imposes a limit on their application. This paper explores the potential of surrogate modelling in multiobjective optimisation of urban wastewater systems with a limited number of model simulations. A surrogate based method, ParEGO, is combined with an integrated urban wastewater model to solve real time control problems. This method is compared with the popular NSGA II, by using performance indicators: the hypervolume indicator, additive binary epsilon-indicator and attainment surface. Comparative results show that ParEGO is an efficient and effective method in deriving optimal control strategies for multiple objective control problems with a small number of simulations. It is suggested that ParEGO can greatly improve computational efficiency in the multiobjective optimisation process, particularly for complex urban wastewater systems.

Global Optimization for Integrated Design and Control of Computationally Expensive Process Models

The problem of integrated design and control optimization of process plants is discussed in this paper. We consider it as a nonlinear programming problem subject to differential-algebraic constraints. This class of problems is frequently multimodal and “costly” (i.e., computationally expensive to evaluate). Thus, on the one hand, local optimization techniques usually fail to locate the global solution, and, on the second hand, most global optimization methods require many simulations of the model, resulting in unaffordable computation times. As an alternative, one may consider global optimization methods which employ surrogate-based approaches to reduce computation times and which require no knowledge of the underlying problem structure. A challenging wastewater treatment plant (WWTP) benchmark model1 is used here to evaluate the performance of these techniques. Numerical experiments with different optimization solvers indicate that the proposed benchmark optimization problem is indeed multimodal, and that via global optimization we can achieve an improvement of the controllers' performance compared to the best tuned controllers' settings available in the literature. Moreover, these results show that surrogate-based methods may reduce computation times while ensuring convergence to the best known solutions.

Estimating the distribution of dynamic invariants: illustrated with an application to human photo-plethysmographic time series

Dynamic invariants are often estimated from experimental time series with the aim of differentiating between different physical states in the underlying system. The most popular schemes for estimating dynamic invariants are capable of estimating confidence intervals, however, such confidence intervals do not reflect variability in the underlying dynamics. We propose a surrogate based method to estimate the expected distribution of values under the null hypothesis that the underlying deterministic dynamics are stationary. We demonstrate the application of this method by considering four recordings of human pulse waveforms in differing physiological states and show that correlation dimension and entropy are insufficient to differentiate between these states. In contrast, algorithmic complexity can clearly differentiate between all four rhythms.

Analysis of Invertebrate Biodiversity in a Philippine Farmer's Irrigated Rice Field

Rank-abundance curves, indices of community structure, and surrogate-based methods of biodiversity assessment were applied to invertebrate time-series data collected from a farmer's irrigated rice ( Oryza sativa L.) field on Luzon Island, Philippines. Canopy and floodwater invertebrates were vacuum- and strainer-sampled, respectively, at roughly weekly intervals after rice seedlings were transplanted up to harvest for a total of 8 sampling dates. The cumulative samples included 202 taxa and 9,570 individuals for the plant canopy and 180 taxa and 84,905 individuals for the floodwater. Rank-abundance curves revealed the following findings: (1) a 10,000-fold range in invertebrate abundance with many abundant taxa remaining abundant over all crop stages; (2) lower evenness (equitability) of invertebrate abundances in the floodwater than the plant canopy, due in large part to the numerical dominance (75%) of 2 crustaceans [ Heterocypris luzonensis Neale, Eucyclops serrulatus (Fischer)]; (3) the preponderance of natural enemies as the largest guild in number of taxa in both the canopy and floodwater, followed by herbivores, detritivores, and tourists (nonpredatory taxa with no known interaction except as prey to rice field predators); and (4) herbivore- and detritivore-dominated faunas typified early crop periods in the canopy and floodwater, respectively, followed by predator-dominated faunas in both systems at mid- and late-crop stages. Rates of community turnover generally increased with crop age in both the canopy and floodwater faunas with the former increasing faster than the latter. Of the 82 terrestrial taxa that were tested as surrogates of all canopy taxa, mymarid wasps ( Anagrus sp.) gave the best fit to the data; however, this taxon's sensitivity to single sampling dates made it and the other canopy taxa unsuitable surrogates of all canopy taxa. Conversely, of the 24 floodwater taxa tested, hydrophilid beetles and 4 other taxa (ostracods, gastropods, corixids and chironomids) passed our sensitivity test and were judged as acceptable surrogates for all floodwater taxa in this dry season rice ecosystem. We propose guidelines linking biodiversity concepts (rank-abundance curves, guilds, community metrics) and assessment methods (surrogate taxa) to agroecology for the purpose of enhancing understanding of tropical rice ecosystems for research and training programs.