Surrogate-assisted Optimization Research Articles

A Multi-level Surrogate-assisted Algorithm for Expensive Optimization Problems

With the development of computer science, more and more complex problems rely on the help of computers for solving. When facing the parameter optimization problem of complex models, traditional intelligent optimization algorithms often require multiple iterations on the target problem. It can bring unacceptable costs and resource costs in dealing with these complex problems. In order to solve the parameter optimization of complex problems, in this paper we propose a multi-level surrogate-assisted optimization algorithm (MLSAO). By constructing surrogate models at different levels, the algorithm effectively explores the parameter space, avoiding local optima and enhancing optimization efficiency. The method combines two optimization algorithms, differential evolution (DE) and Downhill simplex method. DE is focused on global level surrogate model optimization. Downhill simplex is concentrated on local level surrogate model update. Random forest and inverse distance weighting (IDW) are constructed for global and local level surrogate model respectively. These methods leverage their respective advantages at different stages of the algorithm. The MLSAO algorithm is evaluated against other state-of-the-art approaches using benchmark functions of varying dimensions. Comprehensive results from the comparisons showcase the superior performance of the MLSAO algorithm in addressing expensive optimization problems. Moreover, we implement the MLSAO algorithm for tuning precipitation parameters in the Community Earth System Model (CESM). The outcomes reveal its effective enhancement of CESM's simulation accuracy for precipitation in the North Indian Ocean and the North Pacific region. These experiments demonstrate that MLSAO can better address parameter optimization problems under complex conditions.

An efficient parameter-retrieval-based surrogate-assisted optimization of on-platform honeycomb absorbing structures

An electromagnetic parameter-retrieval-based surrogate-assisted optimization (PSAO) algorithm is presented to reduce radar cross section (RCS) by optimizing the on-platform honeycomb absorbing structures. To facilitate the optimization process, the honeycomb structure is transformed to an anisotropic homogeneous slab, and the effective parameters of the slab are extracted by the retrieval algorithm. A multi-fidelity model is employed to reduce the computing-time consumption, in which a Gaussian process (GP) regression model is used as the substitute for the coarse model. The GP model establishes a relationship between the geometry of the honeycomb structure and the RCS response of the target coated with the equivalent slab. Finally, the optimization result of the fine model is achieved through a space mapping strategy. The accuracy of the parameter extraction algorithm is verified by analyzing the honeycomb absorbing structure. Subsequently, the proposed optimization method is applied to a metal plate and a metal cylinder, resulting in a 10 dB reduction of RCS in broadband and wide-angle scenarios. This demonstrates the applicability of the proposed PSAO algorithm to both planar and conformal on-platform honeycomb absorbing structures. Furthermore, an NACA0015 foil is analyzed, showing an average RCS reduction of 10 dB and a minimum RCS reduction of 5 dB in the X-band. These results indicate that the PSAO approach can effectively apply to complicated targets. Additionally, the proposed method exhibits significant advantages in terms of computational accuracy and efficiency compared to the traditional genetic algorithm.

Multi-objective constrained robust design of a metamaterial vibration isolator with a limited budget

Many real-world engineering design optimisation problems encounter input uncertainties, which may introduce undesired fluctuations in design performance or even make the design solution infeasible. Robust design optimisation (RDO) is widely studied due to its ability to deal with uncertainties. RDO methods require many evaluations generally, which is prohibitive when relying on expensive simulations. Surrogate-assisted optimisation can help to reduce the number of simulation calls. An improved constrained multi-objective efficient robust global optimisation method (CMO-ERGO) is proposed in this work, which combines the robust expected improvement of ERGO with the probability of feasibility. CMO-ERGO improves the ERGO framework and quantifies the robustness of the objective and the constraint functions using an analytical uncertainty quantification based on the Kriging surrogate. The proposed method is tested on three numerical benchmark functions and applied to the robust design of a metamaterial vibration isolator with a honeycomb structure to explore its performance in solving engineering applications. The robustness of optimal solutions is demonstrated using the Monte Carlo Method. Moreover, comparisons between CMO-ERGO and several existing RDO methods illustrate the effectiveness and efficiency of CMO-ERGO.

Optimal Design of a Surface Permanent Magnet Machine for Electric Power Steering Systems in Electric Vehicle Applications Using a Gaussian Process-Based Approach

The efficient design optimization of electric machines for electric power steering (EPS) applications poses challenges in meeting demanding performance criteria, including high power density, efficiency, and low vibration. Traditional optimization approaches often fail to find a global solution or suffer from excessive computation time. In response to the limitations of traditional approaches, this paper introduces a novel methodology by incorporating a Gaussian process-based adaptive sampling technique into a surrogate-assisted optimization process using a metaheuristic algorithm. Validation on a 72-slot/8-pole interior permanent magnet (IPM) machine demonstrates the superiority of the proposed approach, showcasing improved exploitation–exploration balance, faster convergence, and enhanced repeatability compared to conventional optimization methods. The proposed design process is then applied to two surface PM (SPM) machine configurations with 9-slot/6-pole and 12-slot/10-pole combinations for EPS applications. The results indicate that the 12-slot/10-pole SPM design surpasses the alternative design in torque density, efficiency, cogging torque, torque ripple, and manufacturability.

A hierarchical surrogate assisted optimization algorithm using teaching-learning-based optimization and differential evolution for high-dimensional expensive problems

Surrogate-assisted evolutionary algorithms (SAEAs) are increasingly used in solving computationally expensive optimization problems. However, when tackling high-dimensional expensive problems, a large number of exact function evaluations (FEs) need to be consumed for existing SAEAs to achieve an acceptable solution. In this paper, a hierarchical surrogate assisted optimization algorithm (HSAOA) using teaching-learning-based optimization and differential evolution is proposed for solving high-dimensional expensive problems with a relatively small number of exact FEs. To keep a balance between global exploration and local exploitation, a hierarchical surrogate framework with hybrid evolutionary algorithms is devised. In the global search phase, a radial basis function surrogate is utilized to assist the teaching-learning-based optimization in locating the promising sub-regions. In the local search phase, a novel dynamic ensemble of surrogates is proposed to assist the differential evolution in speeding up the convergence process. Eight test functions with 10 to 100 dimensions and a spatial truss design problem are employed to compare the proposed method with several state-of-the-art SAEAs. The results show that the proposed HSAOA is superior to the comparison algorithms for solving expensive optimization problems, and needs a much smaller number of exact FEs than other competing SAEAs to produce competitive or even better results for high-dimensional expensive problems.

An online surrogate-assisted neighborhood search algorithm based on deep neural network for thermal layout optimization

Thermal layout optimization problems are common in integrated circuit design, where a large number of electronic components are placed on the layout, and a low temperature (i.e., high efficiency) is achieved by optimizing the positions of the electronic components. The operating temperature value of the layout is obtained by measuring the temperature field from the expensive simulation. Based on this, the thermal layout optimization problem can be viewed as an expensive combinatorial optimization problem. In order to reduce the evaluation cost, surrogate models have been widely used to replace the expensive simulations in the optimization process. However, facing the discrete decision space in thermal layout problems, generic surrogate models have large prediction errors, leading to a wrong guidance of the optimization direction. In this work, the layout scheme and its temperature field are represented by images whose relation can be well approximated by a deep neural network. Therefore, we propose an online deep surrogate-assisted optimization algorithm for thermal layout optimization. First, the iterative local search is developed to explore the discrete decision space to generate new layout schemes. Then, we design a deep neural network to build an image-to-image mapping model between the layout and the temperature field as the approximated evaluation. The operating temperature of the layout can be measured by the temperature field predicted by the mapping model. Finally, a segmented fusion model management strategy is proposed to online updates the parameters of the network. The experimental results on three kinds of layout datasets demonstrate the effectiveness of our proposed algorithm, especially when the required computational budget is limited.

Online model tuning in surrogate-assisted optimization — An effective approach considering the cost–benefit tradeoff

Online model tuning in surrogate-assisted optimization — An effective approach considering the cost–benefit tradeoff

Hierarchical power control of a large-scale wind farm by using a data-driven optimization method.

With the participation in automatic generation control (AGC), a large-scale wind farm should distribute the real-time AGC signal to numerous wind turbines (WTs). This easily leads to an expensive computation for a high-quality dispatch scheme, especially considering the wake effect among WTs. To address this problem, a hierarchical power control (HPC) is constructed based on the geographical layout and electrical connection of all the WTs. Firstly, the real-time AGC signal of the whole wind farm is distributed to multiple decoupled groups in proportion of their regulation capacities. Secondly, the AGC signal of each group is distributed to multiple WTs via the data-driven surrogate-assisted optimization, which can dramatically reduce the computation time with a small number of time-consuming objective evaluations. Besides, a high-quality dispatch scheme can be acquired by the efficient local search based on the dynamic surrogate. The effectiveness of the proposed technique is thoroughly verified with different AGC signals under different wind speeds and directions.

An effective surrogate model assisted algorithm for multi-objective optimization: application to wind farm layout design

Due to the intricate and diverse nature of industrial systems, traditional optimization algorithms require a significant amount of time to search for the optimal solution throughout the entire design space, making them unsuitable for meeting practical industrial demands. To address this issue, we propose a novel approach that combines surrogate models with optimization algorithms. Firstly, we introduce the Sparse Gaussian Process regression (SGP) into the surrogate model, proposing the SGP surrogate-assisted optimization method. This approach effectively overcomes the computational expense caused by the large amount of data required in Gaussian Process model. Secondly, we use grid partitioning to divide the optimization problem into multiple regions, and utilize the multi-objective particle swarm optimization algorithm to optimize particles in each region. By combining the advantages of grid partitioning and particle swarm optimization, which overcome the limitations of traditional optimization algorithms in handling multi-objective problems. Lastly, the effectiveness and robustness of the proposed method are verified through three types of 12 test functions and a wind farm layout optimization case study. The results show that the combination of meshing and SGP surrogate enables more accurate identification of optimal solutions, thereby improving the accuracy and speed of the optimization results. Additionally, the method demonstrates its applicability to a variety of complex multi-objective optimization problems.

Surrogate-assisted distributed swarm optimisation for computationally expensive geoscientific models

Evolutionary algorithms provide gradient-free optimisation which is beneficial for models that have difficulty in obtaining gradients; for instance, geoscientific landscape evolution models. However, such models are at times computationally expensive and even distributed swarm-based optimisation with parallel computing struggle. We can incorporate efficient strategies such as surrogate-assisted optimisation to address the challenges; however, implementing inter-process communication for surrogate-based model training is difficult. In this paper, we implement surrogate-based estimation of fitness evaluation in distributed swarm optimisation over a parallel computing architecture. We first test the framework on a set of benchmark optimisation problems and then apply to a geoscientifc model that features landscape evolution model. Our results demonstrate very promising results for benchmark functions and the Badlands landscape evolution model. We obtain a reduction in computationally time while retaining optimisation solution accuracy through the use of surrogates in a parallel computing environment. The major contribution of the paper is in the application of surrogate-based optimisation for geoscientific models which can in the future help in better understanding of paleoclimate and geomorphology.

Surrogate-assisted optimization for anti-ship missile body configuration considering high-velocity water touching

As a crucial weapon in the sea battle, anti-ship missiles generally employ a sea-skimming penetration strategy to reduce the probability of being detected by the target radar, which greatly increases the risk of touching water caused by sensor errors or random sea conditions. To alleviate the large impact load by high-velocity water touching, a novel anti-ship missile body configuration is proposed in this paper, which is inspired by the idea of hydroplaning. A parametric geometry model is first developed to modify the configuration of the anti-ship missile body. Subsequently, a structured arbitrary Lagrange-Eulerian based Fluid-Structure Interaction (FSI) model is established to analyze the kinematics parameters of the missile body during the hydroplaning process. A missile body configuration optimization problem is then formulated to minimize the impact load considering several constraints, e.g., horizontal velocity loss, pitch angle after touching water, and inside capacity for payload. Due to the time-consuming FSI simulation, a Kriging-assisted constrained differential evolution method is utilized to optimize the missile body configuration for reducing the impact load. During the optimization process, radial basis function and Kriging are combined with evolutionary operators to lead the search to the vicinity of the optimum rapidly. The result shows that the proposed missile body configuration can reduce the impact load by 18.8% compared with the ordinary configuration. Additionally, the optimized configuration can further yield a 17.4% impact load decrease subject to all the constraints and avoid structural damage by the high-velocity water touching, which demonstrates the effectiveness and practicability of the proposed anti-ship missile body configuration and corresponding optimization framework for reducing the impact load.

A novel high-dimension shale gas reservoir hydraulic fracture network parameters optimization framework

Optimization design of hydraulic fracture network parameters significantly reduces the high investment risk in the production of shale gas. This is also a numerical simulation-based optimization problem. Surrogate-assisted optimization method which adopts a simple-yet-accurate approximate model to facilitate the optimization workflow is one of the most promising methods to lessen the computational burden. However, due to multi-stage fracturing and “well factory” production model, the dimension of variables in hydraulic fracture network parameters optimization increases sharply, and a significant number of data are required to ensure the performance of the surrogate model. To address this problem, a novel surrogate-assisted multi-objective optimization method using the tri-training strategy and variable-length non-dominated sorting genetic algorithm-II (VL-NSGA-II) is proposed for high-dimension hydraulic fracture network parameters optimization in shale gas reservoir. The proposed method is called HDWHF. In this HDWHF method, the tri-training strategy is adopted to infill solutions with the highest-confidence fitness as new samples to retrain the surrogate model during each iteration while VL-NSGA-II is adopted as the optimizer. This strategy enables the increase of samples without numerical simulation runs, which realizes a decrease of the numerical simulation runs and enhance the performance of the surrogate model. The superior performance of the HDWHF against conventional surrogate-assisted methods is verified using three high-dimension examples with various numbers of wells and fracture types. This proposed HDWHF workflow provides an intelligent means for efficient optimization and decision-making of high-dimension hydraulic fracture network parameters in shale gas reservoirs.

Surrogate-Assisted Automatic Parameter Adaptation Design for Differential Evolution

In this study, parameter adaptation methods for differential evolution are automatically designed using a surrogate approach. In particular, Taylor series are applied to model the searched dependence between the algorithm’s parameters and values, describing the current algorithm state. To find the best-performing adaptation technique, efficient global optimization, a surrogate-assisted optimization technique, is applied. Three parameters are considered: scaling factor, crossover rate and population decrease rate. The learning phase is performed on a set of benchmark problems from the CEC 2017 competition, and the resulting parameter adaptation heuristics are additionally tested on CEC 2022 and SOCO benchmark suites. The results show that the proposed approach is capable of finding efficient adaptation techniques given relatively small computational resources.

Evolutionary Algorithms for Parameter Optimization-Thirty Years Later.

Thirty years, 1993-2023, is a huge time frame in science. We address some major developments in the field of evolutionary algorithms, with applications in parameter optimization, over these 30 years. These include the covariance matrix adaptation evolution strategy and some fast-growing fields such as multimodal optimization, surrogate-assisted optimization, multiobjective optimization, and automated algorithm design. Moreover, we also discuss particle swarm optimization and differential evolution, which did not exist 30 years ago, either. One of the key arguments made in the paper is that we need fewer algorithms, not more, which, however, is the current trend through continuously claiming paradigms from nature that are suggested to be useful as new optimization algorithms. Moreover, we argue that we need proper benchmarking procedures to sort out whether a newly proposed algorithm is useful or not. We also briefly discuss automated algorithm design approaches, including configurable algorithm design frameworks, as the proposed next step toward designing optimization algorithms automatically, rather than by hand.

Data-Driven Surrogate-Assisted Optimization of Metamaterial-Based Filtenna Using Deep Learning

In this work, a computationally efficient method based on data-driven surrogate models is proposed for the design optimization procedure of a Frequency Selective Surface (FSS)-based filtering antenna (Filtenna). A Filtenna acts as a module that simultaneously pre-filters unwanted signals, and enhances the desired signals at the operating frequency. However, due to a typically large number of design variables of FSS unit elements, and their complex interrelations affecting the scattering response, FSS optimization is a challenging task. Herein, a deep-learning-based algorithm, Modified-Multi-Layer-Perceptron (M2LP), is developed to render an accurate behavioral model of the unit cell. Subsequently, the M2LP model is applied to optimize FSS elements being parts of the Filtenna under design. The exemplary device operates at 5 GHz to 7 GHz band. The numerical results demonstrate that the presented approach allows for an almost 90% reduction of the computational cost of the optimization process as compared to direct EM-driven design. At the same time, physical measurements of the fabricated Filtenna prototype corroborate the relevance of the proposed methodology. One of the important advantages of our technique is that the unit cell model can be re-used to design FSS and Filtenna operating various operating bands without incurring any extra computational expenses.

Optimal design of rail level crossings and associated transition zones using adaptive surrogate-assisted optimization

Transition zones such as level crossing and bridge approaches are critical links in railway networks due to higher degradation rates and maintenance needs. In this context, parametric optimization has been applied to improve the design in transition zones; however, it requires a more computationally efficient tool to support repetitive function evaluations, since the involved vehicle–track dynamic simulations are becoming more expensive to evaluate. For this purpose, a surrogate-based simulation methodology is proposed to search for an optimal combination of parameters relevant to the geometry and elasticity of track structures. Specifically, the presented methodology integrates finite element (FE)-based modeling with surrogate-assisted optimization: (1) the FE model is developed to characterize the dynamic behavior of a level crossing under a moving vehicle; (2) the optimization problem is formulated upon this mechanical model by extending the expensive FE simulations to an adaptive surrogate modeling scheme. This integration facilitates efficient exploration of the track design space (thereby reducing the computational cost), and a reasonable balance can be achieved between solution quality and computational effort. The methodology is applied to a Dutch railway case. Results show that compared to a reference design, the optimized design significantly improves performance indicators relevant to wheel–rail contact forces and energy dissipation in the ballast layer. The solution brings great potential in achieving a more desirable vehicle–track interaction and improving the connecting performance between level crossings and transitions. The methodology is applicable to other railway structures and may also contribute to improvements in current track design practices.

Multi-objective global and local Surrogate-Assisted optimization on polymer flooding

Oil production and polymer injection are two performance indicators of polymer flooding and are usually conflicting objectives. In order to obtain optimal trade-off solutions, this paper proposes a multi-objective global and local surrogate-assisted particle swarm optimization (MO-GLSPSO) method, which consists of alternative steps: global population prescreen and local population search.The global steps use generalized regression neural network (GRNN) to prescreen a better population, and the local steps use radial basis function (RBF) as proxy to search for the next generation. The global steps aim to reduce the chance of generations being trapped in local minima, and the local steps obtain the optimal solutions with a fast convergence rate. The rates (liquid production rate and water injection rate) and polymer injection concentration of wells are tuned to obtain a Pareto-front that maximizes cumulative oil production and minimizes cumulative polymer injection.The MO-GLSPSO method is tested using both synthetic and Brugge benchmark cases. The iterations generally improve the oil production or reduce polymer injection and are stabilized at a Pareto-front of the two objectives. Improved sweep efficiency and polymer utility are also observed in the optimal results. The proposed method is also compared with other two methods, multi-objective genetic algorithm (MOGA) and multi-objective particle swarm optimization (MOPSO), to examine the pros and cons. The results indicate that MO-GLSPSO has a better pareto-front than others.

Choose Appropriate Subproblems for Collaborative Modeling in Expensive Multiobjective Optimization.

In dealing with the expensive multiobjective optimization problem, some algorithms convert it into a number of single-objective subproblems for optimization. At each iteration, these algorithms conduct surrogate-assisted optimization on one or multiple subproblems. However, these subproblems may be unnecessary or resolved. Operating on such subproblems can cause server inefficiencies, especially in the case of expensive optimization. To overcome this shortcoming, we propose an adaptive subproblem selection (ASS) strategy to identify the most promising subproblems for further modeling. To better leverage the cross information between the subproblems, we use the collaborative multioutput Gaussian process surrogate to model them jointly. Moreover, the commonly used acquisition functions (also known as infill criteria) are investigated in this article. Our analysis reveals that these acquisition functions may cause severe imbalances between exploitation and exploration in multiobjective optimization scenarios. Consequently, we develop a new acquisition function, namely, adaptive lower confidence bound (ALCB), to cope with it. The experimental results on three different sets of benchmark problems indicate that our proposed algorithm is competitive. Beyond that, we also quantitatively validate the effectiveness of the ASS strategy, the CoMOGP model, and the ALCB acquisition function.

An overall blockage attenuation-based aerodynamic performance and stability design optimization method for transonic axial flow compressors

PurposeThis study aims to use computational fluid dynamics (CFD) to understand and quantify the overall blockage within a transonic axial flow compressor (AFC), and to develop an efficient collaborative design optimization method for compressor aerodynamic performance and stability in conjunction with a surrogate-assisted optimization technique.Design/methodology/approachA quantification method for the overall blockage is developed to integrate the effect of regional blockages on compressor aerodynamic stability and performance. A well-defined overall blockage factor combined with efficiency drives the optimizer to seek the optimum blade designs with both high efficiency and wide-range stability. An adaptive Kriging-based optimization technique is adopted to efficiently search for Pareto front solutions. Steady and unsteady numerical simulations are used for the performance and flow field analysis of the datum and optimum designs.FindingsThe proposed method not only remarkably improves the compressor efficiency but also significantly enhances the compressor operating stability with fewer CFD calls. These achievements are mainly attributed to the improvement of specific flow behaviors oriented by the objectives, including the attenuation of the shock and weakening of the tip leakage flow/shock interaction intensity.Originality/valueCFD-based design optimization of AFC is inherently time-consuming, which becomes even trickier when optimizing aerodynamic stability since the stall margin relies on a complete simulation of the performance curve. The proposed method could be a good solution to the collaborative design optimization of aerodynamic performance and stability for transonic AFC.

Experimental Case Study of Stochastic Surrogate-Assisted Engine Calibration

Reducing experimental calibration cost to achieve optimal system performance is challenging, especially for highly nonlinear engine systems. With the advancement and widespread adoption of machine learning methods for control applications, it is now possible to use an intelligent black-box model to efficiently optimize nonlinear control systems without the detailed knowledge of system dynamics. The surrogate-assisted optimization approach has been recently proven to work effectively to reduce the overall experimental budget (cost). However, its application to calibrate and optimize a practical control system experimentally is a challenge due to stochastic system and measurement noises. This article experimentally calibrates and optimizes the control parameters of a 6.7L Ford diesel engine using the stochastic surrogate-assisted optimization automatically based on a stochastic Kriging model, which is the first step toward automating the engine calibration process. Two engine control variables, namely, exhaust gas recirculation valve and variable geometry turbocharger vane positions, are calibrated to obtain a tradeoff between engine efficiency, in terms of brake-specific fuel consumption, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{NO}_{x}$</tex-math></inline-formula> emissions, utilizing both the deterministic and stochastic surrogate-assisted learning algorithms. Experimental results show a much better representation of the output response surfaces with the stochastic surrogate-assisted optimization than that from the deterministic approach. Note that the stochastic approach helps direct the search toward the global optimal region without wasting the evaluation budget in exploring the local optima, and the deterministic approach cannot effectively handle stochastic system and measurement noises, resulting in overfitting the data. Therefore, to solve a practical optimization problem with noises, stochastic surrogate-assisted optimization is recommended.